Databricks Data Engineer Job at Darwin Recruitment, San Francisco, CA

VUVUUnp0WVdUQWRySFJQcSsrc1VxNFpsWEE9PQ==
  • Darwin Recruitment
  • San Francisco, CA

Job Description

Location: San Francisco, CA (On-Site) Duration: Contract (6 months, with potential extensions) About the Role: We are seeking an experienced Databricks Data Engineer to join our dynamic team in San Francisco. This role is ideal for someone passionate about data engineering, hands-on in building scalable solutions, and comfortable working in a fast-paced environment. Prior experience supporting supply chain operations and analytics is highly desirable. As a Data Engineer, you will be responsible for designing, building, and optimizing data pipelines and workflows in Databricks , supporting critical business initiatives. You will collaborate closely with data scientists, analysts, and business stakeholders to ensure high data quality, availability, and performance. Key Responsibilities: Develop, optimize, and maintain large-scale data pipelines in Databricks and Apache Spark . Work closely with business teams to understand requirements and translate them into technical solutions. Build robust, scalable ETL processes to ingest, transform, and load data from various sources. Design and implement data models to support advanced analytics and reporting. Ensure best practices around data governance, quality, and security are followed. Monitor, troubleshoot, and enhance existing data workflows. Collaborate with the supply chain, operations, and IT teams to support end-to-end data initiatives. Document technical designs, processes, and tools for ongoing operations and maintenance. Qualifications: 3-5+ years of hands-on experience as a Data Engineer. Strong expertise in Databricks (Delta Lake, Databricks Workflows, Unity Catalog). Proficiency in Apache Spark , SQL , and Python . Solid understanding of ETL/ELT development, big data processing, and cloud data architectures (AWS, Azure, or GCP). Experience working with complex, large-scale datasets. Exposure to supply chain data (inventory, logistics, procurement, fulfillment) strongly preferred. Familiarity with DevOps practices for data pipelines (CI/CD for data). Excellent problem-solving and communication skills. Ability to work independently on-site in San Francisco. Nice to Have: Experience with MLflow, Databricks SQL Warehouses. Familiarity with Airflow, dbt, or similar orchestration and transformation tools. Knowledge of data governance frameworks. Details: Location: Must work in-office in San Francisco, CA ; no remote option. Compensation: Competitive hourly rate, commensurate with experience. No sponsorship available for this role; candidates must be authorized to work in the U.S. Darwin Recruitment is acting as an Employment Business in relation to this vacancy. Ryan Corbett #J-18808-Ljbffr Darwin Recruitment

Job Tags

Hourly pay, Contract work, Remote job,

Similar Jobs

The Summers Agency

Life Insurance Agent (Remote/Entry Level) Job at The Summers Agency

 ...motivated individuals to join our team as Remote Entry-Level Insurance Agents. This role offers the flexibility to work from home, enabling...  ...a balanced lifestyle while pursuing a rewarding career in the life and health insurance sector. What We Do: At The Summers Agency... 

Medtigo

Neurologist BE/BC- Neurologist BE/BC- Erie, Pennsylvania Job at Medtigo

 ...physician to practice with UPMC Northshore Neurology at UPMC Hamot in Erie, PA. It's your opportunity to live in an affordable city on the water, in a family-friendly community with short commutes to the hospital from many surrounding neighborhoods! UPMC Northshore Neurology... 

Greater Public

Part-Time Radio Announcer Job at Greater Public

 ...shift for Nebraska Educational Telecommunications; performs related work as assigned. If you have a strong voice, an eagerness to learn,...  ...class utilizes professional judgment to determine overall content and effectiveness of radio programming and may supervise Radio Announcers... 

SFM Companies (SFM Mutual Insurance)

Insurance Support Representative I Job at SFM Companies (SFM Mutual Insurance)

 ...Insurance Support Reps are the backbone of SFMs overall operation and a gateway to a rewarding career at SFM! In this role, you will provide...  ...to internal and external customers. You will assist the entire company handling and processing correspondence, insurance applications... 

Southern 441 Toyota

Toyota Master Technician Job at Southern 441 Toyota

 ...you say Steakhouses? Seafood? Bakeries? * Again. The beaches!! But also... The lakes, the pools, the waterparks!* Fishing!! Boating!! Amazing fishing charters! Boat Rentals! Minutes to the Ocean!* Minutes away from downtown West Palm Beach and amazing...