Never Miss a New Opportunity

Subscribe and get the latest jobs directly to your inbox

Get a

email of new

jobs

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Data Engineer

Centraprise • Contract • Dallas, TX, US • 1d ago

Databricks Engineer

Plano, TX (Day 1 onsite)

12+ months contract

Job Description:

Required Skills & Qualifications:

Strong hands-on experience with Databricks, Apache Spark, PySpark, and Spark SQL.
Proficiency in Python, SQL, and data modeling concepts.
Experience working with Azure Data Lake Storage (ADLS), Azure Synapse, Azure Data Factory, or equivalent cloud services (AWS/GCP).
Solid understanding of ETL/ELT design patterns, data warehousing, and data lakehouse architectures.
Experience with Delta Lake, Unity Catalog, and MLflow is a plus.
Knowledge of CI/CD pipelines and infrastructure-as-code (IaC) tools such as Terraform or ARM templates.
Familiarity with Git-based version control and Agile delivery methodologies.
Excellent problem-solving, debugging, and performance tuning skills.

Key Responsibilities:

Design, build, and maintain data pipelines and ETL/ELT workflows using Databricks (Spark, PySpark, SQL, Delta Lake).
Develop scalable data ingestion, cleansing, and transformation processes from various structured and unstructured data sources.
Implement and optimize Delta Lake for data versioning, reliability, and performance.
Work closely with data architects, data scientists, and BI teams to deliver high-quality, production-grade datasets.
Tune and optimize Spark clusters and jobs for performance and cost efficiency.
Integrate Databricks with Azure Data Lake Storage (ADLS), Azure Synapse, Event Hubs, and other Azure services.
Manage job scheduling, orchestration, and monitoring (e.g., using Azure Data Factory, Airflow, or Databricks Workflows).
Implement data quality checks, logging, and error handling mechanisms.
Follow DevOps and CI/CD practices for data engineering projects (e.g., Git, Azure DevOps, Jenkins).
Ensure compliance with data governance, security, and privacy standards.