Sr. Data Engineer
Location: San Diego CA (preferred) or REMOTE
Full-Time Employment with CLIENT
The Role:
As we continue to grow as clients, we are seeking a Senior Data Engineer responsible for designing, developing, and optimizing data pipelines and workflows to support business intelligence and analytics initiatives. With extensive experience in Azure Databricks and Azure Data Factory, the Senior Data Engineer will play a key role in transforming raw data into actionable insights, enabling data-driven decision-making across the organization.
This position is eligible for remote – Preferred location is San Diego, CA
The Position:
- Design, develop, and maintain scalable and reliable data pipelines using Azure Databricks and Azure Data Factory.
- Implement ETL processes to integrate data from various sources into a centralized data warehouse or lake.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions that meet their needs.
- Optimize performance of big data workflows and ensure efficient processing of large datasets.
- Monitor and troubleshoot data pipelines to ensure data quality and consistency.
- Develop and enforce best practices for data engineering, including version control, testing, and deployment.
- Ensure compliance with data governance and security standards.
- Provide mentorship to junior team members and contribute to team knowledge sharing.
- Perform other work-related duties as assigned.
Required:
- Bachelor's degree in Computer Science, Information Technology, or a related field.
- 7 years of experience in data engineering with a strong focus on Azure Databricks and Azure Data Factory.
- Proven experience in designing and implementing big data solutions and cloud-based architectures.
- Expertise in Azure Databricks, Azure Data Factory, and other Azure data services.
- Strong programming skills in Python, SQL, and Spark.
- Familiarity with data modeling, data warehousing, and ETL processes.
- Knowledge of data governance, security, and compliance standards.
- Strong problem-solving skills and ability to troubleshoot complex data workflows.
- Excellent communication and collaboration skills.
- This position is not currently eligible for visa sponsorship.
Preferred:
- Experience with DevOps practices and tools for CI/CD in data engineering workflows is a plus.
