Job Title: Data Engineer
Job Location: NYC (Onsite)
Job Summary:
We are seeking a skilled Data Engineer to design, develop, and maintain data pipelines on our modern data platform. The ideal candidate will have hands-on experience with Azure Data Lakehouse, Databricks, and Python/SQL, and will collaborate with analytics, engineering, and business teams to deliver clean, ready-to-use datasets for reporting and analysis.
Key Responsibilities:
- Build, maintain, and optimize data pipelines using Python and SQL.
- Develop and manage workflows in Databricks and migrate existing pipelines from Azure Data Factory.
- Load structured and semi-structured data into the Data Lakehouse (Delta Lake).
- Write reusable, well-documented data transformations in Databricks.
- Monitor pipeline performance, troubleshoot issues, and ensure SLAs are met.
- Collaborate with analytics, engineering, and business teams to deliver high-quality datasets.
- Optimize Databricks views for efficient BI tool consumption (e.g., Power BI).
- Manage metadata and maintain proper documentation for datasets and pipelines.
- Participate in discussions regarding data platform architecture and improvements.
Required Qualifications:
- Bachelor’s degree in Computer Science, Engineering, IT, or related field.
- 5+ years of experience in data engineering or equivalent.
- Strong knowledge of databases, data modeling (star/snowflake), and data engineering best practices.
- Hands-on experience with Azure Data Lake Storage, Databricks, Unity Catalogs, GitHub, and Azure ecosystem tools.
- Proficiency in Python, SQL, R, or Scala.
- Experience in building ELT pipelines and working with modern data stacks.
- Understanding of Data Lakehouse, Delta Lake, and columnar storage.
- Good project management skills; able to prioritize tasks and deliver improvements on time.
- Experience with data governance, dbt, or CI/CD environments is a plus.
- Knowledge of the construction industry is a bonus.
- Strong analytical, problem-solving, and teamwork skills.
- Clear communication skills, able to explain technical concepts to non-technical audiences.
- Proactive and focused on process improvements and learning new technologies.
- Proficient in Microsoft Office and collaboration tools.
- Limited travel required.
