The Intersect Group is partnered with our client seeking a Data Engineer.

Direct Hire (No C2C or third-party submissions)

Location: Fully Remote (Central Standard Time based)

Occasional travel may be required.
Flexible work hours may be necessary to meet project deadlines.

Interview Process: Typically 3 rounds of virtual interviews

We’re looking for a highly skilled, execution-driven Data Engineer to join our growing Data & Analytics team. In this role, you’ll be responsible for designing, building, and maintaining scalable data pipelines on the Databricks Lakehouse Platform, enabling reliable data delivery across the organization.

The ideal candidate brings strong expertise in Python, PySpark, SQL, and medallion architecture, with a proven track record of deploying production-grade pipelines in enterprise environments. You’ll work closely with the Sr. Data Architect, collaborating with business analysts, BI developers, solution architects, and offshore engineering teams to meet evolving data needs.

We’re seeking someone who is not only technically proficient but also curious and adaptable—ready to grow with us as we expand our platform to include generative AI, agent-based automation, and intelligent orchestration frameworks.

Needs:

Bachelor’s degree in Computer Science, Engineering, or a related technical field
3+ years of hands-on experience in data engineering, with proven production use of Databricks, Python, PySpark, and SQL
Strong knowledge of Delta Lake, Databricks DLT pipelines, and medallion architecture principles
Experience with Azure DevOps or GitHub for version control and CI/CD automation
Demonstrated ability to collaborate with and support offshore engineering teams, ensuring accountability and delivery quality
Interest or familiarity with Generative AI, agent-based tooling, and modern orchestration/ML workflows
Databricks certification (Associate Data Engineer or higher) is preferred
Experience working in Azure-based data environments is a plus
Excellent documentation and communication skills

Other skills:

Meticulous attention to detail with a commitment to delivering high-quality outcomes
Proven ability to thrive in hybrid onshore/offshore team environments
Strong organizational skills with a sense of ownership over tasks and deliverables
Self-driven and enthusiastic about learning new tools, technologies, and frameworks
Comfortable working under tight deadlines in a dynamic, fast-paced setting

Duties:

Design, build, and maintain ELT pipelines in Databricks using Delta Live Tables, Python, PySpark, and SQL, following the medallion architecture (Bronze → Silver → Gold)
Package and deploy reusable pipeline components using Databricks Asset Bundles (DABs)
Manage source control and CI/CD workflows across environments using Azure DevOps or GitHub
Conduct code reviews and oversee offshore data engineering deliverables to ensure consistency, quality, and proper documentation
Actively participate in sprint planning, design reviews, and proactively identify and resolve data issues
Continuously explore and evaluate emerging technologies—including LLMs, Generative AI, and agent-based workflows—to enhance and automate data delivery
Support the development and enforcement of data modeling, lineage, and governance standards using Unity Catalog and related tools
Maintain clear, comprehensive documentation and foster effective communication across technical and business teams

The Intersect Group

Latest Jobs

Sr. Business Analyst (Insurance - Claims & Underwriting)

Data Engineer

Data Engineer