Azure Databricks Data Engineer

Position Overview

Contract Duration: 6 months
Location: Remote or Hybrid (Kansas City area)
Rate: Hourly (contract position)
Work Authorization: Must be legally authorized to work in the US without sponsorship

About the Project

Join CFA’s initiative to extend our industry renowned lending platform through Azure Databricks-powered data integration. You’ll architect and integrate with a centralized data lake that unifies disparate systems (Salesforce, loan servicing platforms, document management, DocuSign, Conga) into a single source of truth for agricultural finance operations.

Core Responsibilities

Design and implement Azure Databricks data lake architecture integrating multiple enterprise data sources
Build real-time streaming and batch data processing pipelines for Salesforce, loan servicing, and document management systems
Develop data quality, validation, and cleansing processes to ensure data integrity across integrated systems
Create analytics-ready data structures optimized for business intelligence and operational reporting
Implement data governance, security controls, and compliance measures for SOC 2 Type II requirements
Collaborate with API/service layer team to expose unified data through REST and GraphQL endpoints

Required Technical Skills

Expert-level Azure Databricks experience with Delta Lake, Spark SQL, and PySpark
Azure Cloud Platform: Data Factory, Event Hubs, Blob Storage, Key Vault, Azure AD integration
Data Engineering: ETL/ELT pipeline design, data modeling (dimensional/star schema), data warehouse patterns
Programming: Python (primary), SQL, Scala (plus)
Data Integration: Experience with Salesforce APIs, REST/SOAP integrations, document system connectors
Streaming & Batch: Real-time data ingestion patterns, scheduled synchronization strategies
Security & Compliance: Encryption at rest/in transit (AES-256, TLS 1.3), RBAC, data classification

Preferred Qualifications

Experience integrating loan servicing or financial systems data
Knowledge of document management systems (DocuSign, Conga) and email archival
Familiarity with GraphQL data exposure patterns
Agricultural finance or fintech domain experience
Experience with SOC 2 or similar compliance frameworks

Technical Environment

Cloud: Microsoft Azure (Databricks, Data Factory, Blob Storage, Key Vault)
Tools: Git/GitHub, Azure DevOps/GitHub Actions, Docker, Azure CLI
Data Sources: Salesforce, NLS loan servicing, email archives, DocuSign, Conga
Backend Stack: Node.js/Python APIs, PostgreSQL, Redis
Frontend Stack: React 18+, TypeScript (awareness helpful for API integration)

Success Criteria

Operational data lake with all source systems integrated and synchronized
Data quality processes achieving >95% accuracy and completeness
Real-time pipelines with <5 minute latency for critical data streams
Documentation enabling knowledge transfer and long-term maintainability
Security controls meeting SOC 2 Type II requirements

How to Apply

Submit your resume to cfacareers@cfafs.com with your git portfolio or examples of Azure Databricks projects showing multi-source data integration, real-time and batch pipeline implementations at scale and any security/compliance work (SOC 2, GDPR, etc.) you want to showcase.

CFA is an Equal Opportunity Employer

Job Type: Contract

Pay: $55.00 - $66.00 per hour

Expected hours: 40 per week

Work Location: Remote

The Cooperative Finance Association, Inc

Latest Jobs

Azure Databricks Data Engineer