Azure Data Engineer

Kanak Elite Services • Full-time • Weehawken, NJ, US • 19h ago

Role: AZURE DATA ENGINEER

Location: Weehawken, NJ - 5 days onsite (Need Local Candidates from NY, NJ)

Client: Infosys/UBS

Need PP no. as well !!

Duration: 12 Months

Experience: Need 10+ Years experience

Job Details

We are seeking a skilled Azure Data Factory Engineer/Developer with strong PostgreSQL knowledge to design, develop, and maintain data pipelines for extracting, transforming, and loading (ETL) data from various sources, including PostgreSQL databases, into Azure data platforms

Design, Develop, and Maintain ADF Pipelines:
- Design, develop, and maintain robust and efficient ADF pipelines for data ingestion, transformation, and loading from PostgreSQL databases and other sources.
- Optimize ADF pipelines for performance, scalability, and reliability.
- Develop and implement data quality checks and validation rules within ADF pipelines.
PostgreSQL Expertise:
- Proficiently use SQL to query, manipulate, and extract data from PostgreSQL databases.
- Understand PostgreSQL database architecture, performance tuning, and best practices.
- Troubleshoot and resolve issues related to PostgreSQL connectivity, performance, and data integrity.
Data Integration and Transformation:
- Design and implement data transformation logic within ADF pipelines, leveraging various ADF activities and connectors.
- Develop and maintain data mappings and transformations to ensure data consistency and quality.
- Collaborate with data engineers and business stakeholders to understand data requirements and implement solutions.
Monitoring and Optimization:
- Monitor the performance and health of ADF pipelines and PostgreSQL databases.
- Identify and resolve performance bottlenecks and data quality issues.
- Implement and maintain monitoring and alerting mechanisms for ADF pipelines and PostgreSQL databases.
Skills:
Azure Data Factory (ADF): Strong experience in designing, developing, and maintaining ADF pipelines.
PostgreSQL: Extensive knowledge of PostgreSQL database architecture, SQL, and performance tuning.
SQL: Proficient in writing and optimizing SQL queries.
Data Modeling: Experience in designing and implementing data models.
ETL/Data Integration: Experience with ETL processes and data integration tools.
Cloud Computing: Familiarity with Azure cloud services and technologies.
Problem-Solving: Strong analytical and problem-solving skills.
Communication: Excellent communication and collaboration skills.