Job title: Big Data Engineer
Location: Chandler, AZ
Duration: 12+ Months
Tax terms: W2

Job Overview:

You will be responsible for designing, building, and maintaining scalable data pipelines, applying data transformation logic, and troubleshooting issues to ensure reliable data flows. This role requires proactive problem-solving, accountability, and the ability to work with a modern big-data technology stack in a fast-paced environment.

Candidates needs to have:

4+ years of hands-on Software Engineering or Data Engineering experience.
Big-Data Stack Hadoop Hive pySpark python - Amazon AWS S3, Autosys - PowerBI Dremio - Unix/shell scripting CICD pipeline, GCP cloud, MySQL

Strong experience with:

Big Data Stack: Hadoop, Hive, PySpark, Python
AWS S3: Object storage, security, and integration
Data Modeling & Database Design (MySQL or equivalent)
Job Scheduling: Autosys
Unix/Shell scripting
CI/CD pipelines
Data Visualization: Power BI, Dremio
Ability to proactively identify issues, troubleshoot, and drive solutions.
Strong problem-solving and communication skills with accountability for deliverables.

Job Description:

Participate in moderately complex software engineering projects, ensuring compliance with policies, procedures, and requirements.
Design and build data pipelines using big-data frameworks (Hadoop, Hive, PySpark, Python).
Implement object storage integration and security with Amazon AWS S3.
Develop data models and perform database design (MySQL or equivalent).
Automate workflows using Autosys and Spark/Python.
Support data visualization and reporting through Power BI and Dremio.
Write and maintain Unix/Shell scripts to support pipeline operations.
Integrate and support CI/CD pipelines for data engineering workflows.
Troubleshoot and optimize data pipelines for performance and reliability.
Collaborate with engineers and stakeholders to meet project deliverables.
(Nice to Have) Gain exposure to GCP cloud data engineering.
(Nice to Have) Apply domain knowledge from financial services (not required).

TrekRecruit LLC

Latest Jobs