Job Title - Data Engineer w/ Big Data & AWS exp
Location -onsite
Job Summary
We are seeking a Sr. Developer with 5 to 9 years of experience to join our team. The ideal candidate will have expertise in Spark in Scala Spark and SparkSQL. This role involves working in a hybrid model with a focus on developing and optimizing data processing solutions. Experience in the Finance & Accounting domain is a plus. The position operates during the day shift with no travel requirements.
Responsibilities
-
Develop and optimize scalable data processing solutions using Spark in Scala to enhance data-driven decision-making processes.
-
Collaborate with cross-functional teams to design and implement efficient data pipelines that meet business requirements.
-
Analyze complex data sets and provide insights to improve business operations and strategies.
-
Ensure data quality and integrity by implementing robust data validation and cleansing processes.
-
Troubleshoot and resolve technical issues related to Spark applications to maintain system performance and reliability.
-
Stay updated with the latest industry trends and technologies to continuously improve data processing capabilities.
-
Document technical specifications and processes to ensure knowledge sharing and continuity.
-
Work closely with stakeholders to gather requirements and translate them into technical solutions that drive business value.
-
Optimize SparkSQL queries for performance and efficiency to support real-time data analytics.
-
Provide technical guidance and mentorship to junior developers to foster a collaborative and innovative work environment.
-
Participate in code reviews to ensure adherence to best practices and coding standards.
-
Contribute to the development of data governance policies to ensure compliance with industry regulations.
-
Engage in continuous learning and professional development to enhance technical skills and domain knowledge.
Qualifications
-
Possess strong expertise in Spark in Scala Spark and SparkSQL with a proven track record of successful project delivery.
-
Demonstrate experience in developing and optimizing data processing solutions in a hybrid work model.
-
Have a solid understanding of data warehousing concepts and data modeling techniques.
-
Experience in the Finance & Accounting domain is a plus providing valuable insights into industry-specific challenges.
-
Exhibit excellent problem-solving skills and the ability to troubleshoot complex technical issues.
-
Show proficiency in working with cross-functional teams and effectively communicating technical concepts to non-technical stakeholders.
-
Display a commitment to continuous learning and staying updated with the latest industry trends and technologies.
Certifications Required
Certified Spark Developer Apache Spark Certification
