Data Engineer

Procom • Full-time • Wisconsin, United States, US • 5d ago

The Principal Data Engineer will play a pivotal role in shaping our data strategy and infrastructure. The focus of this role is on technology advancement within the data and analytics space and ensuring systems and data solutions are designed and delivered in a manner that is secure, scalable and sustainable. The Principal Data Engineer will serve both as an internal subject matter expert, and as a liaison between various technology departments to develop data models, data ingestion and orchestration patterns, and advise on technology implementation and delivery.

The ideal candidate will have extensive experience with cloud-based data engineering and a deep understanding of data modeling and data warehousing. This role involves coordinating with enterprise and domain engineers to establish and implement cloud service patterns, designing robust OLTP and OLAP data models, and ensuring all solutions adhere to internal cybersecurity guidelines.

YOUR IMPACT

Coach, mentor and train engineers in industry best practices as well as efficient and secure data solutions design patterns.

Works in a collaborative effort with others to design, deliver and support data and cloud-based solutions.

Create value for internal business teams through connecting their strategic needs to a well curated and quality-assured layer of data solutions.

Hands on approach to designing and implementing data solutions

Develops measurement criteria to evaluate programs and projects, reviews estimate on technical aspect of projects.

Participate in the planning, designing, and implementing of a data strategy as well as related implementation approach.

Drives direction for the long-term scalability, efficiency and sustainability of data platforms and generates buy in from stakeholders.

Other duties as assigned.

KEY RESPONSIBILITIES

Collaborate with enterprise and domain engineers and architects to establish standardized patterns for cloud services within Microsoft Azure.

Utilize infrastructure-as-code tools, such as Terraform, to design, deploy, and manage scalable cloud infrastructure.

Develop and maintain reusable cloud service patterns that are accessible and user-friendly for internal teams focusing on delivering machine learning and AI-based analytics products.

Design logical and physical data models for OLTP and OLAP systems using ER Studio and/or other data modeling tools.

Assist in defining and enforcing data architecture standards, guidelines, and best practices across the organization.

Ensure data models support business requirements, performance objectives, and scalability needs.

Oversee the integration of new data technologies and tools into existing systems.

Ensure all data solutions and cloud service patterns comply with internal cybersecurity policies and standards.

Collaborate on security assessments and audits and implement recommendations to mitigate risks.

MINIMUM QUALIFICATIONS

Bachelors degree with ten (10) or more years of experience in the field or in a related area

Experience providing technical leadership/guidance and influencing other technology teams

4+ years of experience with Python in an analytics / data science environment, including source control systems (Git, SVN, or others).

PREFERRED QUALIFICATIONS

Excellent communication and presentation skills

Experience with CDC-based data ingestion solutions (Qilk Replicate, Striim, Debeezium, etc..)

Familiarity with open-source data transformation tools such as dbt.

Proven professional experience in data architecture and cloud data engineering.

Deep understanding of Microsoft Azure services and cloud architecture.

Experience with the Microsoft Azure Gov Cloud (GCC High) stack

Experience with Databricks and Data Lake Architecture, as well as working with delta format and the Hive metastore.

Experience with orchestration platforms such as Airlfow, Dagster, Luigi and others, as well as shell scripting for automation and process observability.

Familiarity with data governance, data quality, and data security best practices.

Master’s degree in Analytics, Computer Science, Engineering or MBA