Senior Data Engineer
Northern NJ (onsite 1-3 days/week)
Ref#833
A ratings company is seeking a Senior Data Engineer with expertise in DataOps to ensure dependable, scalable and effective data processes and solutions. The ideal candidate will combine components, functions and skills from data engineering, software engineering and IT Operations. This role is critical to the firm’s goal of developing, maintaining and scaling a sophisticated platform for data and analytics.
Responsibilities include:
- Build high performance data systems including databases, APIs, and data integration pipelines
- Implement a metadata-driven architecture and infrastructure as code approach to automate and simplify the design, deployment, and management of data systems
- Facilitate the adoption of data and software engineering best practices, including code review, testing, and continuous integration and delivery (CI/CD)
- Develop and establish a data governance framework
- Continuously monitor process performance and implement improvements for efficiency including fine-tuning existing ETL processes, optimizing queries, or refactoring code
- Assess and make optimal use of cloud platforms and technologies, especially Azure and Databricks to enhance system architecture
- Implement data quality checks and build processes to identify and resolve data issues
- Create and maintain documentation for data architecture, standards, and best practices
- Identify opportunities for process optimization and automation
- Provide technical leadership to the data engineering team and actively lead design discussions
- Ensure that the new data infrastructure remains modern and efficient by staying knowledgeable of the latest tools, technologies, and methodologies in the data and analytics space
Qualifications include:
- 7+ years as a Data Engineer, Data Architect, Database Developer or similar roles
- Bachelor's in Computer Science, Data Science, Engineering or related field
- 7+ years of experience:
- Programming languages: Python, Scala, Java, Rust or similar
- Database design and development: Oracle OLTP
- ETL / ELT data pipelines: Apache Spark, Airflow, dbt or similar
- APIs: REST, SOAP, RPC, GraphQL or similar
- OLAP / data warehouse design /development: dimensional (Kimball star-schema), data vault or Inmon
- methodologies
- 5 years of experience:
- NoSQL database design and development: preferably on MongoDB or similar
- Message broker platforms: RabbitMQ, Apache Kafka, Red Hat AMQ or similar
- Azure or AWS cloud-native technologies/services
- Working knowledge of:
- CI/CD automation: Jenkins, Ansible, Chef, Puppet or similar
- Containerized platform: Docker, Kubernetes or similar
- Prefer experience with/knowledge of:
- Databricks data intelligence platform
- Business intelligence tools / platforms: PowerBI, Cognos, Tableau or similar
- Caching databases: Memcache, Redis or similar
- Search databases and platforms: Elasticsearch, Apache Solr or similar
- Data modeling tools: ERwin, ERStudio, Navicat, SqlDBM or similar
- Governance tools: Colibra, Alation or similar
- Master data management tools: Informatica, TIBCO, Riversand, Ataccama or similar
Compensation commensurate with experience, targeting $140-150K base, bonus eligible. Great benefits, including flexible and hybrid work arrangements, Paid Time Off/Paid company holidays, Medical plan options/prescription drug plan, Dental plan/vision plan options, Flexible spending and health savings accounts, 401(k) retirement savings plan with a Roth savings option and company matching contributions, and educational assistance program.
Must be authorized to work in the U.S. (citizen or permanent resident).
Third party candidates from vendors will not be considered for this position. Applicants are considered for positions without regard to race, religion, gender, native origin, age, disability, or any other category protected by applicable federal, state, or local laws.
