Job Summary

We are seeking a highly skilled Data Developer to join our Intellectual Property Technical Software Division, where you will leverage your expertise in data engineering and automation to design, build, and maintain efficient and scalable data pipelines. The ideal candidate will have deep knowledge of data extraction, transformation, and loading (ETL) processes, as well as experience with distributed computing environments to support our data-driven initiatives.

Responsibilities

Data Extraction & Ingestion:

Develop and manage ETL pipelines using tools like Apache Beam, AWS Glue, or Google Dataflow to extract data from various internet sources.
Ensure reliable and efficient data ingestion processes across diverse data feeds, including web scraping, APIs, and real-time streams.

Data Transformation & Processing:

Design and implement data transformation logic to clean, normalize, and enrich raw data for optimal use in business intelligence (BI) and operational dashboards.
Utilize SQL and NoSQL databases (e.g., MySQL, PostgreSQL, MongoDB) to store and manage structured and unstructured data effectively.

Data Warehousing & Storage:

Build and maintain scalable data warehouses using platforms like AWS Redshift, Google BigQuery, or Snowflake to organize and store large datasets.
Optimize storage solutions to ensure efficient querying and retrieval of data for BI tools and operational reporting.

Automated Data Pipelines:

Automate ETL processes with CI/CD pipelines using tools such as Jenkins, GitLab CI, or GitHub Actions to streamline data flow and reduce manual intervention.
Integrate cloud platforms (AWS, Azure, GCP) to leverage their native services for scalable and secure data processing.

Monitoring & Maintenance:

Implement monitoring tools like Prometheus, Grafana, or Datadog to track data pipeline performance and identify issues proactively.
Troubleshoot production issues related to data pipelines and ensure high availability and resilience of the data infrastructure.

Documentation & Collaboration:

Maintain comprehensive documentation for data pipelines, system architecture, and workflows to facilitate transparency and knowledge sharing within the team.
Collaborate closely with development and operations teams to optimize infrastructure, implement infrastructure-as-code practices (e.g., Terraform, Ansible), and ensure compliance with security and performance standards.

Data Quality & Governance:

Ensure data integrity by implementing quality checks and validation processes throughout the data pipeline lifecycle.
Develop and enforce data governance policies to maintain consistency, accuracy, and reliability of business intelligence datasets.

Experience

Proficiency in data engineering tools such as Apache Beam, AWS Glue, or Google Dataflow for data extraction and transformation.
Experience with cloud-native data platforms (AWS Redshift, Google BigQuery, Snowflake) for data warehousing and storage.
Strong knowledge of SQL and NoSQL databases to manage structured and unstructured data effectively.
Familiarity with CI/CD automation tools like Jenkins, GitLab CI, or GitHub Actions to streamline development and deployment processes.
Proficiency in distributed computing frameworks (e.g., Apache Spark) for handling large-scale data processing tasks.
Strong problem-solving skills with a focus on system architecture, scalability, and operational efficiency.
Experience in automating data pipelines to support business intelligence and operational reporting.

Who Would Thrive

Innovators in the digital landscape! Your expertise will play a crucial role in shaping the future of our web applications.
Current Employees of companies like Meta, AWS, Netflix, Google, Microsoft, Apple.

Job Type: Full-time

Pay: $180,000.00 - $238,000.00 per year

People with a criminal record are encouraged to apply

Work Location: Remote

Quality-Certified-Brand™ | Commerce jobs

Software Senior Developer Intellectual Property Data Engineer ( E5 || L5 )

Never Miss a New Opportunity