We’re hiring a Python-first Data Engineer to partner directly with a business lead working on key projects.
What You’ll Do
- Own end-to-end data pipelines for real-time market data using Python and Google Cloud.
- Productionize the live ticker & public investment tracker (low-latency streaming, resiliency, observability, alerting, SLAs).
- Design and optimize warehouse models (star/snowflake) to support analytics and downstream apps.
- Build reliable batch + streaming jobs (APIs, WebSockets, vendor feeds) with retries, backoff, and idempotency.
- Ship stakeholder-friendly outputs (BigQuery views, materialized tables, and Excel-friendly extracts) with strong data quality checks.
- Implement CI/CD, testing, and IaC for reproducible, secure deployments.
- Collaborate closely with the product owner to scope roadmap, prioritize features, and iterate quickly.
Tech You’ll Use
- Python (pandas, pydantic, SQLAlchemy, FastAPI/Flask), testing (pytest)
- Google Cloud (BigQuery, Cloud Functions/Run, Pub/Sub, Composer/Airflow, Cloud Storage, Secret Manager)
- Data Warehousing (BigQuery preferred; Snowflake/Redshift experience welcome)
- Orchestration & CI/CD (Airflow/Composer, GitHub Actions/Cloud Build, Terraform optional)
- Analytics Outputs: BigQuery, CSV/Parquet, Excel integrations
What We’re Looking For
- Strong Python engineering skills and comfort with SQL and warehouse modeling.
- Experience on GCP (or equivalent cloud) building production data pipelines.
- Pragmatic with Excel outputs and stakeholder deliverables.
- Familiar with real-time/streaming data (nice to have) and market data APIs.
- Obsessed with correctness, latency, and uptime; you measure and improve both.
Who You Are (Fit > Perfect Resume)
- Learner-first: you ramp fast, ask sharp questions, and ship.
- Hustler/Builder: you love turning scrappy wins into durable systems.
- Agile & adaptive: comfortable with evolving requirements and rapid iteration.
- Clear communicator: explain tradeoffs to technical and non-technical partners.
- Ownership mindset: you design, implement, monitor, and improve.
