Senior Data Engineer

Agile Engine
Austin, TX
Category Engineering
Job Description
Role Overview

We are looking for a Senior Data Engineer to design and build scalable data lakes, warehouses, and lakehouse architectures supporting a thematic research platform that processes large volumes of financial data daily.

What You Will Do

- Design and implement Python Data Engineering solutions; - Design and build scalable Data Lakes, Data Warehouses, and Data Lakehouses; - Design and implement robust ETL/ELT processes at scale using Python, incorporating modern pipeline orchestration tools like Airflow;

Why It Might Be a Fit

If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you!

Requirements

  • Authorized to work for ANY employer in the US
  • Bachelor’s degree in computer science/engineering or other technical field, or equivalent experience
  • 5+ years of experience with Python
  • 5+ years of experience with data processing, manipulation, and analytics libraries like Pandas, Polars, PySpark or DuckDB
  • 2+ years of experience with Big Data technologies (Spark, Snowflake)
  • Expert-level knowledge of pipeline orchestration using Airflow or similar industry-standard tools
  • Deep understanding of Medallion Architecture, columnar file formats, and diverse database technologies (SQL, NoSQL, and Lakehouse architectures)
  • Proven ability to work with 3rd party APIs for complex data ingestion tasks
  • Proficiency with modern Cloud platforms (AWS, GCP, Snowflake) and advanced SQL optimization
  • Exceptional soft skills with a proven ability to gather requirements from leadership and collaborate effectively across cross-functional teams
  • Excellence in optimizing complex data pipelines and troubleshooting data latency or consistency issues in massive datasets
  • Upper-intermediate English level

Benefits

  • Professional growth: Mentorship, TechTalks, and personalized growth roadmaps
  • Competitive compensation: USD-based pay with education, fitness, and team activity budgets
  • Exciting projects: Modern solutions with Fortune 500 and top product companies
  • Flextime: Flexible schedule with remote and office options
]]>