Senior Data Engineer

Agile Engine
Blacksburg, VA
Job Description
Role Overview

We are looking for a Senior Data Engineer to design and build scalable data lakes, warehouses, and lakehouse architectures supporting a thematic research platform that processes large volumes of financial data daily.

What You Will Do

- Design and implement Python-based ETL/ELT pipelines, orchestrate workflows with Airflow, develop ingestion workflows from third-party APIs, and work with Snowflake, Spark, and AWS to deliver high-performance data infrastructure.

Why It Might Be a Fit

The role combines hands-on engineering with technical consulting responsibilities, translating business goals into data architecture roadmaps.

Requirements

  • Authorized to work for ANY employer in the US
  • Bachelor’s degree in computer science/engineering or other technical field, or equivalent experience
  • 5+ years of experience with Python
  • 5+ years of experience with data processing, manipulation, and analytics libraries like Pandas, Polars, PySpark or DuckDB
  • 2+ years of experience with Big Data technologies (Spark, Snowflake)
  • Expert-level knowledge of pipeline orchestration using Airflow or similar industry-standard tools
  • Deep understanding of Medallion Architecture, columnar file formats, and diverse database technologies (SQL, NoSQL, and Lakehouse architectures)
  • Proven ability to work with 3rd party APIs for complex data ingestion tasks
  • Proficiency with modern Cloud platforms (AWS, GCP, Snowflake) and advanced SQL optimization
  • Exceptional soft skills with a proven ability to gather requirements from leadership and collaborate effectively across cross-functional teams
  • Upper-intermediate English level

Benefits

  • Professional growth: Mentorship, TechTalks, and personalized growth roadmaps
  • Competitive compensation: USD-based pay with education, fitness, and team activity budgets
  • Exciting projects: Modern solutions with Fortune 500 and top product companies
  • Flextime: Flexible schedule with remote and office options
]]>