Role OverviewWe are seeking an Apache Spark Developer to support NGA's Data Modernization Services (DMS) mission by building and optimizing large-scale data processing pipelines. This role focuses on developing high-performance Spark applications within a containerized, Kubernetes-based environment, supporting mission analytics, data exploitation, and AI/ML integration.
What You Will Do
Design, develop, and maintain Apache Spark pipelines (batch and streaming) using PySpark and/or Scala. Process and transform large-scale datasets using modern data lake architectures (Delta Lake, Parquet). Optimize Spark jobs for performance, including partitioning strategies, shuffle optimization, memory tuning, and file sizing and storage efficiency.
Why It Might Be a Fit
The ideal candidate thrives in distributed data environments, understands performance tuning deeply, and can operate effectively in secure, air-gapped systems. Experience working with Apache Spark, Python, data processing at scale, Parquet, and Delta Lake is required. Strong troubleshooting and performance tuning skills are also necessary.
Requirements
- TS/SCI (eligibility) with ability/willingness to obtain/maintain counterintelligence polygraph
- Bachelor’s degree plus 5 years’ experience in data engineering or Spark development
- Strong hands-on experience with Apache Spark, Python, data processing at scale, Parquet, and Delta Lake
- Experience working with distributed data systems, Docker, and Kubernetes
- Familiarity with Bash or scripting
- Strong troubleshooting and performance tuning skills
Benefits
- Generous PTO plus 11 Federal Holidays
- Retirement Planning – 401k Fully Vested with Match
- Tuition Assistance Program – Annual contributions to help you pay down your loans
- Annual Health and Wellness Allowance – buy an Apple Watch, a treadmill, or hit the gym on us
- Career Development – Annual Funds to spend on Education and Training
- Volunteer Time Off – Annually, all employees can spend 8 hours directly supporting a charity of choice
- Charitable Match – ABSC matches an employee’s donation to a qualifying charity
- Referral Program – We pay for internal and external referrals!
- LOV Awards – Earn bonus awards throughout the year from our Living Our Values awards program
]]>