Role OverviewAs a Senior Python Engineer, you will create challenging tasks and evaluation criteria within realistic simulated environments to evaluate AI coding agents. You will design tasks from intermediate states of these environments, write tests that verify agent solutions, and iterate on tasks and tests based on QA feedback.
What You Will Do
Your main day-to-day responsibilities will include building realistic developer environments, designing tasks, writing tests, and iterating on tasks and tests. You will also need to deeply understand where models fail and what scenarios reveal the difference between a good and a bad solution.
Why It Might Be a Fit
We look for candidates with 8+ years of experience in software development, with a strong understanding of Python, JavaScript/TypeScript, Docker, Postgres, Kafka, and Redis. You should have experience writing tests (functional, integration) and a high level of English proficiency (B2+).
Requirements
- 8+ years in software development
- Core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, Redis
- Experience writing tests (functional, integration)
- English proficiency - B2+
Benefits
- Paid per accepted task
- Rate depends on qualification tier and efficiency
- Up to the equivalent of $200/hr
]]>