Senior Software Engineer – LLM Evaluation

24-MAG
New York, NY
Job Description
Role Overview

We are seeking experienced software engineers to contribute to advanced large language model evaluation, coding benchmark development, and AI-assisted software-engineering research.

What You Will Do

Selected professionals will curate and evaluate code, develop verification mechanisms, assess AI-generated software across multiple programming languages, and help research teams understand how advanced models perform throughout realistic software-development workflows.

Why It Might Be a Fit

Ideal candidates will have 3+ years of professional software-engineering experience, strong full-stack development capabilities, and experience building scalable, production-grade software.

Requirements

  • 3+ years of professional software-engineering experience
  • Strong full-stack development capabilities
  • Experience building scalable, production-grade software
  • Strong understanding of software architecture and system design
  • Deep knowledge of development, debugging, and code-quality assessment
  • Experience reviewing and improving complex software implementations
  • Proficiency in one or more of Python, JavaScript, Java, C++, Rust, or related languages
  • ReactJS, C, or Go experience may also be relevant to project assignments
  • Strong understanding of API design and production implementation
  • Familiarity with software monitoring and operational maintenance
  • Ability to reason across the complete software-engineering lifecycle
  • Strong analytical and problem-solving capabilities
  • Excellent written and verbal communication skills
  • Ability to provide clear, structured evaluation rationales
  • Comfortable collaborating remotely with research and technical teams

Benefits

  • Flexible workload of up to 40 hours per week
  • Minimum commitment: 10 hours per week
]]>