DevOps Engineer - AI Model Evaluator

Mercor
New York, NY
Remote
Job Description
Role Overview

Mercor is seeking a DevOps Engineer with 2+ years of experience to evaluate AI model outputs and apply professional engineering judgment to realistic infrastructure engineering scenarios.

What You Will Do

Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks, review model-generated implementations, identify bugs, and compare outputs from multiple frontier models.

Why It Might Be a Fit

The ideal candidate will have experience with AWS, Azure, GCP, Kubernetes, Terraform, CI/CD pipelines, and observability tooling, as well as regular use of AI coding agents.

Requirements

  • 2+ years of professional DevOps, SRE, or Cloud Engineering experience
  • Experience with AWS, Azure, GCP, Kubernetes, Terraform, CI/CD pipelines, or observability tooling
  • Regular use of AI coding agents such as Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or similar tools
  • Ability to evaluate model-generated infrastructure and reliability engineering solutions

Benefits

  • $400 per accepted task
  • Compensation tied to accepted work
]]>