Research Engineer, Benchmarks
Clera
- КомпанияClera
- СтранаСингапур
- ГрафикOnSite
- АдресSingapore
Описание
ABOUT THE ROLE
Join a small, technical team building benchmarks to evaluate frontier AI agents on realistic, domain-specific workflows. You will own evaluation design and implementation, helping ensure benchmark results are rigorous, credible, and relevant to real-world performance.
Обязанности
- Design, implement, and maintain benchmarks for evaluating frontier agents on domain-specific tasks.
- Work with subject-matter experts to translate real workflows into realistic tasks and evaluation criteria.
- Build reliable infrastructure for running models and agents against benchmark tasks at scale.
- Develop metrics and analyses to assess benchmark difficulty, reliability, and failure modes.
- Validate whether benchmark results correlate with real-world evaluations and user needs.
- Write clear documentation and reports for technical audiences.
Требования
- Two or more years of experience in research engineering or machine learning engineering, building AI benchmarks, evaluation infrastructure, or agent environments.
- Technical education or equivalent grounding, with strong first-principles reasoning about task design, scoring, and failure modes.
- Proficiency in Python, Docker, and Linux, including experience operating infrastructure for AI models or agents.
- Hands-on experience designing and running evaluations for AI agents or large language models, with relevant, focused experience in benchmarks or evaluation.
- Experience developing metrics…
Вакансии этой компании
- Founding Backend / AI Engineer — Clera, США
- Head of Operations / COO — Clera, США
- Software Engineer — Clera, США
- Developer Relations — Clera, США
- GTM Intern — Clera, США
Похожие вакансии
- Research Scientist - Frontier Benchmarks — Snorkel Ai, США
- Program Director - Open Benchmarks Grants — Snorkel Ai, США
- Research Lead, Open Benchmarks Grants — Snorkel Ai, США
- Research Scientist, Open Benchmarks Red Team — Snorkel Ai, США
- Research Engineer, Benchmarks — Hud, США
- Research Engineer - Benchmarks — Firecrawl, США
- Expert Talent Pool, Evals and Benchmarks — Clera
- Expert Talent Pool, Evals and Benchmarks (Associate) — Clera