Research Engineer, Benchmarks
Clera
- КомпанияClera
- СтранаСингапур
- ГрафикOnSite
- АдресSingapore
Описание
ABOUT THE ROLE
Join a small, technical AI evaluation team building rigorous benchmarks for frontier AI agents. You will own the design and implementation of domain-specific evaluations that help research teams and customers understand how agents perform on realistic workflows.
Обязанности
- Design, implement, and maintain internal benchmarks for evaluating frontier agents on domain-specific tasks.
- Partner with subject-matter experts to translate real-world workflows into benchmark tasks and evaluation criteria.
- Build reliable infrastructure to run models and agents against benchmark tasks at scale.
- Develop metrics and analyses to assess benchmark difficulty, reliability, and failure modes.
- Validate whether benchmark results align with real-world evaluations and user needs.
- Write clear technical documentation and reports explaining benchmark methods and results.
Требования
- Two to four years of experience in software engineering, machine learning engineering, or research engineering, including at least two years building AI benchmarks, evaluation infrastructure, or agent environments.
- Technical education or strong fundamentals in a technical field.
- Proficiency with Python, Docker, and Linux for research or production infrastructure.
- Hands-on experience designing and operating evaluations or environments for AI agents or large language models.
- Experience building reliable model or agent execution infrastructure, developing…
Вакансии этой компании
- Account Executive — Clera
- Account Executive — Clera, Германия
- Founding GTM Lead — Clera
- Performance Marketing Manager — Clera, Германия
- Lead Data Platform Engineer — Clera, Германия
Похожие вакансии
- Expert Talent Pool, Evals and Benchmarks (Associate) — Clera
- Expert Talent Pool, Evals and Benchmarks — Clera
- Expert Talent Pool, Evals and Benchmarks (Associate) — Clera
- Expert Talent Pool, Evals and Benchmarks — Clera
- Research Engineer, Benchmarks & Evaluation — RAMP, США
- Research Scientist, APEX Benchmarks — Mercor, США
- Mercor AI Research Fund Grants: Benchmarks — Mercor, США