Research Engineer, Benchmarks
Hud
- КомпанияHud
- СтранаСША
- ГрафикOnSite
- АдресSan Francisco
Описание
ABOUT HUD
HUD https://www.hud.ai/'s mission is to build reliable, fair and open infrastructure for AI data. We want data to be valuable for the people who create it and trustworthy for the labs that train on it. Our team is a quickly growing group of researchers, engineers and operators building the economy that shapes what AI will become. Backed by $16M from top VCs and YC (W25), our marketplace and platform are used by startups, Fortune 500 companies and frontier labs.
ABOUT THE ROLE
We’re looking for Research Engineers to build high-quality benchmarks for evaluating frontier agents on domain-specific tasks. You’ll build benchmarks that are technically rigorous, practically useful, and credible to frontier labs.
Обязанности
- Own the design, implementation, and quality of HUD’s internal agent benchmarks
- Work with subject-matter experts to define tasks and create domain-specific benchmarks that evaluate agents on realistic workflows
- Build infrastructure to reliably run models and agents against benchmark tasks
- Develop metrics and analyses to understand benchmark difficulty, reliability, and failure modes
- Validate whether benchmark performance correlates with real-world evals, customer needs, and lab expectations
- Write clear documentation and benchmark reports that make results legible and credible to technical audiences
EXPERIENCE
You may be a good fit if you have:
- Proficiency in Python, Docker, and Linux environments
- Published papers or…
Вакансии этой компании
- Open Role — Hud, США
- Product Engineer, Frontend & Design — Hud, США
- Outbound Account Executive - Remote (PT) — Hud
- GTM Lead, Commercial Partnerships — Hud, США
- Head of Talent — Hud, США
Похожие вакансии
- Research Engineer, Benchmarks — Clera, Сингапур
- Research Scientist - Frontier Benchmarks — Snorkel Ai, США
- Program Director - Open Benchmarks Grants — Snorkel Ai, США
- Research Lead, Open Benchmarks Grants — Snorkel Ai, США
- Research Scientist, Open Benchmarks Red Team — Snorkel Ai, США
- Research Engineer - Benchmarks — Firecrawl, США
- Expert Talent Pool, Evals and Benchmarks — Clera
- Expert Talent Pool, Evals and Benchmarks (Associate) — Clera