AI Benchmark Engineer | Native Language Specialist - Spanish (Argentina) - Remote

LILT

• КомпанияLILT
• СтранаАргентина
• ФорматУдалённо
• ГрафикRemote
• АдресArgentina (Remote)

Описание

ABOUT THE OPPORTUNITY

We are building a rigorous, verifiable evaluation suite of Terminal-Bench tasks designed to test the limits of large language models on multilingual software challenges. Our goal is to measure multilingual robustness across prompt language effects, non-English data processing, and complex locale/encoding edge cases in terminal workflows.

We are seeking experienced native-speaking software engineers to design, build, and validate these benchmarks. You will create high-signal, high-quality tasks that genuinely test a model's ability to handle multilingual environments without relying on English translation crutches.

Note this is a remote, freelance opportunity

WHAT YOU’LL DELIVER

• Task Engineering: Evaluating Coding Agents.

• Asset Creation: Build realistic task environments using datasets and files in your native language. Crucially, these assets must remain in the target language to genuinely measure multilingual handling.

• Prompting & Translation: finding failure points where AI does not work, in your native language

• Implementation & Verification: Support the development of robust solutions (reference implementations) and write highly reliable, deterministic verifier scripts (using rubric-based judging only when strictly necessary).

• Calibration & Execution: Analyze execution logs and calibrate task difficulty (Easy to Very Hard) using standard Terminal-Bench run configurations against various model tiers (Haiku, Sonnet, Opus).

• Quality…

Вакансия на сайте компании

Откликнуться в JobToo

Вакансии этой компании

• Medical Translators - Haitian Creole - US-based — LILT, США
• Medical Translators - Nepali - US-based — LILT, США
• Program Manager, Life Sciences — LILT, США
• Medical Translators - Farsi - US-based — LILT, США
• Medical Translators - Thai - US-based — LILT, США

Похожие вакансии

• Clinical Field Specialist — Natera
• Customer Financial Services Representative, Credit Controller Specialist — Bloomberg
• FP&A Senior Specialist — Nubank, Мексика
• Sales Engineer, SaaS Product Specialist — Rubrik, США
• Enrollment Specialist — Thyme Care
• Legal Engineering Manager, Product Specialist — Harvey, США
• Finance Business Process Specialist — Tenstorrent, США
• Operations Area Specialist - SDF (Packaging - A Shift) — Novo Nordisk, США