🇺🇸New YorkFull TimeSoftware Development
- Employee
Job Description
📋 Description
- Build benchmarks from real workflows: turn agent traces, customer edge cases and expert labels into
- Own the benchmarking effort: decide what we measure, run benchmark suites across models and agent
- Design graders: rubrics, programmatic checks and llm-as-judge, calibrated against human agreement
- Close the loop with post-training: turn evals into reward functions and RL…
pythonmachine learningsqlllmdatareinforcement learningsftbenchmarksevals
Match
Sign in and add your skills to see how well you match this role.
Tailor your CV to this jobPro
Pro rewrites your CV bullets for this job and checks every line against your CV, dropping anything it can't back up.
See ProBack to all jobs