Applied AI Engineer - India
dscoutRemotely
pythonllmprompt engineeringagent based systemsevaluation harnessesproduction monitoring
Job Description
📋 Description
- Own the production improvement loop across agent behavior, customer and operator feedback
- Instrument agent workflows so model interactions, tool use, decisions, failures, human edits, and
- Define meaningful quality standards, representative evaluation datasets, regression coverage, and
- Investigate why agents underperform across context, knowledge, instructions, tools, routing
- Design and ship targeted behavior improvements, including changes to prompting, context
- Build backend services, APIs, data models, and feedback pipelines that make agent behavior
🎯 Requirements
- 2-5 years of software engineering experience, with hands-on experience building or operating
- Fluency with prompting and context engineering as an engineering discipline: you iterate on
- Experience building or maintaining evaluation harnesses for AI systems: offline eval sets
- Genuine comfort with non-determinism: you reason about agent behavior across a distribution of
- Experience running experiments (A/B, staged rollouts, production replay) to validate whether a
- A track record of shipping features real users depended on, and owning what happened after launch
🎁 Benefits
- A strong and competitive compensation package with a built-in bonus and equity program.
- An incredible and progressive benefits package (for both you and your dependents) to support
- An education stipend to support your growth & development, and a remote work stipend.
- A company that is open and transparent with our team. You will know what is happening and why it