Loading
Senior Software Engineer, Agent Eval Platform
ServiceNow
Senior · 5+ yrs
Full-time
Mountain View, California, United States
USD 161k – USD 274k
Job description
Data Science · Artificial Intelligence
Skills
Primary
Secondary
Responsibilities
Build the judgement layer of our agent evaluation platform: the rubrics, the judges, the calibration against human labels
Establishing a reliability floor and an SLO for the harness itself
Getting to self-serve so any team runs an eval without bespoke integration
Leading the move to OpenTelemetry-native observability for the agent platform
Designing the span data model for agent trajectories — prompts, tool calls, plan updates, outcomes