Agent Evaluation (AEOS)
AEOS Overview
The Agent Evaluation & Observability Service — register agents, run benchmarks, and detect drift and regressions.
Overview
AEOS is a cloud-native, event-driven service for evaluating AI agents. Register an agent, run it against a benchmark to produce an evaluation, store the resulting trajectories, and score them. Over time AEOS compares runs to surface drift and regressions. All endpoints live under the base path /api/v1/aeos.
Agents & benchmarks
| GET | /api/v1/aeos/agents | List registered agents. |
| POST | /api/v1/aeos/agents | Register an agent. |
| GET | /api/v1/aeos/benchmarks | List evaluation benchmarks. |
| POST | /api/v1/aeos/benchmarks | Create a benchmark. |
Evaluations & trajectories
| POST | /api/v1/aeos/evaluations | Start an evaluation run. |
| GET | /api/v1/aeos/evaluations/{id} | Retrieve an evaluation and its status. |
| GET | /api/v1/aeos/trajectories | List stored agent trajectories. |
| GET | /api/v1/aeos/scores | Read scores produced for an evaluation. |
Drift & regressions
| GET | /api/v1/aeos/drift | List detected behavioral drift. |
| GET | /api/v1/aeos/regressions | List regressions between runs. |
| POST | /api/v1/aeos/comparisons | Compare two evaluation runs. |
See every endpoint, schema, and event type in the AEOS API reference.