rea
Reverse engineer anything with agents, from app behavior down to native binaries.
Open Source219.4K
7views3clicks
A
APIEval-20
An open benchmark for AI agents that test APIs
Development
24views2clicks
About
APIEval-20 is a black-box benchmark for API testing agents. Each agent gets only a JSON schema and one sample payload, then generates a test suite. We run those tests against live reference APIs with planted bugs and score bug detection, API coverage, and efficiency. Unlike LLM-as-judge evals, scoring is fully objective: a bug is either caught or it isn’t. Tasks span auth, errors, pagination, schemas, and multi-step flows. Open on Hugging Face.
