Compare projects
Select 2–4 projects to compare on the same basis.
Only identical known values are hidden. Unknown information remains visible.
Ppromptfoo
Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.
Popularity
Repository stars6,13326,748Highest
7-day net stars——
Fork1,0412,633Highest
30-day trendInsufficient historyInsufficient history
Maintenance
Last push2026-09-182026-11-02
Maintenance notesUnknownNot verified
Repository created2023-10-042023-04-28
License & use
LicenseMIT · PermissiveMIT · Permissive
PricingPricing not verifiedPricing not verified
Basics
Browse by purposeModel and data evaluation · Software testingContinuous delivery · Model and data evaluation · Retrieval-augmented generation · Software testing
TypeDatasets & evaluationsApps & systems
SubcategoryBenchmarksSecurity tools
PlatformNot verifiedNot verified
Cost detailsCheck hosted-service, model and infrastructure costs for the option you choose.Check current official software, hosting, model and infrastructure pricing.
Pricing source——
First collected2026-11-022026-11-03
MetricRepository Stars, not unique usersRepository stars, not unique users or individual-skill installs
LanguagePythonTypeScript
About this projectOrganize software-engineering agent evaluation around real repository issues.Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.
Measured at2026-11-02 18:45 UTC2026-11-02 19:01 UTC
License tiers are derived from metadata. Check obligations for your use; unknown does not mean prohibited. Ranked by repository stars, one position per repository. Stars and forks are not user counts.