All projects
Repositories in this selection 22 · UTC 2026-11-04
CompareRankProjectStarFork30-day trend7-day net changeLast pushSign in to save
1—Llangfuse / langfuse🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.TypeScriptNOASSERTION · Restricted / unknownInsufficient rank history36,8964,146——2026-11-02
2—Mmlflow / mlflowThe open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.PythonApache-2.0 · PermissiveInsufficient rank history28,6306,571——2026-11-02
3—Ppromptfoo / promptfooTest your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.TypeScriptMIT · PermissiveInsufficient rank history26,7482,633——2026-11-02
4—Ccomet-ml / opikAuthor description: Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.PythonApache-2.0 · PermissiveInsufficient rank history23,1221,923——2026-11-02
5—Cconfident-ai / deepevalThe LLM Evaluation FrameworkPythonApache-2.0 · PermissiveInsufficient rank history19,2662,163——2026-11-02
6—AArize-ai / phoenixAI Observability & EvaluationPythonNOASSERTION · Restricted / unknownInsufficient rank history12,1031,258——2026-11-02
7—NNVIDIA / garakthe LLM vulnerability scannerPythonApache-2.0 · PermissiveInsufficient rank history9,8421,437——2026-11-02
8—Ooumi-ai / oumiEasily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!PythonApache-2.0 · PermissiveInsufficient rank history9,421798——2026-11-03
9—TTencent / AI-Infra-GuardA full-stack AI Red Teaming platform securing AI ecosystems via Agent Scan, Skills Scan, MCP scan, AI Infra scan and LLM jailbreak evaluation.PythonApache-2.0 · PermissiveInsufficient rank history7,353728——2026-11-03
10—Jjeinlee1991 / chinese-llm-benchmarkNon-Linear Intelligence NoneLinear - ReLE evaluation: Chinese AI large model capability evaluation (continuously updated): currently includes 374 large models, covering chatgpt, gpt-5.4, Google gemini-3.1-pro, Claude-4.6, Wenxin ERNIE-X1.1, ERNIE-5.0, qwen3.6-max, qwen3.6-plus, Baichuan, iFlytek Spark, SenseChat and other commercial models. As well as step3.5-flash, kimi-k2.6, ernie4.5, MiniMax-M2.7, deepseek-v4, Qwen3.6, llama4, GLM-5.1, MiMo-V2, LongCat, gemma4, mistral and other open source large models. It not only provides rankings, but also provides a large model defect library with a scale of over 2 million! It is convenient for the community to research, analyze and improve large models.UnknownUnknown · Restricted / unknownInsufficient rank history6,499262——2026-10-27
11—QQ00 / ouroborosAgent OS: the agent gets smarter on its own. We just hold the line: Interview-gated, staged evaluation, budgeted evolution loop. MCP server, 14 runtimes: Claude Code, Codex CLI, Gemini CLI, OpenCode, Copilot, Kiro and more.PythonMIT · PermissiveInsufficient rank history6,308638——2026-11-01
12—HHelicone / helicone🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓TypeScriptApache-2.0 · PermissiveInsufficient rank history6,254701——2026-10-28
13—GGiskard-AI / giskard-oss🐢 Open-Source Evaluation & Testing library for LLM AgentsPythonApache-2.0 · PermissiveInsufficient rank history5,941560——2026-11-02
14—PPacktPublishing / LLM-Engineers-HandbookAuthor description: The LLM's practical guide: From the fundamentals to deploying advanced LLM and RAG apps to AWS using LLMOps best practicesPythonMIT · PermissiveInsufficient rank history5,4141,330——2026-04-22
15—EEvolvingLMMs-Lab / lmms-evalA multimodal evaluation toolkit covering text, image, video, and audio tasks.PythonNOASSERTION · Restricted / unknownInsufficient rank history4,479683——2026-11-02
16—Hhemansnation / AI-Engineer-HeadquartersA collection of scientific methods, processes, algorithms, and systems to build stories & models.Jupyter NotebookUnknown · Restricted / unknownInsufficient rank history3,776703——2026-10-28
17—Ttruera / trulensEvaluation and Tracking for LLM Experiments and AI AgentsPythonMIT · PermissiveInsufficient rank history3,626384——2026-11-03
18—Llmnr-ai / lmnrLaminar - open-source observability platform purpose-built for AI agents. YC S24.TypeScriptApache-2.0 · PermissiveInsufficient rank history3,472258——2026-11-02
19—Ggenieincodebottle / generative-aiComprehensive resources on Generative AI, including a detailed roadmap, projects, use cases, interview preparation, and coding preparation.Jupyter NotebookMIT · PermissiveInsufficient rank history2,659645——2026-10-26
20—Ffuture-agi / future-agiOpen-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.PythonApache-2.0 · PermissiveInsufficient rank history2,241704——2026-11-03
License tiers are derived from metadata. Check obligations for your use; unknown does not mean prohibited.