brainstate

discover · compare · collections · requirements · contribute · feed · watching · stats · help with your tokens

confident-ai/deepeval

The LLM Evaluation Framework

Python★ 19.3kApache-2.0GitHub ↗

History and source changes · Reports and owner responses · Compare

Similar

  1. promptfoo/promptfoo

    Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

    TypeScript★ 26.8kMITGitHub ↗

    Why this match

    topics evaluation-framework, llm-evaluation, llm-evaluation-framework; language differs: Python → TypeScript; license differs: Apache-2.0 → MIT

  2. EleutherAI/lm-evaluation-harness

    A framework for few-shot evaluation of language models.

    Python★ 14.4kMITGitHub ↗

    Why this match

    topics evaluation-framework; license differs: Apache-2.0 → MIT

  3. jeinlee1991/chinese-llm-benchmark

    非线智能 NoneLinear - ReLE评测：中文AI大模型能力评测（持续更新）：目前已囊括374个大模型，覆盖chatgpt、gpt-5.4、谷歌gemini-3.1-pro、Claude-4.6、文心ERNIE-X1.1、ERNIE-5.0、qwen3.6-max、qwen3.6-plus、百川、讯飞星火、商汤senseChat等商用模型， 以及step3.5-flash、kimi-k2.6、ernie4.5、MiniMax-M2.7、deepseek-v4、Qwen3.6、llama4、智谱GLM-5.1、MiMo-V2、LongCat、gemma4、mistral等开源大模型。不仅提供排行榜，也提供规模超200万的大模型缺陷库！方便广大社区研究分析、改进大模型。

    ★ 6.5kGitHub ↗

    Why this match

    topics llm-evaluation

  4. NVIDIA/garak

    the LLM vulnerability scanner

    Python★ 9.9kApache-2.0GitHub ↗

    Why this match

    topics llm-evaluation

  5. pyenv/pyenv

    Simple Python version management

    Shell★ 45.2kMITGitHub ↗

    Why this match

    topics python; language differs: Python → Shell; license differs: Apache-2.0 → MIT

  6. wanghao-SunSky/cs329z-chinese

    斯坦福 Stanford CS329Z Engineering AI Agents 中文教程：面向 Python 开发者的 AI 智能体课程，19 讲与 4 项实践，涵盖 LLM、RAG、工具调用、记忆、多智能体、评测与安全。完整 Markdown、在线与离线 HTML。Chinese course notes.

    HTML★ 486GitHub ↗

    Why this match

    topics llm-evaluation, python; language differs: Python → HTML

  7. PyModel/jev-judge-mcp

    Typed judgment tools for MCP agents. TypeSafe's Jev model as verify, screen, find, classify, rerank, decide, compare, extract, review, gate, and score: the model judges, policy decides auto, review, or escalate.

    Python★ 49MITGitHub ↗

    Why this match

    topics llm-evaluation, python; license differs: Apache-2.0 → MIT