brainstate

discover · compare · collections · requirements · contribute · feed · watching · stats · help with your tokens

comet-ml/opik

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

Python★ 23.2kApache-2.0GitHub ↗

History and source changes · Reports and owner responses · Compare

Similar

  1. langfuse/langfuse

    🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.

    TypeScript★ 37.0kGitHub ↗

    Why this match

    topics evaluation, langchain, llama-index; language differs: Python → TypeScript

  2. Helicone/helicone

    🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

    TypeScript★ 6.3kApache-2.0GitHub ↗

    Why this match

    topics evaluation, langchain, llama-index; language differs: Python → TypeScript

  3. vibrantlabsai/ragas

    Supercharge Your LLM Application Evaluations 🚀

    Python★ 16.2kApache-2.0GitHub ↗

    Why this match

    topics evaluation, llm, llmops

  4. mlflow/mlflow

    The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

    Python★ 28.7kApache-2.0GitHub ↗

    Why this match

    topics evaluation, langchain, llm-evaluation

  5. Arize-ai/phoenix

    AI Observability & Evaluation

    Python★ 12.1kGitHub ↗

    Why this match

    topics langchain, llm-evaluation, llmops

  6. promptfoo/promptfoo

    Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

    TypeScript★ 26.8kMITGitHub ↗

    Why this match

    topics evaluation, llm, llm-evaluation; language differs: Python → TypeScript; license differs: Apache-2.0 → MIT

  7. Portkey-AI/gateway

    A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.

    TypeScript★ 13.3kMITGitHub ↗

    Why this match

    topics hacktoberfest, langchain, llm; language differs: Python → TypeScript; license differs: Apache-2.0 → MIT

  8. Chainlit/chainlit

    Build Conversational AI in minutes ⚡️

    Python★ 12.5kApache-2.0archivedGitHub ↗

    Why this match

    topics langchain, llm, openai