discover · compare · collections · requirements · contribute · feed · watching · stats · help with your tokens
Helicone/helicone
🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓
History and source changes · Reports and owner responses · Compare
Similar
- langfuse/langfuse
🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
Why this match
topics analytics, evaluation, langchain
- comet-ml/opik
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
Why this match
topics evaluation, langchain, llama-index; language differs: TypeScript → Python
- vibrantlabsai/ragas
Supercharge Your LLM Application Evaluations 🚀
Why this match
topics evaluation, llm, llmops; language differs: TypeScript → Python
- mlflow/mlflow
The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.
Why this match
topics evaluation, langchain, llm-evaluation; language differs: TypeScript → Python
- NirDiamant/RAG_Techniques
This repository showcases various advanced techniques for Retrieval-Augmented Generation (RAG) systems. Each technique has a detailed notebook tutorial.
Why this match
topics gpt, langchain, llama-index; language differs: TypeScript → Jupyter Notebook
- Arize-ai/phoenix
AI Observability & Evaluation
Why this match
topics langchain, llm-evaluation, llmops; language differs: TypeScript → Python
- nebuly-ai/optimate
A collection of libraries to optimise AI model performances
Why this match
topics analytics, large-language-models, llm; language differs: TypeScript → Python
- promptfoo/promptfoo
Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.
Why this match
topics evaluation, llm, llm-evaluation; license differs: Apache-2.0 → MIT