GitHub topic

llm-evaluation GitHub repositories

Explore 22 tracked GitHub repositories tagged llm-evaluation. Compare their recorded star activity and implementation languages, then follow related topics to find other projects.

Snapshot: 2026-10-28. The collection gained 277 stars since the previous daily snapshot.

Tracked repositories: 22
Total stars: 181,081
Stars gained · 7 days: +1,748
Stars gained · 30 days: +6,213

Compare star activity

Up to five projects on this page, ordered by total stars. Open a project for its description and recorded star history.

Swipe to compare all star periods →

Star activity for llm-evaluation repositories on this page
Repository	Total stars	Today	7 days	30 days
langfuse/langfuse	36.6K	+74	+516	+1,504
mlflow/mlflow	28.6K	+17	+93	+409
promptfoo/promptfoo	26.5K	+44	+314	+1,013
comet-ml/opik	23.0K	+27	+189	+714
Arize-ai/phoenix	12.0K	+19	+106	+394

Explore repositories

Showing 1–22 of 22 tracked repositories.

langfuse/langfuse

🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

★ 28.6K+17 todayPython

ai machine-learning

View ↗

promptfoo/promptfoo

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

★ 26.5K+44 todayTypeScript

llm rag

View ↗

comet-ml/opik

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

★ 23.0K+27 todayPython

llm openai

View ↗

Arize-ai/phoenix

AI Observability & Evaluation

★ 12.0K+19 todayPython

agents openai

View ↗

oumi-ai/oumi

Easily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!

★ 9.4K+2 todayPython

llm agentic-ai

View ↗

Tencent/AI-Infra-Guard

A full-stack AI Red Teaming platform securing AI ecosystems via Agent Scan, Skills Scan, MCP scan, AI Infra scan and LLM jailbreak evaluation.

Q00/ouroboros

Agent OS: the agent gets smarter on its own. We just hold the line: Interview-gated, staged evaluation, budgeted evolution loop. MCP server, 14 runtimes: Claude Code, Codex CLI, Gemini CLI, OpenCode, Copilot, Kiro and more.

Helicone/helicone

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

Giskard-AI/giskard-oss

🐢 Open-Source Evaluation & Testing library for LLM Agents

★ 5.9K+6 todayPython

llm mlops

View ↗

hemansnation/AI-Engineer-Headquarters

A collection of scientific methods, processes, algorithms, and systems to build stories & models.

★ 3.8K+1 todayJupyter Notebook

llm ai

View ↗

truera/trulens

Evaluation and Tracking for LLM Experiments and AI Agents

★ 3.6K+2 todayPython

machine-learning ai-agents

View ↗

lmnr-ai/lmnr

Laminar - open-source observability platform purpose-built for AI agents. YC S24.

★ 3.5K+6 todayTypeScript

ai agents

View ↗

cvs-health/uqlm

[JMLR 2026] UQLM: A Python package for LLM hallucination detection using uncertainty quantification

★ 1.2K+0 todayPython

llm llm-evaluation

View ↗

jwuthri-cloud/Tracely-ai

Trace-native CI/CD for AI agents: grade production traces, cluster failures, freeze them into replayable regression tests, and block PRs that regress. Self-hostable.

★ 1.2K+12 todayPython

ai-agents mcp

View ↗

TIGER-AI-Lab/ClawBench

Open-source benchmark for browser AI agents on daily tasks.

JudgmentLabs/judgeval

The Continuous-Improvement Stack for Agents. Our environment data and evals power agent improvement and monitoring.

★ 1.1K+0 todayPython

llm agents

View ↗

benchflow-ai/awesome-evals

A curated, non-BS library of the best resources for building and evaluating AI agents — papers, blogs, talks, tools, benchmarks. Maintained by BenchFlow.

★ 1.0K+1 todayUnknown

llm ai-agents

View ↗

keli-wen/agy-staff

Hire Google's Antigravity CLI (agy) as a fast Gemini staffer for Claude Code and OpenAI Codex.

v-modal/awesome-jev-tools

A curated list of tools built for Jev — TypeSafe AI's System One model for typed decisions.

overmind-core/overmind

The platform for continuously improving AI agents.

★ 703+14 todayPython

llm ai-agents

View ↗

HowieHwong/TrustLLM

[ICML 2024] TrustLLM: LLM trustworthiness evaluation across truthfulness, safety, fairness, robustness, privacy and ethics. Python/CLI toolkit for local models, compatible APIs and AI agent orchestration.

llm-evaluation GitHub repositories

Compare star activity

Explore repositories

langfuse/langfuse

mlflow/mlflow

promptfoo/promptfoo

comet-ml/opik

Arize-ai/phoenix

oumi-ai/oumi

Tencent/AI-Infra-Guard

Q00/ouroboros

Helicone/helicone

Giskard-AI/giskard-oss

hemansnation/AI-Engineer-Headquarters

truera/trulens

lmnr-ai/lmnr

cvs-health/uqlm

jwuthri-cloud/Tracely-ai

TIGER-AI-Lab/ClawBench

JudgmentLabs/judgeval

benchflow-ai/awesome-evals

keli-wen/agy-staff

v-modal/awesome-jev-tools

overmind-core/overmind

HowieHwong/TrustLLM

Sign in to continue

Loading

Account settings

Delete listing?