GitHub topic

llm-evaluation GitHub repositories

Explore 22 tracked GitHub repositories tagged llm-evaluation. Compare their recorded star activity and implementation languages, then follow related topics to find other projects.

Snapshot: 2026-10-28. The collection gained 277 stars since the previous daily snapshot.

Tracked repositories: 22
Total stars: 181,081
Stars gained · 7 days: +1,748
Stars gained · 30 days: +6,213

Compare star activity

Up to five projects on this page, ordered by daily star growth. Open a project for its description and recorded star history.

Swipe to compare all star periods →

Star activity for llm-evaluation repositories on this page
Repository	Total stars	Today	7 days	30 days
langfuse/langfuse	36.6K	+74	+516	+1,504
promptfoo/promptfoo	26.5K	+44	+314	+1,013
Tencent/AI-Infra-Guard	7.1K	+33	+225	+543
comet-ml/opik	23.0K	+27	+189	+714
Arize-ai/phoenix	12.0K	+19	+106	+394

Explore repositories

Showing 1–22 of 22 tracked repositories.

langfuse/langfuse

🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

★ 26.5K+44 todayTypeScript

llm rag

View ↗

Tencent/AI-Infra-Guard

A full-stack AI Red Teaming platform securing AI ecosystems via Agent Scan, Skills Scan, MCP scan, AI Infra scan and LLM jailbreak evaluation.

comet-ml/opik

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

★ 23.0K+27 todayPython

llm openai

View ↗

Arize-ai/phoenix

AI Observability & Evaluation

★ 12.0K+19 todayPython

agents openai

View ↗

mlflow/mlflow

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

★ 28.6K+17 todayPython

ai machine-learning

View ↗

overmind-core/overmind

The platform for continuously improving AI agents.

★ 703+14 todayPython

llm ai-agents

View ↗

jwuthri-cloud/Tracely-ai

Trace-native CI/CD for AI agents: grade production traces, cluster failures, freeze them into replayable regression tests, and block PRs that regress. Self-hostable.

★ 1.2K+12 todayPython

ai-agents mcp

View ↗

TIGER-AI-Lab/ClawBench

Open-source benchmark for browser AI agents on daily tasks.

Q00/ouroboros

Agent OS: the agent gets smarter on its own. We just hold the line: Interview-gated, staged evaluation, budgeted evolution loop. MCP server, 14 runtimes: Claude Code, Codex CLI, Gemini CLI, OpenCode, Copilot, Kiro and more.

lmnr-ai/lmnr

Laminar - open-source observability platform purpose-built for AI agents. YC S24.

★ 3.5K+6 todayTypeScript

ai agents

View ↗

Giskard-AI/giskard-oss

🐢 Open-Source Evaluation & Testing library for LLM Agents

★ 5.9K+6 todayPython

llm mlops

View ↗

keli-wen/agy-staff

Hire Google's Antigravity CLI (agy) as a fast Gemini staffer for Claude Code and OpenAI Codex.

Helicone/helicone

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

truera/trulens

Evaluation and Tracking for LLM Experiments and AI Agents

★ 3.6K+2 todayPython

machine-learning ai-agents

View ↗

oumi-ai/oumi

Easily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!

★ 9.4K+2 todayPython

llm agentic-ai

View ↗

benchflow-ai/awesome-evals

A curated, non-BS library of the best resources for building and evaluating AI agents — papers, blogs, talks, tools, benchmarks. Maintained by BenchFlow.

★ 1.0K+1 todayUnknown

llm ai-agents

View ↗

hemansnation/AI-Engineer-Headquarters

A collection of scientific methods, processes, algorithms, and systems to build stories & models.

★ 3.8K+1 todayJupyter Notebook

llm ai

View ↗

cvs-health/uqlm

[JMLR 2026] UQLM: A Python package for LLM hallucination detection using uncertainty quantification

★ 1.2K+0 todayPython

llm llm-evaluation

View ↗

JudgmentLabs/judgeval

The Continuous-Improvement Stack for Agents. Our environment data and evals power agent improvement and monitoring.

★ 1.1K+0 todayPython

llm agents

View ↗

HowieHwong/TrustLLM

[ICML 2024] TrustLLM: LLM trustworthiness evaluation across truthfulness, safety, fairness, robustness, privacy and ethics. Python/CLI toolkit for local models, compatible APIs and AI agent orchestration.

v-modal/awesome-jev-tools

A curated list of tools built for Jev — TypeSafe AI's System One model for typed decisions.

llm-evaluation GitHub repositories

Compare star activity

Explore repositories

langfuse/langfuse

promptfoo/promptfoo

Tencent/AI-Infra-Guard

comet-ml/opik

Arize-ai/phoenix

mlflow/mlflow

overmind-core/overmind

jwuthri-cloud/Tracely-ai

TIGER-AI-Lab/ClawBench

Q00/ouroboros

lmnr-ai/lmnr

Giskard-AI/giskard-oss

keli-wen/agy-staff

Helicone/helicone

truera/trulens

oumi-ai/oumi

benchflow-ai/awesome-evals

hemansnation/AI-Engineer-Headquarters

cvs-health/uqlm

JudgmentLabs/judgeval

HowieHwong/TrustLLM

v-modal/awesome-jev-tools

Sign in to continue

Loading

Account settings

Delete listing?