GitHub topic

evaluation GitHub repositories

Explore 31 tracked GitHub repositories tagged evaluation. Compare their recorded star activity and implementation languages, then follow related topics to find other projects.

Snapshot: 2026-10-28. The collection gained 351 stars since the previous daily snapshot.

Tracked repositories: 31
Total stars: 210,850
Stars gained · 7 days: +2,261
Stars gained · 30 days: +10,840

Compare star activity

Up to five projects on this page, ordered by total stars. Open a project for its description and recorded star history.

Swipe to compare all star periods →

Star activity for evaluation repositories on this page
Repository	Total stars	Today	7 days	30 days
langfuse/langfuse	36.6K	+74	+516	+1,504
Tencent/WeKnora	35.7K	+112	+798	+5,060
mlflow/mlflow	28.6K	+17	+93	+409
promptfoo/promptfoo	26.5K	+44	+314	+1,013
comet-ml/opik	23.0K	+27	+189	+714

Explore repositories

Showing 1–31 of 31 tracked repositories.

langfuse/langfuse

🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.

Tencent/WeKnora

Open-source LLM knowledge platform: turn raw documents into a queryable RAG, an autonomous reasoning agent, and a self-maintaining Wiki.

★ 35.7K+112 todayGo

llm ai

View ↗

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

★ 28.6K+17 todayPython

ai machine-learning

View ↗

promptfoo/promptfoo

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

★ 26.5K+44 todayTypeScript

llm rag

View ↗

comet-ml/opik

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

★ 23.0K+27 todayPython

llm openai

View ↗

open-compass/opencompass

OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.

★ 7.6K+2 todayPython

llm evaluation

View ↗

Helicone/helicone

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

Kiln-AI/Kiln

Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.

langwatch/langwatch

Open-Source Agent Observability & Evals for AI Platform teams. Trace, test, route and govern every agent & coding assistant in the company.

★ 5.0K+3 todayTypeScript

llm ai

View ↗

open-compass/VLMEvalKit

Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks

★ 4.5K+5 todayPython

llm claude

View ↗

modelscope/evalscope

A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.

★ 3.6K+5 todayPython

llm rag

View ↗

lmnr-ai/lmnr

Laminar - open-source observability platform purpose-built for AI agents. YC S24.

★ 3.5K+6 todayTypeScript

ai agents

View ↗

huggingface/evaluate

🤗 Evaluate: A library for easily evaluating machine learning models and datasets.

★ 2.5K+0 todayPython

machine-learning evaluation

View ↗

Sahir619/fable-method

The Fable Workflow: how Claude Fable 5 worked, distilled into skills any model can run, with the eval that keeps it honest. Think / act / prove.

★ 2.3K+2 todayPython

llm ai-agents

View ↗

Xnhyacinth/Awesome-LLM-Long-Context-Modeling

📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥

★ 2.2K+0 todayUnknown

llm agent

View ↗

Cloud-CV/EvalAI

:cloud: :rocket: :bar_chart: :chart_with_upwards_trend: Evaluating state of the art in AI

★ 2.0K+0 todayPython

ai machine-learning

View ↗

trpc-group/trpc-agent-go

A Go framework for building production agent systems with graph workflows, tools, memory, A2A, AG-UI, MCP, evaluation, and observability.

★ 1.9K+1 todayGo

llm ai

View ↗

onestardao/WFGY

WFGY is heading toward WFGY 5.0 Polaris Protocol, a major open-source release for AI reasoning, RAG, agents, and real-world workflows. Includes Problem Map, Global Debug Card, WFGY 4.0, and the CFV Easter Egg.

★ 1.8K+0 todayJupyter Notebook

llm ai-agents

View ↗

MLGroupJLU/LLM-eval-survey

The official GitHub page for the survey paper "A Survey on Evaluation of Large Language Models".

★ 1.6K+0 todayUnknown

llm llms

View ↗

sepandhaghighi/pycm

Multi-class confusion matrix library in Python

NVIDIA-NeMo/Gym

Evaluate and improve models and agents using environments

★ 1.3K+4 todayPython

llm agents

View ↗

chunxiaoxx/nautilus-compass

Open memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform

★ 1.3K+0 todayPython

llm mcp

View ↗

TIGER-AI-Lab/ClawBench

Open-source benchmark for browser AI agents on daily tasks.

agentscope-ai/OpenJudge

OpenJudge: A Unified Framework for Holistic Evaluation and Quality Rewards

★ 914+22 todayPython

llm agent

View ↗

nolabs-ai/deepfabric

Generate High-Quality Synthetics, Train, Measure, and Evaluate in a Single Pipeline

★ 891+0 todayPython

ai machine-learning

View ↗

chrisliu298/awesome-llm-unlearning

A resource repository for machine unlearning in large language models

HowieHwong/TrustLLM

[ICML 2024] TrustLLM: LLM trustworthiness evaluation across truthfulness, safety, fairness, robustness, privacy and ethics. Python/CLI toolkit for local models, compatible APIs and AI agent orchestration.

run-llama/ParseBench

ParseBench - A Document Parsing Benchmark for AI Agents

★ 618+0 todayPython

llm machine-learning

View ↗

NVIDIA/SkillEvaluator

Multi-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior.

★ 607+8 todayPython

claude-code codex

View ↗

DocAILab/XRAG

XRAG: eXamining the Core - Benchmarking Foundational Component Modules in Advanced Retrieval-Augmented Generation

★ 583+7 todayPython

llm rag

View ↗

facebookresearch/meta-agents-research-environments

Meta Agents Research Environments is a comprehensive platform designed to evaluate AI agents in dynamic, realistic scenarios. Unlike static benchmarks, this platform introduces evolving environments where agents must adapt their strategies as new information becomes available, mirroring real-world challenges.

★ 575+1 todayPython

llm ai

View ↗

evaluation GitHub repositories

Compare star activity

Explore repositories

langfuse/langfuse

Tencent/WeKnora

mlflow/mlflow

promptfoo/promptfoo

comet-ml/opik

open-compass/opencompass

Helicone/helicone

Kiln-AI/Kiln

langwatch/langwatch

open-compass/VLMEvalKit

modelscope/evalscope

lmnr-ai/lmnr

huggingface/evaluate

Sahir619/fable-method

Xnhyacinth/Awesome-LLM-Long-Context-Modeling

Cloud-CV/EvalAI

trpc-group/trpc-agent-go

onestardao/WFGY

MLGroupJLU/LLM-eval-survey

sepandhaghighi/pycm

NVIDIA-NeMo/Gym

chunxiaoxx/nautilus-compass

TIGER-AI-Lab/ClawBench

agentscope-ai/OpenJudge

nolabs-ai/deepfabric

chrisliu298/awesome-llm-unlearning

HowieHwong/TrustLLM

run-llama/ParseBench

NVIDIA/SkillEvaluator

DocAILab/XRAG

facebookresearch/meta-agents-research-environments

Sign in to continue

Loading

Account settings

Delete listing?