GitHub topic

model-serving GitHub repositories

Explore 16 tracked GitHub repositories tagged model-serving. Compare their recorded star activity and implementation languages, then follow related topics to find other projects.

Snapshot: 2026-10-28. The collection gained 113 stars since the previous daily snapshot.

Tracked repositories: 16
Total stars: 140,408
Stars gained · 7 days: +1,412
Stars gained · 30 days: +3,570

Compare star activity

Up to five projects on this page, ordered by daily star growth. Open a project for its description and recorded star history.

Swipe to compare all star periods →

Star activity for model-serving repositories on this page
Repository	Total stars	Today	7 days	30 days
vllm-project/vllm	95.7K	+86	+1,283	+2,908
flyteorg/flyte	7.8K	+9	+33	+183
vllm-project/vllm-ascend	3.0K	+6	+26	+128
pegainfer-project/pegainfer	729	+3	+7	+17
kserve/kserve	6.2K	+2	+22	+159

Explore repositories

Showing 1–16 of 16 tracked repositories.

vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

★ 95.7K+86 todayPython

llm openai

View ↗

flyteorg/flyte

Type-safe, distributed orchestration of agents, ML pipelines, and real-time inference on your k8s — in pure Python with async/await, also other languages (rust, go and ts)

★ 7.8K+9 todayGo

machine-learning python

View ↗

vllm-project/vllm-ascend

Community maintained hardware plugin for vLLM on Huawei Ascend

★ 3.0K+6 todayPython

llm mlops

View ↗

pegainfer-project/pegainfer

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

★ 729+3 todayRust

llm deepseek

View ↗

kserve/kserve

Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes

bentoml/BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

★ 8.9K+2 todayPython

llm machine-learning

View ↗

alibaba/rtp-llm

RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

★ 1.4K+1 todayPython

llm gpt

View ↗

ModelTC/LightLLM

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

★ 4.3K+1 todayPython

llm deep-learning

View ↗

kitops-ml/kitops

An open source DevOps tool from the CNCF for packaging and versioning AI/ML models, datasets, code, and configuration into an OCI Artifact.

ome-projects/ome

Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton

★ 521+1 todayGo

llm deepseek

View ↗

thu-pacman/chitu

High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.

★ 3.0K+1 todayPython

llm pytorch

View ↗

basetenlabs/truss

The simplest way to serve AI/ML models in production

★ 1.2K+0 todayPython

machine-learning artificial-intelligence

View ↗

openvinotoolkit/model_server

A scalable inference server for models optimized with OpenVINO™

★ 952+0 todayC++

ai machine-learning

View ↗

MLRun is an open source MLOps platform for quickly building and managing continuous ML applications across their lifecycle. MLRun integrates into your development and CI/CD environment and automates the delivery of production data, ML pipelines, and online applications.

★ 1.7K+0 todayPython

machine-learning data-science

View ↗