GitHub topic

inference-engine GitHub repositories

Explore 14 tracked GitHub repositories tagged inference-engine. Compare their recorded star activity and implementation languages, then follow related topics to find other projects.

Snapshot: 2026-10-28. The collection gained 105 stars since the previous daily snapshot.

Tracked repositories: 14
Total stars: 36,090
Stars gained · 7 days: +604
Stars gained · 30 days: +2,233

Compare star activity

Up to five projects on this page, ordered by total stars. Open a project for its description and recorded star history.

Swipe to compare all star periods →

Star activity for inference-engine repositories on this page
Repository	Total stars	Today	7 days	30 days
halfrost/Halfrost-Field	13.3K	+0	+3	+16
FareedKhan-dev/kimi-k3-in-c	9.2K	+10	+71	+481
ovg-project/kvcached	2.0K	+3	+122	+503
dphnAI/sonar	1.9K	+3	+15	+26
nobodywho-ooo/nobodywho	1.7K	+5	+66	+214

Explore repositories

Showing 1–14 of 14 tracked repositories.

halfrost/Halfrost-Field

✍🏻 Source Code Deep Dives, System Design & Engineering Blogs | Halfrost-Field 冰霜之地：源码解析、系统设计与工程实践笔记

★ 13.3K+0 todayGo

llm machine-learning

View ↗

FareedKhan-dev/kimi-k3-in-c

A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.

ovg-project/kvcached

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

★ 2.0K+3 todayPython

llm ollama

View ↗

dphnAI/sonar

Large-scale LLM inference engine

★ 1.9K+3 todayPython

machine-learning cuda

View ↗

nobodywho-ooo/nobodywho

NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.

★ 1.7K+5 todayRust

llm ai

View ↗

datawhalechina/zero-to-sglang

Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.

★ 1.6K+8 todayPython

llm llm-inference

View ↗

qualcomm/ai-hub-models

Qualcomm® AI Hub Models is our collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.

★ 1.2K+2 todayPython

machine-learning pytorch

View ↗

peonist-ai/halogen-flash-server

Local inference that behaves like a frontier API: Qwen3.8-Flash-Next on AMD Strix Halo plus small NPU models, over the OpenAI and Anthropic APIs on one port.

★ 1.1K+12 todayShell

llm qwen

View ↗

jjang-ai/mlxstudio

MLX Studio - Easiest way to run LLM's on your Mac. All in one engine.

★ 988+0 todayUnknown

llm ai

View ↗

pegainfer-project/pegainfer

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

★ 729+3 todayRust

llm deepseek

View ↗

A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, iPhone App, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows/MacOS/iOS/Linux with full GPU capability

★ 632+2 todayC#

llm ai

View ↗

carloslfu/slotstream

Run a 105 GB AI model on a Mac that can't hold it. Qwen3.8-Flash-Next 4-bit, streamed from SSD on 16–64 GB Apple Silicon Macs. Fully native Swift, MLX and Metal. Offline, no Python. CLI, Swift library, and Ollama/OpenAI/Anthropic APIs. Works with Claude Code and Codex.

★ 595+24 todaySwift

llm claude-code

View ↗

VecSzn/PolyStrata

Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V on a second engine written for PolyStrata. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost.

★ 574+33 todayC++

llm self-hosted

View ↗

zengxiao-he/tessera

From teacher to tiles — a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous batching, speculative decoding, a Rust gateway, a JAX oracle, and interpretability tooling.

★ 559+0 todayPython

llm pytorch

View ↗

inference-engine GitHub repositories

Compare star activity

Explore repositories

halfrost/Halfrost-Field

FareedKhan-dev/kimi-k3-in-c

ovg-project/kvcached

dphnAI/sonar

nobodywho-ooo/nobodywho

datawhalechina/zero-to-sglang

qualcomm/ai-hub-models

peonist-ai/halogen-flash-server

jjang-ai/mlxstudio

pegainfer-project/pegainfer

zhongkaifu/TensorSharp

carloslfu/slotstream

VecSzn/PolyStrata

zengxiao-he/tessera

Sign in to continue

Loading

Account settings

Delete listing?