cuda GitHub repositories

sgl-project/sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

★ 38.3K+72 todayPython

llm deepseek

tracel-ai/burn

Burn is a next generation tensor library and Deep Learning Framework that doesn't compromise on flexibility, efficiency and portability.

★ 16.2K+6 todayRust

isl-org/Open3D

Open3D: A Modern Library for 3D Data Processing

LMCache/LMCache

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

★ 12.2K+10 todayPython

llm pytorch

replicate/cog

Containers for machine learning

★ 9.5K+0 todayGo

ai machine-learning

A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

★ 9.2K+4 todayC++

machine-learning python

gpustack/gpustack

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

★ 5.9K+6 todayPython

llm openai

NVIDIA/cuml

NVIDIA cuML: GPU-Accelerated Machine Learning

★ 5.3K+1 todayPython

machine-learning gpu

iree-org/iree

A retargetable MLIR-based machine learning compiler and runtime toolkit.

★ 4.0K+2 todayC++

machine-learning pytorch

NVIDIA/skills

Agent Skills for NVIDIA products — install into Claude Code, Codex, and other coding agents to run Physical AI, robotics, simulation, CUDA, and RAG workflows end to end.

★ 3.8K+16 todayPython

llm ai-agents

NVIDIA/TransformerEngine

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide better performance with lower memory utilization in both training and inference.

★ 3.6K+4 todayPython

BBuf/how-to-optim-algorithm-in-cuda

how to optimize some algorithm in cuda.

★ 3.3K+0 todayCuda

llm

containers/ramalama

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.

★ 3.1K+0 todayPython

heavyai/heavydb

HeavyDB (formerly MapD/OmniSciDB)

★ 3.1K+0 todayC++

machine-learning gpu

pytorch/TensorRT

PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT

★ 3.0K+1 todayPython

CVCUDA/CV-CUDA

CV-CUDA™ is an open-source, GPU accelerated library for cloud-scale image processing and computer vision.

★ 2.7K+0 todayC++

machine-learning python

withcatai/node-llama-cpp

Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level

★ 2.2K+1 todayTypeScript

mw00/project-maya

Project Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniMax-M3.1-Flash (428B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools.

★ 2.1K+100 todayCuda

llm self-hosted

pykeen/pykeen

🤖 A Python library for learning and evaluating knowledge graph embeddings

★ 2.0K+0 todayPython

dphnAI/sonar

Large-scale LLM inference engine

★ 1.9K+3 todayPython

machine-learning cuda

beam-cloud/beta9

Ultrafast serverless GPU inference, sandboxes, and background jobs

★ 1.8K+3 todayGo

llm self-hosted

tenstorrent/tt-metal

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.

★ 1.7K+3 todayPython

datawhalechina/zero-to-sglang

Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.

★ 1.6K+8 todayPython

llm llm-inference

uccl-project/uccl

UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)

★ 1.6K+2 todayC++

mratsim/Arraymancer

A fast, ergonomic and portable tensor library in Nim with a deep learning focus for CPU, GPU and embedded devices via OpenMP, Cuda and OpenCL backends

jmaczan/tiny-vllm

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

★ 1.2K+2 todayC++

NVIDIA/raft

RAFT contains fundamental widely-used algorithms and primitives for machine learning and information retrieval. The algorithms are CUDA-accelerated and form building blocks for more easily writing high performance applications.

NVIDIA/cuvs

cuVS - a library for vector search and clustering on the GPU

★ 877+3 todayCuda

llm machine-learning

invergent-ai/surogate

Train and serve LLMs at extreme speed and massive throughput.

★ 859+3 todayC++

llm deep-learning

machine-learning neural-network

brucefan1983/GPUMD

Graphics Processing Units Molecular Dynamics

★ 859+0 todayCuda

jasmcaus/caer

High-performance Vision library in Python. Scale your research, not boilerplate.

TrenTorch/TrenTorch

Don't memorize ML. Understand it from first principles including Inference, Cuda, Deep Learning, Data Science and Reinforcement Learning

★ 734+15 todayPython

pegainfer-project/pegainfer

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

★ 729+3 todayRust

llm deepseek

MarioSieg/magnetron

A zero-dependency ML framework in C with a modern Python API for full control over execution and memory.

★ 708+0 todayC

machine-learning python

hybridgroup/yzma

Go with your own intelligence - Write Go applications that directly integrate llama.cpp for local inference using hardware acceleration on Linux, macOS, Windows, & WebAssembly.

★ 661+0 todayGo

llm llama

zhongkaifu/TensorSharp

A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, iPhone App, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows/MacOS/iOS/Linux with full GPU capability

★ 632+2 todayC#

VecSzn/PolyStrata

Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V on a second engine written for PolyStrata. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost.

★ 574+33 todayC++

llm self-hosted

zengxiao-he/tessera

From teacher to tiles — a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous batching, speculative decoding, a Rust gateway, a JAX oracle, and interpretability tooling.

★ 559+0 todayPython

llm pytorch