Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.
Resolve production issues, fast. An open source observability platform unifying session replays, logs, metrics, traces and errors powered by ClickHouse and OpenTelemetry.
Popularity
Repository stars2529,985Highest
7-day net stars——
Fork43510Highest
30-day trendInsufficient historyInsufficient history
Cost detailsCheck current official software, hosting, model and infrastructure pricing.Check current official software, hosting, model and infrastructure pricing.
Pricing source——
First collected2026-11-032026-11-03
MetricRepository stars, not unique users or individual-skill installsRepository stars, not unique users or individual-skill installs
LanguageGoTypeScript
About this projectKubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.Resolve production issues, fast. An open source observability platform unifying session replays, logs, metrics, traces and errors powered by ClickHouse and OpenTelemetry.
Measured at2026-11-03 08:28 UTC2026-11-02 19:03 UTC
License tiers are derived from metadata. Check obligations for your use; unknown does not mean prohibited. Ranked by repository stars, one position per repository. Stars and forks are not user counts.