C++ runtime for running LLMs locally on CPU + GPU. The backbone of every privacy-LLM stack.
Free · MIT · C++ · CPU/CUDA/ROCm/Metal
AI / LLMs — privacy-respecting options
LLM access without surveillance. Confidential-compute hosted (TEE), local runtimes, crypto-paid API gateways. Order matters: local first, then confidential-compute, then crypto-paid proxies, then mainstream APIs with email-only signup.
last reviewed 2026-05-13
› AI / LLMs — privacy-respecting options » Local Runtime 1