appLocal Models
Inference engine written in C that runs very large mixture-of-experts open models on hardware you already own, keeping active experts in memory and streaming the rest from disk. Supports CPU, CUDA and Vulkan.
Every tool tagged βcβ, grouped by category.
Inference engine written in C that runs very large mixture-of-experts open models on hardware you already own, keeping active experts in memory and streaming the rest from disk. Supports CPU, CUDA and Vulkan.