Reading the repo — stats and the shipped feed land in about ten seconds…
A high-throughput and memory-efficient inference and serving engine for LLMs
89.7k 21.1k 6.9k openLast shipped 15m ago
A high-throughput and memory-efficient inference and serving engine for LLMs