vLLM

vLLM is an open-source library for high-throughput Large Language Model inference and serving, utilized by companies to deploy production-grade generative AI applications efficiently.