Search. Pull. Build.

A Kubernetes-native platform for serving machine learning models at scale

A Kubernetes-native platform for serving machine learning models at scale

LiteLLM is a unified interface to call 100+ LLMs using the OpenAI format, providing a proxy server for multiple LLM providers.

LMCache is an LLM serving engine extension that stores and reuses KV caches across requests to reduce time-to-first-token (TTFT) and increase throughput. It integrates with vLLM to provide GPU-accelerated inference with shared KV cache management.

NVIDIA NeMo Framework is an end-to-end, cloud-native framework to build, customize, and deploy generative AI models anywhere.

NVIDIA NeMo Gym is a library for building reinforcement learning environments for large language models and serving them for rollout collection, reward computation, and training.

NVIDIA NeMo RL is a scalable post-training library for reinforcement learning and supervised fine-tuning of large language models (GRPO, DPO, SFT, distillation and reward modeling), built on Ray, vLLM, PyTorch and Megatron.