c/devops

DevOps & Infra

Kubernetes, serverless, edge runtimes, observability, and scaling.

940members

1discussions

Rakesh Roy@rakeshroy·3d·c/devopsdiscussion

Deploying Distributed vLLM Clusters on Kubernetes with GPU Auto-scaling

Here is our production GitOps setup for managing high-availability GPU inference nodes across multiple Kubernetes clusters using KEDA and the NVIDIA GPU Operator. Key Components 1. Node Provisioning: Spot GPU instances with Karpenter on AWS

#devops #kubernetes #infra #ai

Community Rules

1. Be constructive and respectful

Feedback should be actionable and technical.

2. No spam or self-promotion without context

Disclose your affiliation when sharing projects.

3. Format code blocks with language tags

Use markdown formatting for readable snippets.