Problem Statement

Design an LLM serving system with continuous batching, KV cache management, and multi-GPU model parallelism.

GPU schedulingbatchingKV cachemodel shardinginference

Estimated: 60 min

OpenAIAnthropicGoogle

This is a Pro feature

Upgrade to access all 98 problems and unlimited AI reviews.

Tap components above to start building

Add at least 2 components, then connect them

Design Assistant

Sees your canvas

AI Design Assistant

Ask about your architecture, request suggestions, or explore trade-offs. I can see your current canvas.

LLM Serving Infrastructurehard

00:00