TECH RADAR / SERVING ENGINES
vLLM
LLM serving with paged KV memory.
What it covers
Scheduling, prefix caching, and distributed execution.
What we would check
Pin the runtime and compare first-token latency, output-token latency, and useful throughput on your request mix.
Go to the source
A curated ecosystem entry. The evaluation guidance is our interpretation, not a report of a completed test or a performance ranking.