THE INFERENCE STACK
Tech Radar
A curated map of the inference stack, organized by layer.
View
Tile size
Layer
01Serving engines
4Run the model.
vLLMLLM serving with paged KV memory.Official documentation ↗
SGLangModel serving with RadixAttention and prefix reuse.Official documentation ↗
- TensorRT-LLMTensorRT-LLMNVIDIA GPU inference runtime and optimizations.Official documentation ↗
llama.cppPortable C/C++ inference across CPU and GPU.Official documentation ↗
02Kernels & compilers
3Execute the computation.
03Systems & caching
3Coordinate serving and state.
04Measurement & evaluation
3Measure speed and quality.
No projects match this search.
Curated coverage. Inclusion is not a performance ranking.
