ANALYSIS / MEASUREMENT
The fastest kernel depends on how you measure.
The measurement boundary is part of the experiment.
What changed
FlashInfer’s September 2026 Autotuner v2 separates measurement policies for eager execution and CUDA Graph replay. Device-only timing can miss recurring host costs; replay changes those costs again. A configuration that wins one measurement can lose another.
Why it matters
Kernel choice depends on more than a tensor shape. GPU, software versions, execution policy, and operation settings define whether a previous tuning result remains applicable. FlashInfer’s updated cache keeps those contexts distinct.
Treat a kernel winner as a result for a particular execution environment, not as a universal ranking.
A useful experiment
Measure the same supported configurations in eager and graph-replay modes. Select using one set of timings, then validate the choice with fresh measurements. Compare the complete serving path as well as kernel time, and report warmup overhead separately.
The limit of the claim
The published autotuner results evaluate selection quality. They do not establish an equivalent end-to-end improvement in request throughput. That requires a serving experiment on the target workload.
Go to the source
A synthesis of the linked primary sources, reviewed 16 November 2026. The research questions and proposed experiments are keplerfold’s interpretation; these notes do not report original benchmark results.