Verdict
If your model has a trained EAGLE-3 head available and you serve at low-to-mid concurrency, EAGLE-3 is the clear winner: 2.89x in the independent 500-prompt production benchmark at youngju.dev versus 2.21x for Medusa-2 and 1.95x for a vanilla draft model, and up to 2.5x in Red Hat's vLLM testing. Medusa has quietly lost the ecosystem war - it is no longer listed among the supported methods on vLLM's current speculative decoding docs page. But the method choice is secondary to the go/no-go math: practitioner guidance (Introl, Red Hat) converges on speculation turning net negative below roughly 50% acceptance, healthy at 60-80%, and the serving-latency literature (arXiv 2406.14066) measures gains eroding and then reversing as request rate saturates the GPU. Prompt-lookup decoding - no trained head, no draft model - wins the zero-effort row outright for retrieval- and edit-heavy workloads, and 'no speculation' is the correct configuration for saturated high-QPS serving.
The methods differ in where draft tokens come from, and that determines both ceiling and operational cost. A vanilla draft model runs a second, smaller LLM (1.95x speedup, 0.62 accept ratio in the youngju.dev benchmark) - effective but memory-hungry and a second artifact to align. Medusa-2 bolts extra decoding heads onto the target model (2.21x, 0.68) - lighter, but its acceptance ceiling shows and current vLLM docs no longer list it as a supported method (docs.vllm.ai/en/latest/features/speculative_decoding). EAGLE predicts from the target model's own feature layer rather than raw tokens: EAGLE-1 reaches 2.52x/0.73, and EAGLE-3's feature-level drafting scales further to 2.89x/0.81 in the same harness - the official EAGLE-3 paper (arXiv 2503.01840) claims 3.0-6.5x versus vanilla decoding with acceptance lengths up to ~7.5 tokens on HumanEval, and the SafeAILab/EAGLE repo reports roughly 1.6x over Medusa in its own comparisons. Prompt-lookup (n-gram) drafts by copying from the prompt itself: only 1.38x/0.41 on general chat, but it costs nothing to enable and vLLM's docs explicitly recommend n-gram-style methods when you must avoid adding drafting load at peak traffic. Red Hat's vLLM work measured EAGLE-3 at up to 2.5x across scenarios with per-request latency down 1.8x (8B) and 1.6x (70B) at low request rates (developers.redhat.com, 'Fly, Eagle(3), fly'), with gains shrinking as request rates climb into compute saturation - the same pattern the arXiv serving study quantifies: past a request rate of ~12 req/s in their setup, proposing 5 tokens stopped helping, and past ~16, even 3-token proposals degraded performance.
Decision Table
| Criterion | Edge | Explanation |
|---|---|---|
| Raw speedup on general workloads | EAGLE (EAGLE-1/EAGLE-3 feature-level speculators) Best edge | The independent 500-prompt production benchmark (youngju.dev, 'LLM Serving: Speculative Decoding Production Benchmark 2026') puts EAGLE-3 at 2.89x and EAGLE-1 at 2.52x versus Medusa-2's 2.21x and a vanilla draft model's 1.95x, with accept ratios 0.81/0.73 vs 0.68/0.62. The official EAGLE-3 paper (arXiv 2503.01840) claims 3.0-6.5x with the largest gains on code (acceptance length up to ~7.5 on HumanEval), and the EAGLE repo's own numbers show ~1.6x over Medusa. Feature-level drafting simply accepts more tokens per step than token-level heads. |
| Setup cost: training and artifacts required | Tie | The third option wins this row outright: prompt-lookup/n-gram decoding requires no trained head, no draft model, and no new weights - one config flag in vLLM. Every other method has a real bill: EAGLE needs a trained speculator head for your exact target model (fine if one exists for your Llama/gpt-oss variant, a training project if not), Medusa needs its heads trained, and a draft model must share tokenizer and behavior with the target - pairing mismatches quietly destroy acceptance rates. If you cannot invest in training or find a published head, prompt-lookup is the only same-day option. |
| Ecosystem support in current serving stacks | EAGLE (EAGLE-1/EAGLE-3 feature-level speculators) Best edge | vLLM's current speculative decoding docs list EAGLE, MTP, draft models, PARD, MLP speculators, n-gram, and suffix decoding - Medusa is absent from the supported-methods list (docs.vllm.ai/en/latest/features/speculative_decoding). Red Hat's production writeups center EAGLE-3 for vLLM (up to 2.5x, 'Fly, Eagle(3), fly', developers.redhat.com), and SGLang likewise ships EAGLE-family speculation as its headline method. Betting on Medusa in 2026 means betting against where the maintained fast paths are. |
| Memory and deployment footprint | EAGLE (EAGLE-1/EAGLE-3 feature-level speculators) Best edge | A vanilla draft model costs a full second model's VRAM plus its KV cache - real capacity you could have spent on batch size. EAGLE and Medusa heads are small attachments to the target model, and EAGLE's single-layer feature predictor is the lightest path to high acceptance. On memory-constrained single-GPU deployments the draft-model approach is usually disqualified before benchmarks even enter the discussion. |
| Retrieval-heavy and edit-heavy workloads (RAG, code editing, summarization) | Tie | Third option again: when outputs copy long spans from the input - RAG answers quoting context, code edits echoing existing code - prompt-lookup's n-gram matching drafts those spans for free, and its economics beat its 1.38x general-chat average precisely where trained speculators gain least. vLLM's docs make the same operational point: simpler methods like n-gram and suffix decoding provide modest speedups without increasing drafting workload during peak traffic, which model-based speculators do. |
| High-concurrency, throughput-bound serving | Tie | Nobody wins this row - the correct answer is often no speculation at all. The serving-latency study (arXiv 2406.14066) measured speculation's benefit eroding as request rate rises: past ~12 req/s proposing 5 tokens gave no improvement, and past ~16 req/s even 3-token proposals degraded performance; Red Hat's guidance similarly targets 60-80% acceptance and flags saturated GPUs as the regime to avoid, since draft-plus-verify work competes with batch throughput. Red Hat's gpt-oss EAGLE-3 testing shows gains can persist to higher concurrency in favorable setups, but the burden of proof is on your own load test - enabling a speculator on a saturated cluster can make it slower. |
Choose EAGLE (EAGLE-1/EAGLE-3 feature-level speculators) if...
- Production vLLM/SGLang deployments of Llama-class or gpt-oss models where a trained EAGLE-3 speculator is published or trainable.
- Latency-sensitive chat and agent serving at low-to-mid concurrency - the 1.8x (8B) and 1.6x (70B) per-request latency cuts in Red Hat's testing landed at low request rates.
- Code-generation workloads: EAGLE-3's largest published gains (up to 6.5x, acceptance length ~7.5) are on HumanEval-style tasks (arXiv 2503.01840).
- Memory-constrained single-GPU serving where a second draft model's VRAM is unaffordable but a small speculator head is fine.
Choose Medusa-2 and vanilla draft-model speculation if...
- Teams with an existing, well-aligned small model in the same family who want speculation without training anything new (vanilla draft-model route).
- Stacks pinned to older serving engines that still ship Medusa/draft-model paths and cannot move to current vLLM.
- Cross-model setups where one draft model serves several target variants - a flexibility trained per-model heads do not have.
- Research and ablation work where Medusa's simple multi-head design remains a useful baseline (it is still the reference point EAGLE papers compare against).
Decision rules
- 01First decide IF, then which: measure acceptance rate on your real traffic before committing. Below ~0.5 acceptance, speculation is net negative (Introl practitioner guidance); Red Hat's target band is 60-80%. No method choice rescues a workload that drafts badly.
- 02Check your serving regime: latency-bound at 1-10 concurrent requests is speculation's sweet spot; if your GPUs run saturated and throughput-bound, run no speculator - the arXiv 2406.14066 measurements show proposals actively degrading performance past ~16 req/s in their setup.
- 03If a published EAGLE-3 head exists for your model family (Llama-class, gpt-oss), take it: 2.89x independent, up to 2.5x in Red Hat's vLLM testing, best accept ratios, first-class vLLM support.
- 04If no trained head exists and you cannot train one, enable prompt-lookup today (zero artifacts) rather than shipping a mismatched draft model - a bad tokenizer pairing yields acceptance rates that make speculation a slowdown.
- 05If your workload is RAG or code-edit heavy, benchmark prompt-lookup before paying for any trained speculator - n-gram drafting from the prompt wins exactly where your outputs quote their inputs.
- 06Do not build new deployments on Medusa: it is absent from vLLM's current supported-methods docs, and the independent numbers (2.21x vs EAGLE-3's 2.89x) give no reason to swim against the ecosystem.
Migration Notes
- Moving draft-model → EAGLE-3 in vLLM is a config change plus the speculator weights - but re-run your acceptance-rate measurement on production traffic, not benchmark prompts; domain shift between benchmark and your traffic is the main reason measured speedups disappoint.
- Leaving Medusa: current vLLM docs no longer list it, so treat any engine upgrade as a forced migration - EAGLE-3 is the like-for-like replacement, and the youngju.dev harness suggests roughly a further +0.7x over Medusa-2 if acceptance holds.
- Instrument before and after: log acceptance rate, accepted length, and p95 TPOT per workload class. Speculation is a bet that re-prices with every model, prompt-template, and traffic change - a quarterly re-check is cheap insurance against silently running a net-negative speculator.