AI-SOVEREIGN
Home / AI Research / NarrowSpec: Draft Hidden Width Need Not Match the Target in Speculative Decoding
NAAI Research

NarrowSpec: Draft Hidden Width Need Not Match the Target in Speculative Decoding

by nvidia · Alagappan Valliappan, Ankit More, Dheevatsa Mudigere, Jatin Mitra

Speculative decoding is widely used to speed up generation in large language models (LLMs), with a small draft model proposing tokens that one forward pass of the target model verifies. Draft models that read the target's hidden states, such as DFlash and EAGLE-3, typically inherit the target's hidden width. We show that the draft's whole residual stream can run at half or a quarter of that width while largely preserving acceptance. For DFlash, which borrows the target's vocabulary projection, the trained output adapter folds into it, giving one narrower projection with logits unchanged up to floating-point rounding. We test two drafter families, DFlash and EAGLE-3, on two targets,…

Upvotes5
Comments2
Published17 Nov 2026

More from nvidia