LLMs are getting smarter, but they still generate one token at a time. 🐢

For agentic workloads involving long reasoning, tool calls, and code generation, that sequential process results in higher latency, higher compute costs, and lower GPU efficiency.

Speculative decoding helps by letting a smaller drafter propose several tokens for the target model to verify together. DFlash went further by drafting an entire block in parallel.

The remaining challenge is coherence. Each prediction may look right on its own, but the complete sequence may not fit together.