ReplaySSM: Rethinking State Storage for Faster Hybrid-Model Decoding

<img width="768" height="432" src="https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-768x432.png" class="webfeedsFeaturedVisual wp-post-image" alt="A cube assembled from smaller dark blocks, with glowing cyan, pink, yellow, and green sections against a black background." style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" srcset="https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-768x432.png 768w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-179x101.png 179w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-300x169.png 300w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-625x352.png 625w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-1536x864.png 1536w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-645x363.png 645w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-660x370.png 660w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-500x281.png 500w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-160x90.png 160w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-362x204.png 362w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-196x110.png 196w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-1024x576.png 1024w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-960x540.png 960w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background.webp 1920w" sizes="(max-width: 768px) 100vw, 768px" title="cube-black-background" />Hybrid models interleave state-space model (SSM) layers with a few attention layers. NVIDIA Nemotron 3, Qwen3.5 through Qwen3.8, Kimi-K3, and GLM-5.3-Flash are...<img width="768" height="432" src="https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-768x432.png" class="webfeedsFeaturedVisual wp-post-image" alt="A cube assembled from smaller dark blocks, with glowing cyan, pink, yellow, and green sections against a black background." style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" loading="lazy" srcset="https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-768x432.png 768w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-179x101.png 179w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-300x169.png 300w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-625x352.png 625w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-1536x864.png 1536w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-645x363.png 645w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-660x370.png 660w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-500x281.png 500w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-160x90.png 160w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-362x204.png 362w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-196x110.png 196w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-1024x576.png 1024w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background-960x540.png 960w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/06/cube-black-background.webp 1920w" sizes="auto, (max-width: 768px) 100vw, 768px" title="cube-black-background" /><p>Hybrid models interleave state-space model (SSM) layers with a few attention layers. NVIDIA Nemotron 3, Qwen3.5 through Qwen3.8, Kimi-K3, and GLM-5.3-Flash are all built this way, with Mamba-2, Gated DeltaNet (GDN), or Kimi Delta Attention (KDA) as the SSM layer. SSM layers are designed to make decoding fast: they use a fixed-size state in place of a key-value (KV) cache that grows with…</p> <p><a href="https://developer.nvidia.com/blog/replayssm-rethinking-state-storage-for-faster-hybrid-model-decoding/" rel="nofollow" data-wpel-link="internal" target="_self">Source</a></p>