3 Ways to Boost MoE Inference on NVIDIA DGX Spark

<img width="768" height="432" src="https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-768x432.png" class="webfeedsFeaturedVisual wp-post-image" alt="Abstract green computing cube with layered panels, performance icons, and streams of data on a black background" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" srcset="https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-768x432.png 768w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-179x101.png 179w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-300x169.png 300w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-625x352.png 625w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-1536x864.png 1536w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-645x363.png 645w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-660x370.png 660w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-500x281.png 500w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-160x90.png 160w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-362x204.png 362w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-196x110.png 196w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-1024x576.png 1024w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-960x540.png 960w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured.webp 1920w" sizes="(max-width: 768px) 100vw, 768px" title="dgx-spark-inference-featured" />Generating tokens faster starts with identifying what limits inference performance. In the batch-size-1 evaluation of Ling-3.0-flash from Inclusion AI on NVIDIA...<img width="768" height="432" src="https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-768x432.png" class="webfeedsFeaturedVisual wp-post-image" alt="Abstract green computing cube with layered panels, performance icons, and streams of data on a black background" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" link_thumbnail="" decoding="async" srcset="https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-768x432.png 768w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-179x101.png 179w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-300x169.png 300w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-625x352.png 625w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-1536x864.png 1536w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-645x363.png 645w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-660x370.png 660w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-500x281.png 500w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-160x90.png 160w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-362x204.png 362w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-196x110.png 196w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-1024x576.png 1024w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured-960x540.png 960w, https://developer-blogs.nvidia.com/wp-content/uploads/2026/10/dgx-spark-inference-featured.webp 1920w" sizes="(max-width: 768px) 100vw, 768px" title="dgx-spark-inference-featured" /><p>Generating tokens faster starts with identifying what limits inference performance. In the batch-size-1 evaluation of Ling-3.0-flash from Inclusion AI on NVIDIA DGX Spark, profiling showed that decoding was memory-bound: reading model weights drove execution time. Two operations stood out—the mixture-of-experts (MoE) computation and the output-layer projection that produces scores across the model’…</p> <p><a href="https://developer.nvidia.com/blog/3-ways-to-boost-moe-inference-on-nvidia-dgx-spark/" rel="nofollow" data-wpel-link="internal" target="_self">Source</a></p>