Choosing a Framework Desktop for Local AI: 32GB, 64GB, and 128GB

<p>The Framework Desktop comes with three memory configurations built around <a href="https://frame.work/blog/framework-desktop-deep-dive-ryzen-ai-max" target="_blank" rel="noopener">AMD&#39;s Ryzen AI Max</a> APU, codenamed Strix Halo: 32GB, 64GB, and 128GB. When Framework Desktop launched in 2025, much of the local AI community naturally focused on the 128GB version. It&#39;s easy to see why: Ryzen AI Max brought 128GB of memory shared between the CPU and integrated GPU to a desktop system, making it possible to run local models that would not fit into the memory on most consumer graphics cards.</p> <p>Fast forward to September 2026, and <a href="https://frame.work/blog/updates-on-memory-pricing-and-navigating-the-volatile-memory-market" target="_blank" rel="noopener">memory supply constraints across the industry</a> have pushed up prices, including for the Framework Desktop. Fortunately, this has coincided with the release of new, smaller open-weight models that are well suited to the 32GB and 64GB configurations, making the less expensive versions more capable and helping to offset the impact of higher memory prices. The new <a href="https://huggingface.co/unsloth/Qwen3.8-27B-GGUF" target="_blank" rel="noopener">Qwen3.8-27B</a> can run within 32GB with a suitable quantization, while models such as <a href="https://github.com/QwenLM/Qwen-Image" target="_blank" rel="noopener">Qwen Image</a> and <a href="https://huggingface.co/Comfy-Org/MiniMax-H3" target="_blank" rel="noopener">MiniMax-H3</a> bring image, video, and audio generation within reach on the same configuration.</p> <p>The 128GB configuration still gives you the widest choice of models, quantizations, and context sizes, but it is no longer the only version worth considering. This guide looks at what you can run on all three configurations, with specific model and quantization recommendations, tested Linux performance, and complete image and video workflows. The video below goes deeper into inference engines, quantization, mixture-of-experts models, and speculative decoding.</p> <iframe width="100%" class="aspect-video" src="https://www.youtube.com/embed/mmntN7zIekU?si=1AumlHm5jmZ16XG8" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe> <h3>The inference engine</h3> <p>If you&#39;re new to running LLMs locally, the first thing to choose is the inference engine. There are many engines that can load model weights and generate a response, but <a href="https://github.com/ggml-org/llama.cpp" target="_blank" rel="noopener"><strong>llama.cpp</strong></a>, created by Georgi Gerganov, has the strongest ecosystem for local inference based around the popular GGUF format for model weights.</p> <p>Popular user interfaces like <a href="https://lmstudio.ai/" target="_blank" rel="noopener">LM Studio</a> and <a href="https://ollama.com/" target="_blank" rel="noopener">Ollama</a> use llama.cpp as their backend. <a href="https://lemonade-server.ai/" target="_blank" rel="noopener">Lemonade</a> uses it as its main LLM backend and also lets you choose others.</p> <p>There are now several experimental llama.cpp forks made specifically for Strix Halo. <a href="https://buymeacoffee.com/nathanw1014" target="_blank" rel="noopener">Nathan Wilson</a>, from the <a href="https://discord.gg/pnPRyucNrG" target="_blank" rel="noopener">Strix Halo Homelab Discord</a>, has done a lot of work on a <a href="https://github.com/Nathanw1014/strix-halo-llamacpp" target="_blank" rel="noopener">Strix Halo fork</a> that improves performance on this hardware. Some of that work is now being prepared for upstream llama.cpp. Other engines target specific model families. <a href="https://github.com/antirez/ds4" target="_blank" rel="noopener">DwarfStar</a>, for example, is dedicated to DeepSeek V4 Flash. <a href="https://github.com/kyuz0/ai-toolbox-cockpit" target="_blank" rel="noopener">AI Toolbox Cockpit</a> provides pre-built Linux containers for the stable and experimental llama.cpp and DwarfStar configurations tested on the Framework Desktop.</p> <p><strong>vLLM</strong> is another option here, but it is more geared towards discrete GPUs and data centres. It is designed to serve many users at once, using a different architecture to maximise throughput across concurrent requests. It can run on Strix Halo, but at the low concurrency of a personal AI server like the Framework Desktop, llama.cpp usually performs better. vLLM is also not supported on as many types of hardware as llama.cpp.</p> <p></p> <h3>Which memory configuration should you choose?</h3> <p>These are my recommendations for the different Framework Desktop configurations as of August 2026. Most use Dynamic GGUF quantizations from <a href="https://unsloth.ai/" target="_blank" rel="noopener">Unsloth</a>.</p> <iframe title="Which memory configuration should you choose?" height="250" src="https://docs.google.com/spreadsheets/d/e/2PACX-1vQN6cwX8kNFHlPD1hQVaUNuPwNKZQqD3c3wBu0JBYaaSQpP8-SONWLdkHucVMmNalTwwXMLIPezvNYi/pubhtml?gid=0&single=true&amp;range=A1:D4&amp;single=true&amp;widget=false&amp;chrome=false&amp;headers=false"></iframe> <h4>32GB</h4> <p>For the 32GB Framework Desktop, my current recommendation is <strong>Qwen3.8-27B</strong> with the Unsloth Dynamic four-bit quantization. In my <a href="https://kyuz0.github.io/terminal-bench-mini/" target="_blank" rel="noopener">Terminal Bench Mini</a> tests, it has been the strongest model at this size for coding and agentic work. The 17.6GB GGUF also leaves enough memory for the operating system, the runtime, and a useful context window. Not every workload needs a dense 27B model, however. For simpler tasks, my recommendation is <strong>Qwen3.6-35B-A3B</strong>. It is a mixture-of-experts model and, at this size, offers the best compromise between speed and broad capability across different domains, including coding and agentic work.</p> <h4>64GB</h4> <p>With 64GB, you can run <strong>Qwen3.8-27B</strong> or <strong>Qwen3.6-35B-A3B</strong> at higher precision and still leave room for context. You can also run a larger model such as <strong>Qwen3.5-122B-A10B</strong>. Its 52.5GB Q3 file fits, but leaves less memory for context than the smaller models. This is the main choice at 64GB: a larger model, higher precision, more context, or more than one model loaded at once.</p> <h4>128GB</h4> <p>For the 128GB configuration, my current recommendation is <strong>DeepSeek V4 Flash 0731</strong> with the 90.9GB Unsloth Dynamic two-bit GGUF tested here. In my <a href="https://kyuz0.github.io/terminal-bench-mini/" target="_blank" rel="noopener">Terminal Bench Mini</a> tests, it has been the best option at this size for larger coding and agentic workloads, and there is still room for the KV cache and the runtime.</p> <p><strong>Inkling-Small</strong> remains another option if you need multimodal input, because it accepts text, images, and audio. Its 82.3GB two-bit file leaves more memory for context, while the 107GB and 119GB three-bit files use that memory for higher precision instead.</p> <h4>Qwen3.8-Flash-Next</h4> <p><a href="https://huggingface.co/Qwen/Qwen3.8-Flash-Next" target="_blank" rel="noopener"><strong>Qwen3.8-Flash-Next</strong></a> is a new alternative for the 128GB Framework Desktop. It has a 125-billion-parameter main model, with 6 billion parameters active for each token, plus a 51-billion-parameter n-gram embedding table. Its lower active parameter count may make it faster on Strix Halo. Support was merged into upstream <a href="https://github.com/ggml-org/llama.cpp/pull/27742" target="_blank" rel="noopener">llama.cpp</a> on 27 August 2026. This is still an early implementation, so local prompt-processing, token-generation, and coding-agent results are pending. On the 128GB Framework Desktop, both Unsloth&#39;s 78.9GB UD-Q2_K_XL and 82GB UD-IQ3_XXS GGUFs fit.</p> <p>In theory, a 2-bit quant should also fit on the 64GB Framework Desktop. The complication is the model&#39;s additional 51-billion-parameter n-gram table, a large lookup table for pairs and triples of nearby tokens. It gives the model more capacity with little extra computation, but adds a large amount of data that must be available during inference. Qwen designed this table so it can stay in main memory or even on disk, with the rows it needs fetched while the model continues to run. That is useful on a discrete GPU, where the table can stay in normal system RAM instead of consuming GPU VRAM. On Strix Halo, the GPU and system RAM are the same unified pool, so keeping the table in main memory does not directly solve the 64GB fit. With the right software support, it should be possible to keep the n-gram table on disk. I will test that configuration and update this article soon.</p> <h4>GLM-5.3-Flash</h4> <p><a href="https://huggingface.co/zai-org/GLM-5.3-Flash" target="_blank" rel="noopener"><strong>GLM-5.3-Flash</strong></a> is a 321-billion-parameter mixture-of-experts model with 18 billion active parameters per token. On the 128GB Framework Desktop, it is limited to two-bit and three-bit quantizations. <a href="https://github.com/ggml-org/llama.cpp/pull/27754" target="_blank" rel="noopener">llama.cpp support for GLM-5.3-Flash</a> is still a draft pull request. I will not recommend it for Strix Halo until the implementation and local tests have settled.</p> <h4>DeepSeek V4 Flash and DwarfStar</h4> <p>For DeepSeek V4 Flash, I have also worked on <a href="https://github.com/antirez/ds4" target="_blank" rel="noopener">DwarfStar</a>, also known as ds4, a dedicated inference engine for this model family. In my current Strix Halo testing, it is the best-performing engine for DeepSeek V4 Flash. The tested build is available through <a href="https://github.com/kyuz0/ai-toolbox-cockpit" target="_blank" rel="noopener">AI Toolbox Cockpit</a>.</p> <p></p> <h3>Making memory available to models</h3> <p>Because Strix Halo uses unified memory, not all of the installed memory can be used for inference. The operating system, inference engine, and other software also need memory. You can statically allocate a fixed amount of memory to the GPU, but I recommend letting the GPU dynamically expand into unified memory that is not being used by the system. Framework&#39;s <a href="https://frame.work/blog/using-a-framework-desktop-for-local-ai" target="_blank" rel="noopener">local AI guide</a> explains how to configure this on Windows. For Linux, follow my <a href="https://strix-halo-toolboxes.com/#host-config" target="_blank" rel="noopener">Strix Halo Toolboxes host configuration</a>.</p> <p></p> <h3>How fast will it run?</h3> <p>A model that fits also needs to run fast enough for the work you want to do. Strix Halo can hold large models in unified memory, but its 256GB/s memory bandwidth limits how quickly the GPU can read the weights. This is why mixture-of-experts models are often a better choice when speed matters. All the weights still have to fit in memory, but the model uses only some of its experts for each token, so it has fewer active parameters than a dense model of a similar size.</p> <p>Qwen3.6-35B-A3B is a good example. It has 35 billion parameters in total but uses only about 3 billion for each token, while the dense Qwen3.8-27B uses all 27 billion. That is why Qwen3.6 can run much faster even though its GGUF is larger. Qwen3.5-122B-A10B works in the same way, with 122 billion total parameters and 10 billion active. This does not mean that every mixture-of-experts model is better than a dense model. Training and architecture still matter, and a faster model can perform worse on the task. On Strix Halo, prefer mixture-of-experts models when inference speed is the priority.</p> <p>Two figures tell you how fast a model runs. <strong>Prompt processing</strong>, also called <strong>prefill</strong>, is how quickly the model reads the context before it starts responding. <strong>Token generation</strong>, also called <strong>decode</strong>, is how quickly it produces the response. Which figure matters most depends on the workload. A short chat prompt needs little prompt processing, so token generation has more effect on how fast it feels. A coding agent repeatedly reads source files and command output before generating code, so both figures matter.</p> <h4>Linux inference performance in August 2026</h4> <p>These results use concurrency one, a 2,048-token prompt, and 128 generated tokens. The figures are measured at the start of context.</p> <iframe title="Linux performance" height="300" width="800" src="https://docs.google.com/spreadsheets/d/e/2PACX-1vSiKk5rwTp_M50b41-nYv_4t9IzvbfqMW0Cyg-jiqnSwJtw1WVnt6-8WxGLcCX3nKasbvMeUg0U_Dog/pubhtml?gid=0&amp;widget=false&amp;chrome=false&amp;headers=false"></iframe> <p><strong>ROCm and Vulkan in llama.cpp:</strong> The rows labelled llama.cpp use the official<a href="https://github.com/ggml-org/llama.cpp" target="_blank" rel="noopener"> llama.cpp</a> project, with either the ROCm or Vulkan GPU backend. ROCm is AMD&#39;s compute platform; Vulkan uses the RADV driver. In practice, both run the model on the GPU, but their performance can differ by model and quantization. The Vulkan RADV-performance containers used here are maintained by <a href="https://buymeacoffee.com/nathanw1014" target="_blank" rel="noopener">Nathan Wilson</a>.</p> <p><strong>Strix Halo-specific builds and quantizations:</strong> There are also experimental llama.cpp forks and quantizations built for Strix Halo. <a href="https://github.com/charlie12345/ROCmFPX" target="_blank" rel="noopener">Carlo&#39;s ROCmFPX</a> fork includes Q4_0_ROCMI4, which targets a four-bit matrix-multiplication path specific to Strix Halo. It trades some precision for speed, but the result above shows the potential benefit. The ROCm, Vulkan RADV-performance, and ROCmFPX configurations are all available as pre-built containers in the <a href="https://github.com/kyuz0/ai-toolbox-cockpit" target="_blank" rel="noopener">AI Toolbox Cockpit</a> for Strix Halo.</p> <p>As you can see, Qwen3.6-35B-A3B is much faster because it is a mixture-of-experts model, making it a better choice for most everyday tasks.</p> <h3>Speculative decoding</h3> <p>Speculative decoding can make <strong>token generation</strong> faster by predicting several next tokens before the main model checks them. When those predictions are accepted, the model has produced several tokens in roughly the time it would normally take to produce one. I tested the Qwen models using their multi-token prediction (MTP) heads to draft three tokens at a time, and DeepSeek V4 Flash with DSpark.</p> <iframe title="Speculative decoding" height="250" width="800" src="https://docs.google.com/spreadsheets/d/e/2PACX-1vQ5iStxlgygW6jSDUMN95yQwFy1FxS8kMK1_jz4K9ONJ8duLVC14AeTfR6Se2vw7BOs7H71jtWC99an/pubhtml?gid=0&amp;widget=false&amp;chrome=false&amp;headers=false"></iframe> <p>These tests used the same 2,048-token prompt and generated 128 tokens at concurrency one. The table reports generation speed measured by the client at the start of context. The improvement varies with the model, prompt, and runtime configuration.</p> <h3>Where to find updated results</h3> <p>The figures in this article are an August 2026 snapshot, but I will keep the results for these models updated at <a href="https://local-llm-benchmarks.dev/" target="_blank" rel="noopener">Local LLM Benchmarks</a>. The site lets you choose the exact model, quantization, backend, hardware, and concurrency you want to compare. It shows prompt-processing and token-generation speed at increasing context depths, so you can see how performance changes as the context grows instead of relying on one result from the start of a session.</p> <p>Speed does not tell you how well a model does the work. <a href="https://kyuz0.github.io/terminal-bench-mini/" target="_blank" rel="noopener">Terminal Bench Mini</a> tracks how local models perform on agentic terminal tasks, so you can compare task performance as well as inference speed. Qwen3.8-Flash-Next is now upstream in llama.cpp, while GLM-5.3-Flash support is still in development. I expect to add both models to these sites over the next week or so, once the implementations are stable enough to test. I will publish a follow-up video comparing their inference speed and coding-agent results.</p> <h3>What about image and video generation?</h3> <p>The 32GB Framework Desktop can also run local image generation, image editing, and short video generation with audio. For these workloads, <a href="https://www.comfy.org/" target="_blank" rel="noopener">ComfyUI</a> is the main engine. It is a visual application for building and running image and video workflows locally.</p> <p>These workflows require several sets of models, such as the main diffusion model, text encoder, VAE (variational autoencoder), and sometimes a LoRA (low-rank adaptation). The main operation in image and video generation is denoising: the model starts with noise and refines it over a series of denoising steps. A workflow can take 20 to 50 steps, and every step runs the main diffusion model, which makes this work slow and expensive. A LoRA is a small set of extra weights applied on top of the main model to change how it generates. Some are trained to approximate a longer denoising schedule in fewer steps. Here, the <a href="https://huggingface.co/lightx2v/Qwen-Image-2512-Lightning" target="_blank" rel="noopener">Qwen Lightning LoRAs</a> reduce generation from 20 denoising steps to four. The <a href="https://huggingface.co/larryvrh/MiniMax-H3-Turbo-Lora" target="_blank" rel="noopener">MiniMax-H3 Turbo LoRA</a> is a separate adapter built for MiniMax-H3: it lets the model generate video and audio in four to eight steps instead of the usual 20, with six to eight giving better results. Reducing the number of steps is particularly useful on hardware like Strix Halo.</p> <p>The table below maps the complete configurations I recommend for the 32GB configuration, including every GGUF and LoRA needed for each one. You can install the pre-built Strix Halo ComfyUI container, including these workflows, through the <a href="https://github.com/kyuz0/ai-toolbox-cockpit" target="_blank" rel="noopener">AI Toolbox Cockpit</a>.</p> <iframe title="Image and video generation" height="375" width="800" src="https://docs.google.com/spreadsheets/d/e/2PACX-1vRVHNhslz9OmvoXINRztIcX0hs9snUMQYMhdEG3h5UUJ3Lp9p03FoO9hT28qX3HsCgAioaOBVaKtHnn/pubhtml?gid=0&amp;widget=false&amp;chrome=false&amp;headers=false"></iframe> <p>Resolution, frame count, and the number of denoising steps still affect memory use. The <a href="https://github.com/kyuz0/amd-strix-halo-comfyui-toolboxes" target="_blank" rel="noopener">Strix Halo ComfyUI toolbox</a> includes the workflows and the download links for every GGUF, VAE, text encoder, projector, and LoRA in the table. With 64GB, you can use higher-precision versions of the same models for better output quality.</p> <h3>Conclusion</h3> <p>Although the 128GB configuration is still one of the best options, it is no longer the only configuration worth considering for local AI, especially in light of the August 2026 releases of extremely capable small models. The 32GB version can run useful models for chat, coding, agentic work, image generation, and video generation, while 64GB opens the door to larger models, higher-precision quantizations, and more context. Choose the configuration around the work you want to do, rather than assuming that local AI requires 128GB.</p>