Open-Weight AI Models Now Trail Closed Rivals by About Four Months, Making Local Use Viable

Open-weight artificial intelligence models now lag the strongest closed systems by roughly four months on average, according to Decrypt, and many of them can run on consumer hardware that people already own.

That narrowing gap, combined with free desktop software, means a capable chatbot no longer necessarily requires a monthly subscription or the transfer of personal conversations to a third party's servers.

What open weights are and why they matter

A model's weights are the files that store everything it absorbed during training. When a developer publishes them, anyone can download the model and run it on their own machine rather than reaching it through a company's website or paid interface.

The practical significance is that the intelligence sits locally. Once the files are on a computer, the model does not need to communicate with an outside service to answer questions. Two free programmes, Ollama and LM Studio, handle the job of loading those files and presenting a chat interface.

The download sizes are manageable for many users. Decrypt reported that versions of Google's Gemma 4 in Ollama's library range from about 4.6GB to 20GB, comparable to a large video game or a few high-definition films.

A four-month lag is short by the standards of most technology markets. It suggests that the capabilities users pay for today in cloud services tend to appear in freely downloadable form within a matter of months, which changes the calculation for individuals and organisations weighing whether to rent or own their AI tools.

Privacy and cost as the main draws

The clearest argument for running a model locally concerns data. A cloud chatbot transmits every prompt to the provider's infrastructure, where it may be retained. Decrypt noted that a court ordered OpenAI to preserve ChatGPT logs, including conversations users had deleted, illustrating how little control people have once their data leaves their device.

A local model keeps the exchange on the user's own computer. LM Studio's documentation states that the app "can operate entirely offline" once model files have been downloaded, and that documents attached to a conversation can also be processed without an internet connection. For professionals handling sensitive material, that distinction may matter more than raw performance.

Cost is the second factor. Paid chatbot subscriptions typically run at about $20 a month, whereas the software described here is free. The expense does not vanish, however; it moves to hardware. Users with a recent computer may already have enough capacity, while others would face an upfront outlay to match the performance of cloud services.

What the hardware needs to handle

Models are measured in parameters, the internal values tuned during training, and larger models require bigger downloads and more memory. The model must fit within a graphics card's video memory, or VRAM, or within the unified memory of an Apple Silicon Mac, where the processor and graphics chip draw on a shared pool.

As a broad guide, Decrypt suggests at least 8GB of VRAM, enough for smaller, compressed models. More demanding users are pointed towards gaming graphics cards with 24GB of VRAM or more, combinations of several cards, or dedicated machines with unified memory such as the Mac Mini or Nvidia's DGX Spark.

Compression is what makes many of these models practical on ordinary hardware. The technique, known as quantisation, stores each weight using fewer bits. Hugging Face's documentation describes replacing the standard 32-bit floating-point format with lower-precision formats such as 8-bit integers. The result is a smaller memory footprint, at the price of some loss in accuracy.

LM Studio publishes its own requirements:

• Apple Silicon Macs on macOS 14 or later, with 16GB of RAM or more recommended, though 8GB machines can run smaller models
• Windows PCs with at least 16GB of RAM, 4GB of dedicated VRAM and a processor supporting AVX2 instructions
• Intel-based Macs are not supported

Specific models give a sense of scale. Ollama's library lists Gemma 4 26B at between 16GB and 19GB and the 31B version at 19GB to 20GB. OpenAI has said its gpt-oss-20b model needs only 16GB of memory, while the much larger gpt-oss-120b runs within 80GB, a figure that places it beyond most home setups.

Choosing between Ollama and LM Studio

The two leading tools suit different audiences. LM Studio offers a polished graphical interface and is likely the more accessible starting point for non-technical users. Installation involves downloading the app from the developer's website, searching for a model within it and opening a chat once the weights have downloaded. The search draws on Hugging Face, the public repository where most open models are published.

Ollama is geared towards developers and those comfortable working in a command-line terminal. It installs via a single command on macOS, Linux or Windows, after which a user can fetch a model with a pull command and start a conversation with a run command. Ollama also exposes a local programming interface on the user's machine, allowing other applications on the same computer to call the model, which makes it useful as a building block for custom tools.

There is a caveat for privacy-minded users. Model names in Ollama ending in ":cloud" run on Ollama's own servers rather than locally and require no download. These defeat the purpose for anyone seeking to keep data on their device. Decrypt nonetheless describes them as a reasonable fallback for those needing more powerful models than their hardware allows, reporting that prompts handled this way are not stored on the providers' servers or used for training.

Performance improvements are also arriving. Google has released Multi-Token Prediction drafters for Gemma 4, available through Ollama under the Apache 2.0 licence, which Decrypt said can speed up text generation by as much as three times. The method, called speculative decoding, has a small model propose several tokens in advance while the larger model verifies them in one pass.

Licences and limitations

Downloading weights does not automatically confer the right to do anything with them; the licence governs that. DeepSeek's R1 series is released under the MIT Licence, and its model card says the models permit commercial use, modification and derivative works. Google's Gemma 4 and OpenAI's gpt-oss are released under Apache 2.0, another permissive licence.

These permissive terms allow users and developers to adapt models for narrow purposes. A model fine-tuned for medical questions, for instance, need not be strong at writing code. That kind of specialisation is harder to achieve with closed services, where the provider decides what the model does and how.

The trade-offs remain real. Local models are, on Decrypt's measure, still several months behind the leading closed systems, and quantised versions sacrifice some accuracy to fit into memory. Users with older or lower-specification machines may find themselves limited to smaller models, and those wanting to run the largest open models face hardware costs that could exceed years of subscription fees. Responsibility for setup, updates and troubleshooting also shifts from the provider to the user.

What to watch

The key variable is whether the gap between open and closed models continues to shrink. If it does, the case for paying for cloud access weakens for many everyday tasks, particularly where confidentiality matters.

Hardware is the other factor. Machines with large pools of unified memory and graphics cards with generous VRAM are becoming more central to how individuals use AI, and techniques such as quantisation and speculative decoding are lowering the threshold for what counts as sufficient.

For now, the barrier to entry is modest: a free application, a download of a few gigabytes and a computer with reasonable memory are enough to run a capable assistant entirely offline.