{"id":863527,"date":"2025-07-19T07:12:31","date_gmt":"2025-07-19T12:12:31","guid":{"rendered":"https:\/\/newsycanuse.com\/index.php\/2025\/07\/19\/how-to-run-an-llm-on-your-laptop\/"},"modified":"2025-07-19T07:12:31","modified_gmt":"2025-07-19T12:12:31","slug":"how-to-run-an-llm-on-your-laptop","status":"publish","type":"post","link":"https:\/\/newsycanuse.com\/index.php\/2025\/07\/19\/how-to-run-an-llm-on-your-laptop\/","title":{"rendered":"How to run an LLM on your laptop"},"content":{"rendered":"
\n

MIT Technology Review\u2019s How To<\/a> series helps you get things done.\u00a0<\/em><\/strong><\/p>\n

Simon Willison has a plan for the end of the world. It\u2019s a USB stick, onto which he has loaded a couple of his favorite open-weight LLMs\u2014models that have been shared publicly by their creators and that can, in principle, be downloaded and run with local hardware. If human civilization should ever collapse, Willison plans to use all the knowledge encoded in their billions of parameters for help. \u201cIt\u2019s like having a weird, condensed, faulty version of Wikipedia, so I can help reboot society with the help of my little USB stick,\u201d he says.<\/p>\n<\/p><\/div>\n

\n

But you don\u2019t need to be planning for the end of the world to want to run an LLM on your own device. Willison, who writes a popular blog about local LLMs and software development, has plenty of compatriots: r\/LocalLLaMA<\/a>, a subreddit devoted to running LLMs on your own hardware, has half a million members.<\/p>\n

For people who are concerned about privacy, want to break free from the control of the big LLM companies, or just enjoy tinkering, local models offer a compelling alternative to ChatGPT and its web-based peers.<\/p>\n

The local LLM world used to have a high barrier to entry: In the early days, it was impossible to run anything useful without investing in pricey GPUs. But researchers have had so much success in shrinking down and speeding up models that anyone with a laptop, or even a smartphone, can now get in on the action. \u201cA couple of years ago, I\u2019d have said personal computers are not powerful enough to run the good models. You need a $50,000 server rack to run them,\u201d Willison says. \u201cAnd I kept on being proved wrong time and time again.\u201d<\/p>\n<\/p><\/div>\n

\n

Why you might want to download your own LLM<\/strong><\/h3>\n

Getting into local models takes a bit more effort than, say, navigating to ChatGPT\u2019s online interface. But the very accessibility of a tool like ChatGPT comes with a cost. \u201cIt\u2019s the classic adage: If something\u2019s free, you\u2019re the product,\u201d says Elizabeth Seger, the director of digital policy at Demos, a London-based think tank.\u00a0<\/p>\n

OpenAI, which offers both paid and free tiers, trains its models on users\u2019 chats by default. It\u2019s not too difficult to opt out of this training, and it also used to be possible to remove your chat data from OpenAI\u2019s systems entirely, until a recent legal decision in the New York Times<\/em>\u2019 ongoing lawsuit against OpenAI required the company to maintain all user conversations with ChatGPT.<\/p>\n

Google, which has access to a wealth of data about its users, also trains its models on both free and paid users\u2019 interactions with Gemini, and the only way to opt out of that training is to set your chat history to delete automatically\u2014which means that you also lose access to your previous conversations. In general, Anthropic does not train its models using user conversations, but it will train on conversations that have been \u201cflagged for Trust & Safety review.\u201d\u00a0<\/p>\n

Training may present particular privacy risks because of the ways that models internalize, and often recapitulate, their training data. Many people trust LLMs with deeply personal conversations\u2014but if models are trained on that data, those conversations might not be nearly as private as users think, according to some experts.<\/p>\n

\u201cSome of your personal stories may be cooked into some of the models, and eventually be spit out in bits and bytes somewhere to other people,\u201d says Giada Pistilli, principal ethicist at the company Hugging Face, which runs a huge library of freely downloadable LLMs and other AI resources.<\/p>\n<\/p><\/div>\n

\n

For Pistilli, opting for local models as opposed to online chatbots has implications beyond privacy. \u201cTechnology means power,\u201d she says. \u201cAnd so who[ever] owns the technology also owns the power.\u201d States, organizations, and even individuals might be motivated to disrupt the concentration of AI power in the hands of just a few companies by running their own local models.<\/p>\n

Breaking away from the big AI companies also means having more control over your LLM experience. Online LLMs are constantly shifting under users\u2019 feet: Back in April, ChatGPT suddenly started sucking up<\/a> to users far more than it had previously, and just last week Grok started calling itself MechaHitler on X.<\/p>\n

Providers tweak their models with little warning, and while those tweaks might sometimes improve model performance, they can also cause undesirable behaviors. Local LLMs may have their quirks, but at least they are consistent. The only person who can change your local model is you.<\/p>\n

Of course, any model that can fit on a personal computer is going to be less powerful than the premier online offerings from the major AI companies. But there\u2019s a benefit to working with weaker models\u2014they can inoculate you against the more pernicious limitations of their larger peers. Small models may, for example, hallucinate more frequently and more obviously than Claude, GPT, and Gemini, and seeing those hallucinations can help you build up an awareness of how and when the larger models might also lie.<\/p>\n

\u201cRunning local models is actually a really good exercise for developing that broader intuition for what these things can do,\u201d Willison says.<\/p>\n

How to get started<\/strong><\/h3>\n

Local LLMs aren\u2019t just for proficient coders. If you\u2019re comfortable using your computer\u2019s command-line interface, which allows you to browse files and run apps using text prompts, Ollama<\/a> is a great option. Once you\u2019ve installed the software, you can download and run any of the hundreds of models they offer with a single command<\/a>.\u00a0<\/p>\n

If you don\u2019t want to touch anything that even looks like code, you might opt for LM Studio<\/a>, a user-friendly app that takes a lot of the guesswork out of running local LLMs. You can browse models from Hugging Face from right within the app, which provides plenty of information to help you make the right choice. Some popular and widely used models are tagged as \u201cStaff Picks,\u201d and every model is labeled according to whether it can be run entirely on your machine\u2019s speedy GPU, needs to be shared between your GPU and slower CPU, or is too big to fit onto your device at all. Once you\u2019ve chosen a model, you can download it, load it up, and start interacting with it using the app\u2019s chat interface.<\/p>\n

As you experiment with different models, you\u2019ll start to get a feel for what your machine can handle. According to Willison, every billion model parameters require about one GB of RAM to run, and I found that approximation to be accurate: My own 16 GB laptop managed to run Alibaba\u2019s Qwen3 14B<\/a> as long as I quit almost every other app. If you run into issues with speed or usability, you can always go smaller\u2014I got reasonable responses from Qwen3 8B as well.<\/p>\n<\/p><\/div>\n

\n

And if you go really small, you can even run models on your cell phone. My beat-up iPhone 12 was able to run Meta\u2019s Llama 3.2 1B using an app called LLM Farm. It\u2019s not a particularly good model\u2014it very quickly goes off into bizarre tangents and hallucinates constantly\u2014but trying to coax something so chaotic toward usability can be entertaining. If I\u2019m ever on a plane sans Wi-Fi and desperate for a probably false answer to a trivia question, I now know where to look.<\/p>\n

Some of the models that I was able to run on my laptop were effective enough that I can imagine using them in my journalistic work. And while I don\u2019t think I\u2019ll depend on phone-based models for anything anytime soon, I really did enjoy playing around with them. \u201cI think most people probably don\u2019t need to do this, and that\u2019s fine,\u201d Willison says. \u201cBut for the people who want to do this, it\u2019s so much fun.\u201d<\/p>\n<\/p><\/div>\n

Read More<\/a>
\n Grace Huckins<\/p>\n","protected":false},"excerpt":{"rendered":"

MIT Technology Review\u2019s How To series helps you get things done.\u00a0 Simon Willison has a plan for the end of the world. It\u2019s a USB stick, onto which he has loaded a couple of his favorite open-weight LLMs\u2014models that have been shared publicly by their creators and that can, in principle, be downloaded and run<\/p>\n","protected":false},"author":1,"featured_media":863528,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2910,46],"tags":[],"class_list":["post-863527","post","type-post","status-publish","format-standard","has-post-thumbnail","category-laptop","category-technology"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/posts\/863527","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/comments?post=863527"}],"version-history":[{"count":0,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/posts\/863527\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/media\/863528"}],"wp:attachment":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/media?parent=863527"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/categories?post=863527"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/tags?post=863527"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}