{"id":617755,"date":"2023-03-14T09:49:06","date_gmt":"2023-03-14T14:49:06","guid":{"rendered":"https:\/\/news.sellorbuyhomefast.com\/index.php\/2023\/03\/14\/you-can-now-run-a-gpt-3-level-ai-model-on-your-laptop-phone-and-raspberry-pi\/"},"modified":"2023-03-14T09:49:06","modified_gmt":"2023-03-14T14:49:06","slug":"you-can-now-run-a-gpt-3-level-ai-model-on-your-laptop-phone-and-raspberry-pi","status":"publish","type":"post","link":"https:\/\/newsycanuse.com\/index.php\/2023\/03\/14\/you-can-now-run-a-gpt-3-level-ai-model-on-your-laptop-phone-and-raspberry-pi\/","title":{"rendered":"You can now run a GPT-3 level AI model on your laptop, phone, and Raspberry Pi"},"content":{"rendered":"
\n
\n

\n Pocket-sized hallucination on demand \u2014
\n<\/h4>\n

Thanks to Meta LLaMA, AI text models have their “Stable Diffusion moment.”<\/h2>\n
\n

\n Benj Edwards<\/span><\/a>
\n –

\n
\n \"An
\n

Ars Technica<\/p>\n<\/figcaption><\/figure>\n

Things are moving at lightning speed in AI Land. On Friday, a software developer named Georgi Gerganov created a tool called “llama.cpp”<\/a> that can run Meta’s new GPT-3-class AI large language model, LLaMA<\/a>, locally on a Mac laptop. Soon thereafter, people worked out how to run LLaMA on Windows<\/a> as well. Then someone showed it running<\/a> on a Pixel 6 phone, and next came a Raspberry Pi<\/a> (albeit running very slowly).<\/p>\n

If this keeps up, we may be looking at a pocket-sized ChatGPT competitor<\/a> before we know it.<\/p>\n

But let’s back up a minute, because we’re not quite there yet. (At least not today\u2014as in literally today, March 13, 2023.) But what will arrive next week, no one knows.<\/p>\n

Since ChatGPT launched<\/a>, some people have been frustrated by the AI model’s built-in limits that prevent it from discussing topics that OpenAI has deemed sensitive. Thus began the dream<\/a>\u2014in some quarters\u2014of an open source large language model (LLM) that anyone could run locally without censorship and without paying API fees<\/a> to OpenAI.<\/p>\n

Open source solutions do exist (such as GPT-J<\/a>), but they require<\/a> a lot of GPU RAM and storage space. Other open source alternatives could not boast GPT-3-level performance on readily available consumer-level hardware.<\/p>\n

Enter LLaMA, an LLM available in parameter sizes ranging from 7B to 65B (that’s “B” as in “billion parameters,” which are floating point numbers stored in matrices that represent what the model “knows”). LLaMA made a heady claim: that its smaller-sized models could match OpenAI’s GPT-3<\/a>, the foundational model that powers ChatGPT, in the quality and speed of its output. There was just one problem\u2014Meta released the LLaMA code open source, but it held back the “weights” (the trained “knowledge” stored in a neural network) for qualified researchers only.<\/p>\n

Flying at the speed of LLaMA<\/h2>\n

Meta’s restrictions on LLaMA didn’t last long, because on March 2, someone leaked the LLaMA weights<\/a> on BitTorrent. Since then, there has been an explosion of development surrounding LLaMA. Independent AI researcher Simon Willison has compared<\/a>\u00a0this situation to the release of Stable Diffusion<\/a>, an open source image synthesis model that launched last August. Here’s what he wrote in a post on his blog:<\/p>\n

\n

It feels to me like that Stable Diffusion moment back in August kick-started the entire new wave of interest in generative AI\u2014which was then pushed into over-drive by the release of ChatGPT at the end of November.<\/p>\n

That Stable Diffusion moment is happening again right now, for large language models\u2014the technology behind ChatGPT itself. This morning I ran a GPT-3 class language model on my own personal laptop for the first time!<\/p>\n

AI stuff was weird already. It\u2019s about to get a whole lot weirder.<\/p>\n<\/blockquote>\n

Typically, running GPT-3 requires several datacenter-class A100<\/a> GPUs (also, the weights for GPT-3 are not public), but LLaMA made waves because it could run on a single beefy consumer GPU. And now, with optimizations that reduce the model size using a technique called quantization, LLaMA can run on an M1 Mac or a lesser Nvidia consumer GPU (although “llama.cpp” only runs on CPU at the moment\u2014which is impressive and surprising in its own way).<\/p>\n

Things are moving so quickly that it’s sometimes difficult to keep up<\/a> with the latest developments. (Regarding AI’s rate of progress, a fellow AI reporter told Ars, “It’s like those videos of dogs where you upend a crate of tennis balls on them. [They] don’t know where to chase first and get lost in the confusion.”)<\/p>\n

For example, here’s a list of notable LLaMA-related events based on a timeline<\/a> Willison laid out in a Hacker News comment:<\/p>\n