{"id":822903,"date":"2025-01-29T12:12:54","date_gmt":"2025-01-29T18:12:54","guid":{"rendered":"https:\/\/newsycanuse.com\/index.php\/2025\/01\/29\/setting-up-nvidia-jetson-nano-install-pytorch-and-start-working-with-llms\/"},"modified":"2025-01-29T12:12:54","modified_gmt":"2025-01-29T18:12:54","slug":"setting-up-nvidia-jetson-nano-install-pytorch-and-start-working-with-llms","status":"publish","type":"post","link":"https:\/\/newsycanuse.com\/index.php\/2025\/01\/29\/setting-up-nvidia-jetson-nano-install-pytorch-and-start-working-with-llms\/","title":{"rendered":"Setting Up Nvidia Jetson Nano, Install Pytorch and Start Working With LLMs"},"content":{"rendered":"<article id=\"post-200632\" itemtype=\"https:\/\/schema.org\/CreativeWork\" itemscope>\n<div itemprop=\"text\">\n<p>Youtuber Ominous Industries provides a training video for setting up your Jetson Nano, installing Pytorch and setting up your own (likely open source LLM). <\/p>\n<p><img loading=\"lazy\" decoding=\"async\" alt width=\"570\" height=\"538\"   data-eio=\"l\" data-old-src=\"data:image\/png;base64,iVBORw0KGgoAAAANSUhEUgAAAjoAAAIaAQAAAAABn9pbAAAAAnRSTlMAAHaTzTgAAAA9SURBVHja7cExAQAAAMKg9U9tB2+gAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABeA5lqAAELRcm4AAAAAElFTkSuQmCC\" src=\"https:\/\/nextbigfuture.s3.amazonaws.com\/uploads\/2025\/01\/Screenshot-2025-01-27-at-9.47.16-AM.jpg\" srcset=\"https:\/\/nextbigfuture.s3.amazonaws.com\/uploads\/2025\/01\/Screenshot-2025-01-27-at-9.47.16-AM.jpg 570w, https:\/\/nextbigfuture.s3.amazonaws.com\/uploads\/2025\/01\/Screenshot-2025-01-27-at-9.47.16-AM-300x283.jpg 300w\"><br \/>\n<img loading=\"lazy\" decoding=\"async\" alt width=\"418\" height=\"369\"   data-eio=\"l\" data-old-src=\"data:image\/png;base64,iVBORw0KGgoAAAANSUhEUgAAAaIAAAFxAQAAAAAZ2pFDAAAAAnRSTlMAAHaTzTgAAAAqSURBVHja7cExAQAAAMKg9U9tB2+gAAAAAAAAAAAAAAAAAAAAAAAAAPgNTdYAAXe7CRwAAAAASUVORK5CYII=\" src=\"https:\/\/nextbigfuture.s3.amazonaws.com\/uploads\/2025\/01\/Screenshot-2025-01-27-at-9.48.06-AM.jpg\" srcset=\"https:\/\/nextbigfuture.s3.amazonaws.com\/uploads\/2025\/01\/Screenshot-2025-01-27-at-9.48.06-AM.jpg 418w, https:\/\/nextbigfuture.s3.amazonaws.com\/uploads\/2025\/01\/Screenshot-2025-01-27-at-9.48.06-AM-300x265.jpg 300w\"><\/p>\n<p>This takes the NVIDIA Jetson Nano to the next level by training a Large Language Model (LLM) completely from scratch. It uses the lightweight and efficient NanoGPT framework,<\/p>\n<p>This covers setting up your environment to running your very own trained model.<\/p>\n<p>This work is taking what Andre Karpathy did for teaching people how to make an LLM and applying to the Nvidia Nano developer kits.<\/p>\n<p><iframe width=\"560\" height=\"315\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen data-src=\"https:\/\/www.youtube.com\/embed\/l8pRSuU81PU?si=LzbzBKlDs45k0KJ8\"><\/iframe><\/p>\n<p><iframe loading=\"lazy\" width=\"560\" height=\"315\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen data-src=\"https:\/\/www.youtube.com\/embed\/zduSFxRajkE?si=dS0W9k2DvjuuU54i\"><\/iframe><\/p>\n<p><iframe loading=\"lazy\" width=\"560\" height=\"315\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen data-src=\"https:\/\/www.youtube.com\/embed\/JWfNLF_g_V0?si=0uJ9zgKQBd3OsNRs\"><\/iframe><\/p>\n<p>He begins by preparing the Jetson Nano with all the necessary tools, including installing and testing PyTorch. <\/p>\n<p>The concept and processes of tokenization, dataset preparation, and optimizing the Jetson Nano for training efficiency are reviewed. <\/p>\n<p><img loading=\"lazy\" decoding=\"async\" alt width=\"406\" height=\"533\"   data-eio=\"l\" data-old-src=\"data:image\/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZYAAAIVAQAAAADaT+TQAAAAAnRSTlMAAHaTzTgAAAAySURBVHja7cExAQAAAMKg9U9tCy+gAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACApwFsRAABzwaCDAAAAABJRU5ErkJggg==\" src=\"https:\/\/nextbigfuture.s3.amazonaws.com\/uploads\/2025\/01\/Screenshot-2025-01-27-at-9.50.34-AM1.jpg\" srcset=\"https:\/\/nextbigfuture.s3.amazonaws.com\/uploads\/2025\/01\/Screenshot-2025-01-27-at-9.50.34-AM1.jpg 406w, https:\/\/nextbigfuture.s3.amazonaws.com\/uploads\/2025\/01\/Screenshot-2025-01-27-at-9.50.34-AM1-229x300.jpg 229w\"><\/p>\n<p>He shows the training process and demonstrate how to run the trained model directly on the device.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" alt width=\"544\" height=\"635\"   data-eio=\"l\" data-old-src=\"data:image\/png;base64,iVBORw0KGgoAAAANSUhEUgAAAiAAAAJ7AQAAAADyuxeFAAAAAnRSTlMAAHaTzTgAAABBSURBVHja7cEBDQAAAMKg909tDwcUAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAhwarJwABLpDHLgAAAABJRU5ErkJggg==\" src=\"https:\/\/nextbigfuture.s3.amazonaws.com\/uploads\/2025\/01\/Screenshot-2025-01-27-at-9.51.46-AM.jpg\" srcset=\"https:\/\/nextbigfuture.s3.amazonaws.com\/uploads\/2025\/01\/Screenshot-2025-01-27-at-9.51.46-AM.jpg 544w, https:\/\/nextbigfuture.s3.amazonaws.com\/uploads\/2025\/01\/Screenshot-2025-01-27-at-9.51.46-AM-257x300.jpg 257w\"><\/p>\n<p>This guide is perfect for beginners and AI enthusiasts looking to explore LLM training on affordable hardware. With a focus on step-by-step instructions, you\u2019ll be able to replicate the process and unlock the power of local LLMs on your Jetson Nano.<\/p>\n<p><iframe loading=\"lazy\" width=\"560\" height=\"315\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen data-src=\"https:\/\/www.youtube.com\/embed\/s1uFVfuT2aw?si=cxSGk2ogY2mmw_Rk\"><\/iframe><\/p>\n<p>Timestamps:<\/p>\n<p>00:00 \u2013 Intro<br \/>\n01:08 \u2013 Setting Up Our Environment<br \/>\n05:30 \u2013 Installing Pytorch<br \/>\n13:50 \u2013 Testing Pytorch<br \/>\n09:00 \u2013 NanoGPT Setup<br \/>\n16:03 \u2013 Understanding Tokenization<br \/>\n23:24 \u2013 Data Set Prep<br \/>\n24:20 \u2013 Training Intro<br \/>\n27:33 \u2013 Training Setup<br \/>\n33:48 \u2013 Jetson Optimizations<br \/>\n36:10 \u2013 Training The Model<br \/>\n40:05 \u2013 Running The Model<br \/>\n45:38 \u2013 Closing Thoughts<\/p>\n<p><a href=\"https:\/\/gist.github.com\/OminousIndustries\/109672cc8f32bf1ea1998bd986f6b96a\" target=\"_blank\" rel=\"noopener\">https:\/\/gist.github.com\/OminousIndustries\/109672cc8f32bf1ea1998bd986f6b96a<\/a><\/p>\n<p><a href=\"https:\/\/www.amazon.com\/onboard-Robotics-Machine-Learning-Version\/dp\/B0B8DMPWJL?psc=1&#038;smid=A3FZJTJP2FUJVA&#038;gPromoCode=15001837506844198539&#038;gQT=2&#038;utm_source=Perplexity&#038;utm_medium=referral\" target=\"_blank\" rel=\"noopener\">Amazon \u2013 Jetson Nano 4GB RAM 16G eMMC  for $215 (and a 5% coupon to get to about $204).<\/a><\/p>\n<p><a href=\"https:\/\/www.amazon.com\/Developer-Without-Learning-Development-Platform\/dp\/B0BYJMHMRB\/ref=sr_1_2?crid=1LJ19P2RTCMPF&#038;dib=eyJ2IjoiMSJ9.bqSeB6haHQM6JppJPjSYU9PJgSNJhz4lJGj03WiUYVa89xQ3PmA3jGv_ILyAxaiCuT4-PDhMVJpJOR5ecP2AMmWrN7yRHRmw3JTECgVEgqgLCiWBJfQWisegmcf84dta8rQdCH5QNcjdAyZx6HtyCi8r-eHLJTp6c5Q5-FycL69_szJwiw5X5f_Znyvfv4nw8TZX6IjYG2tcxh-sZF__a8V2jANZqPSA_oVpCyeQ6rAMRvAyNk9cDVlyg8rMD9B2iglwvJCqYukjkkKIpMjBF0M4DigJK9xs6HjfvIKkOU80IVb5xHbeHKJNxU5HRxBUaUQigUGP4VrWW8v1Y7TzhRPXiJXm4TxaSniAjyyLzwc.kQ7iasxLMp1Ybruu_mccOARJ_nWnVrF2yrXHkTgORio&#038;dib_tag=se&#038;keywords=Jetson+Nano+2GB&#038;qid=1737999396&#038;s=electronics&#038;sprefix=jetson+nano+2gb%2Celectronics%2C109&#038;sr=1-2\" target=\"_blank\" rel=\"noopener\">Amazon Nvidia Jetson Nano 2GB Developer without Wifi $160.<\/a><\/p>\n<div itemtype=\"http:\/\/schema.org\/Person\" itemscope itemprop=\"author\">\n<p><img loading=\"lazy\" decoding=\"async\" width=\"100\" height=\"100\" alt itemprop=\"image\" data-eio=\"l\" data-old-src=\"data:image\/png;base64,iVBORw0KGgoAAAANSUhEUgAAAGQAAABkAQAAAABYmaj5AAAAAnRSTlMAAHaTzTgAAAAUSURBVDjLY2AYBaNgFIyCUUBPAAAFeAABKXG5\/AAAAABJRU5ErkJggg==\" src=\"https:\/\/nextbigfuture.s3.amazonaws.com\/uploads\/2020\/08\/brianheadshot.jpg\"><\/p>\n<div itemprop=\"description\">\n<p>Brian Wang is a Futurist Thought Leader and a popular Science blogger with 1 million readers per month. His blog Nextbigfuture.com is ranked #1 Science News Blog. It covers many disruptive technology and trends including Space, Robotics, Artificial Intelligence, Medicine, Anti-aging Biotechnology, and Nanotechnology.<\/p>\n<p>Known for identifying cutting edge technologies, he is currently a Co-Founder of a startup and fundraiser for high potential early-stage companies. He is the Head of Research for Allocations for deep technology investments and an Angel Investor at Space Angels.<\/p>\n<p>A frequent speaker at corporations, he has been a TEDx speaker, a Singularity University speaker and guest at numerous interviews for radio and podcasts.\u00a0 He is open to public speaking and advising engagements.<\/p>\n<\/div>\n<\/div><\/div>\n<p><a href=\"https:\/\/www.nextbigfuture.com\/2025\/01\/setting-up-nvidia-jetson-nano-install-pytorch-and-start-working-with-llms.html\" class=\"button purchase\" rel=\"nofollow noopener\" target=\"_blank\">Read More<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Youtuber Ominous Industries provides a training video for setting up your Jetson Nano, installing Pytorch and setting up your own (likely open source LLM). This takes the NVIDIA Jetson Nano to the next level by training a Large Language Model (LLM) completely from scratch. It uses the lightweight and efficient NanoGPT framework, This covers setting [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":822904,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[22291,24029,104640],"tags":[],"class_list":["post-822903","post","type-post","status-publish","format-standard","has-post-thumbnail","category-nvidia","category-setting","category-youtube-videos"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/posts\/822903","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/comments?post=822903"}],"version-history":[{"count":0,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/posts\/822903\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/media\/822904"}],"wp:attachment":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/media?parent=822903"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/categories?post=822903"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/tags?post=822903"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}