Tag Archives: Artificial intelligence

No Cloud. No Internet. No Problem. Two Commands for Local LLM on Jetson Orin Nano

June 7, 2026 admin

Hey guys, welcome back to the channel. Paul McWhorter here from TopTechBoy.com. Today, we aren’t just messing around with simple circuits or basic scripts—we are going to take that NVIDIA Jetson Orin Nano we rescued from the brink of destruction in the last video, and we are going to turn it into a completely sovereign, local thinking machine.

I don’t know about you, but I am tired of Big Tech telling me I need a credit card, a monthly subscription, and a constant high-speed internet connection just to make an AI model reply to a prompt. Today, we are going to do it completely naked. We are going to cut the cord, pull the ethernet, and run cutting-edge Large Language Models entirely on the local physical silicon of your Jetson Orin Nano.

And we are going to do it in exactly two commands. One to build the engine room, and one to fire up the mind.

Let’s get started.

The Hardware Architecture

Before we drop the code into the terminal, let’s understand exactly what we are building today. We are dealing with three core components working together in a unified system.

The Model (The Fuel): This is your raw neural network file (like Google Gemma or Meta Llama). It contains the weights, vocabulary, and potential intelligence. On its own, it’s just a massive, inert file sitting on your storage drive.
Ollama (The Engine Room): This is the heavy lifter. Ollama is a local execution framework that takes that raw model file and boots it directly into the Jetson’s unified RAM and CUDA cores. It handles the brutal mathematical calculations required to generate tokens.
The Terminal Chat (The Dashboard): This is your interface. It provides the clean command-line text box for you to type your prompts and prints the model’s responses back to you in real time.

The Two-Command Installation

Go ahead and fire up your Jetson Orin Nano, open a fresh terminal window, and get ready to type. Remember: copying and pasting makes you weak. Type these out like a real engineer so your hands learn the muscle memory.

Command 1: Install the Ollama Engine

This command fetches the official automated bootstrapper script from Ollama and executes it locally to configure the background system service on your host OS.

curl -fsSL https://ollama.com/install.sh | sh

1	curl -fsSL https://ollama.com/install.sh \| sh

Command 2: Fire Up the Local Model

Once the installation script finishes, your engine room is live. Now, tell Ollama to pull down the optimized 1-billion parameter Google Gemma model and launch an interactive local dialog loop instantly:

ollama run gemma3:1b

1	ollama run gemma3:1b

The moment you hit enter, your Jetson will download the model weights directly to your local drive, load them straight into the VRAM, and drop you into a clean prompt box. Type a question, hit enter, and watch your local silicon generate answers with zero cloud dependencies.

Choosing the Right Mind for Your Machine

The beautiful part about setting up Ollama is that you aren’t locked into just one model. Different models have different parameter sizes and strengths. On the 8GB Jetson Orin Nano, you want to balance model size against your available hardware headroom to keep your generation speeds crisp.

Here are the verified, hardware-accelerated local models you can experiment with right out of the box:

Launch Command	Model Family	Size / Parameter Count	Best Used For
`ollama run gemma3:1b`	Google Gemma 3	1 Billion	Ultra-fast responses, light footprint
`ollama run llama3.2:1b`	Meta Llama 3.2	1 Billion	High-efficiency conversational loops
`ollama run phi4-mini:3.8b`	Microsoft Phi-4	3.8 Billion	Heavy reasoning and coding logic
`ollama run qwen3:4b`	Alibaba Qwen 3	4 Billion	Structured data and multilingual logic
`ollama run qwen3.5:4b`	Alibaba Qwen 3.5	4 Billion	Advanced context processing
`ollama run gemma3:4b`	Google Gemma 3	4 Billion	Maximum analytical depth on Orin Nano

⚠️ Paul’s Engineering Note on Headroom

The 1B (1-Billion parameter) models are incredibly light and will run at lightning speed on the Orin Nano. If you want to push the machine harder for more complex reasoning, step up to the 3.8B or 4B models. Just keep an eye on your system resources—running a 4B model pushes close to the limits of the Orin Nano’s 8GB unified memory architecture, especially if you are running a heavy graphical desktop environment in the background!

To exit out of any active terminal chat session and return to your standard command prompt, simply type:

/<span class="hljs-built_in">exit</span>

				1
2

						/<span class="hljs-built_in">exit</span>

Homework Assignment

Alright, you have the hardware running, you have the engine installed, and you know how to switch out the minds of your machine. Now it’s time for your homework.

I want you to install both the gemma3:1b model and the heavier gemma3:4b model on your Jetson Orin Nano. Run them both through a test sequence: ask them to write a simple Python script, and then ask them a complex logic riddle.

I want you to observe the difference in quality of thought versus speed of generation. Is the 4-billion parameter model smart enough to justify the extra computation time on your hardware, or does the 1-billion parameter model give you the snappy responsiveness you need for a real-time edge application?

Leave a comment down under the video showing your results, tell me which model you prefer running natively on your bench, and I will see you guys in the next lesson!

AI on the Edge LESSON 4: Python Averaging Grades Homework Solution

April 8, 2026 admin

Hey everyone, and welcome back to the AI on the Edge series!

In Lesson 3, I gave you your first real programming homework — to create a program that lets the user enter multiple grades and then calculates the average. In today’s Lesson 4, we go through the solution together step-by-step.

This lesson is all about learning how to work with lists (also called arrays), using for loops effectively, and building clean, organized code. Even though averaging grades might seem simple, these are fundamental programming skills that we will use constantly as we move forward in this class. Whether we’re averaging sensor readings, smoothing camera data, calculating confidence scores from AI models, or processing batches of information — the ability to collect data, store it, and process it is extremely important.

In the video, I walk you through a clean solution that uses a list to store all the grades, then loops through that list to calculate the total before dividing by the number of grades. You’ll also see how to display the original grades back to the user and present the final average in a nice, readable way.

I really enjoy these early lessons because this is where you start developing good programming habits. The techniques you learn here — using lists, loops, and organizing your code — will become the building blocks for much more powerful AI projects later in the series.

By the end of this lesson, you should feel much more comfortable working with lists and loops in Python. These skills are going to be used over and over again as we start reading sensors, processing camera frames, and handling data from AI models.

So if you tried the homework, awesome! If you got stuck, that’s perfectly okay — that’s exactly why we go through the solution together. Take the code, run it, and then I strongly encourage you to modify it. Try adding letter grades (A, B, C), calculate the highest and lowest grade, or make it keep running until the user wants to quit. The more you play with it, the faster you’ll learn.

You’re doing great! These early Python lessons are the foundation we need before we start combining code with real hardware and AI in the coming lessons. Keep going — we’re building something really cool here!

This is my homework solution.

while True:
    bucket = 0
    numGrades = int(input('How Many Grades do you Have: '))
    grades = []
    for i in range(numGrades):
        grade=int(input('Enter Your '+str(i+1)+' Grade '))
        grades.append(grade)
        print('You Entered: ',grade)
    for i in range(numGrades):
        bucket=bucket+grades[i]
    myAverage=bucket/numGrades
    print('Your Grades Are: ')
    for i in range(numGrades):
        print(grades[i])
    print('Your Average is: ',myAverage)

while True:

bucket = 0

numGrades = int(input('How Many Grades do you Have: '))

grades = []

for i in range(numGrades):

grade=int(input('Enter Your '+str(i+1)+' Grade '))

grades.append(grade)

print('You Entered: ',grade)

for i in range(numGrades):

bucket=bucket+grades[i]

myAverage=bucket/numGrades

print('Your Grades Are: ')

for i in range(numGrades):

print(grades[i])

print('Your Average is: ',myAverage)

AI On the Edge

AI on the Edge LESSON 2: Raspberry Pi Operating System for Artificial Intelligence

March 29, 2026 admin

The major challenge we face in this AI on the Edge class is getting a Raspberry Pi 5 configures where you have all the AI Models, Libraries, Modules and Methods installed, and where they all play nicely together. Often, when you add a new model, the old model becomes broken. This is because when you install something new, it often times updates the dependencies. That means it updates a library already on your system. For example, lets say you have numpy 14, working with YOLO 11. Now you install mediapipe, and it updates numpy 14 to numpy 15. This then ‘Breaks’ your YOLO, as it wanted a different version of numpy. Likely you will get frustrated and quit before you get the dependency problems solved. In order to get around this, you can use a special education version of the Bookworm OS, which has all the needed libraries installed already and working nicely with each other. The video above shows you how to install this OS. Once you do, no not update it, do not upgrade it, do not touch it. Use it to develop your programs and projects for this class. If you want to do something else with your pi, have a separate SD card.

AI On the Edge

AI on the Edge LESSON 1: Introduction and Class Overview

March 29, 2026 admin

Welcome to our all new AI on the Edge class! I will need you to buckle up, get your hardware together, and get ready to teach AI who is boss! We will be using a Pi 5, and the Fusion AI Lab kit. I will show links to the hardware below. In today’s lesson I describe the Class Introduction, and will show you some demos of the types of projects we will be doing. You will either Drive AI or your will be Destroyed by AI. Don’t be one of the ones who will be eaten by it

The Future will Belong to Those Who Can Drive AI

Guys, get your gear, and make sure you end up on the right side of the Dystopian future that awaits the world.

I have provided Amazon links, so you can order everything in the same place

You Will need a Raspberry Pi 5
Order Pi 5

You will need a heat sink and fan
Order Heat Sink and Fan

You Will Need the Fusion AI Lab Kit
Order Fusion AI Lab Kit

You Will Need a 25 Watt Power Supply
Order Power Supply

You Will Need a Micro HDMI Cable
Order Micro HDMI Cable

You Will Need a Keyboard and Mouse
Order Wireless Keyboard and Mouse

This isn’t just another Raspberry Pi class. This is a complete journey where we’re going to take the powerful Raspberry Pi 5, combine it with the SunFounder Fusion AI Lab kit, and build real, practical, intelligent systems that run completely on the edge — no cloud, no internet required.

In this class, you’re not going to just learn how to blink an LED or run someone else’s pre-made script. You’re going to learn how to build smart machines that can see, listen, speak, think, and act in the real world. We’re going to combine computer vision, voice recognition, speech synthesis, sensor reading, motor control, and modern AI techniques — all running locally on your Raspberry Pi 5.

Over the course of this series, you will learn how to:

Capture and process live video from the Raspberry Pi Camera
Detect faces and track objects in real time using MediaPipe and OpenCV
Control hardware with voice commands
Make your Raspberry Pi speak with natural-sounding Text-to-Speech
Build smooth, responsive control systems using threading
Use displays like the SSD1306 OLED to show live information
Combine everything into impressive AI-powered projects

This class is designed for makers, students, hobbyists, and engineers who want to move beyond basic tutorials and start building real intelligent edge devices. Whether you dream of building smart robots, autonomous monitoring systems, interactive AI companions, or just want to gain serious skills in modern embedded AI, this class is for you.

I’m going to teach this the way I always do — step by step, clearly, and with lots of hands-on projects. We’ll start with the fundamentals and gradually build up to more advanced and exciting projects as the class progresses.

If you’ve ever wanted to move from “playing with the Raspberry Pi” to “building truly intelligent systems,” then you’re in the right place. This is going to be a fun, challenging, and incredibly rewarding journey.

So if you’re ready to stop just watching AI videos and start building your own AI on the edge… then buckle up, because we’re about to do exactly that.

Welcome to the class! I’m really glad you’re here. Let’s get started!

OpenCV, Python, Raspberry Pi

Object Detection Using YOLO and RTSP Camera on Raspberry Pi 5

January 2, 2026 admin

OK guys, you spoke, and I listened. You all are asking for a lesson on how to do object detection on a Pi 5 using YOLO and an IP Camera. Well, you are about to get what you asked for. We will make this work, or we will DIE TRYING. Never fear, once you watch the video you will both understand and be able to do it on your own. First, I am assuming you watched our previous lesson where I showed you how to do the basic install and setup of YOLO. If not, never fear, I have the commands below. NOTE: This tutorial is geared towards bookworm OS. I strongly suggest you start with a fresh bookworm SC card, as there are many dependencies, and it is most likely to work if you start exactly where I am starting . . . with a fresh OS. Thes these are the commands I shared last week to get YOLO up and working: (just open a terminal, and paste these commands one at a time)

# 1. Configure X11 (manual steps required)
sudo raspi-config
# → Go to Advanced Options → X11 → Enable X11
# → Finish and reboot when prompted

# 5. Update system and install OpenCV
sudo apt update
sudo apt full-upgrade -y
sudo apt install python3-opencv -y

# Verify OpenCV
python3 -c "import cv2; print('OpenCV version:', cv2.__version__)"
# Expected output: something like "OpenCV version: 4.6.0" or higher

# Create and activate virtual environment for YOLO11 (Ultralytics)
python3 -m venv --system-site-packages YOLO
source YOLO/bin/activate

# You are now inside the (YOLO) virtual environment
# Install Ultralytics YOLO11 inside it
pip install "numpy<2.0" ultralytics

# Now create a Pi friendly YOLO11 model
yolo export model=yolo11n.pt format=ncnn

# Optional: Verify YOLO installation
python -c "from ultralytics import YOLO; print('Ultralytics YOLO ready')"

# When finished working with YOLO, you can deactivate with:
# deactivate

#Now open Thonny, and you need to point thonny to the virtual environment you 
#just created. Open tools- options, select 'interpreter' tab, then click they Python
#executable, selecting ... and navigate from home directory, 
#to YOLO, to bin, and then select python

# 1. Configure X11 (manual steps required)

sudo raspi-config

# → Go to Advanced Options → X11 → Enable X11

# → Finish and reboot when prompted

# 5. Update system and install OpenCV

sudo apt update

sudo apt full-upgrade -y

sudo apt install python3-opencv -y

# Verify OpenCV

python3 -c "import cv2; print('OpenCV version:', cv2.__version__)"

# Expected output: something like "OpenCV version: 4.6.0" or higher

# Create and activate virtual environment for YOLO11 (Ultralytics)

python3 -m venv --system-site-packages YOLO

source YOLO/bin/activate

# You are now inside the (YOLO) virtual environment

# Install Ultralytics YOLO11 inside it

pip install "numpy<2.0" ultralytics

# Now create a Pi friendly YOLO11 model

yolo export model=yolo11n.pt format=ncnn

# Optional: Verify YOLO installation

python -c "from ultralytics import YOLO; print('Ultralytics YOLO ready')"

# When finished working with YOLO, you can deactivate with:

# deactivate

#Now open Thonny, and you need to point thonny to the virtual environment you

#just created. Open tools- options, select 'interpreter' tab, then click they Python

#executable, selecting ... and navigate from home directory,

#to YOLO, to bin, and then select python

Now, I will explain this code, and will help you configure it for your cameras, but you will need to open up thonny, and paste in the following code as a start. IMPORTANT, as mentioned above, you need to set interpreter in thonny to the virtual environment set up in the process above. If this is not familiar to you, go back and watch last weeks lesson (click previous at the bottom of this post). Without further adue, here is the code we will work with today:

import cv2
from ultralytics import YOLO
#import secret
import threading
import time

W=1280
H=720
#Now, as eplained in the video, edit the line below where 
#you put your wifi user name in place of user, and your 
#wifi password in place of password, and you put in your 
#IP address
RTSP_URL = "rtsp://user:password@192.168.88.44:554/cam/realmonitor?channel=1&subtype=0"
#RTSP_URL = secret.RTSP_URL1
#I have put my wifi credentials in a secret file so no
#one sees my credentials. You will just use the RTSP_URL line above. 
# Load the exported NCNN model (replace with your model path)
model = YOLO("/home/pjm/yolo11n_ncnn_model/", task="detect")
lock = threading.Lock()
running = True
def frameGrabber(url):
    global ipFrame, running
    cap = cv2.VideoCapture(url, cv2.CAP_FFMPEG)
    cap.set(cv2.CAP_PROP_BUFFERSIZE, 1)
    cap.set(cv2.CAP_PROP_POS_FRAMES, 0)  # Reset frame position
    while running:
        ret, frame = cap.read()
        if ret:
            with lock:
                #frame = cv2.resize(frame, (W, H))
                ipFrame = frame.copy()
    cap.release()
    print("Thread Terminated")
thread = threading.Thread(target=frameGrabber, args=(RTSP_URL,), daemon=True)
thread.start()
tStart=time.time()
time.sleep(2)
fps=0
cnt=0
# Set resolution for faster processing (optional, adjust based on your needs)
while True:
    deltaT=time.time()-tStart
    fps= fps*.9 + .1/deltaT
    tStart=time.time()
    with lock:
        ipFrameShow=ipFrame.copy()
    results = model(ipFrameShow, conf=0.25, verbose=False)[0]  # conf: confidence threshold; adjust as needed

    # Annotate the frame with detections (boxes, labels, scores)
    annotatedFrame = results.plot()
    annotatedFrame=cv2.resize(annotatedFrame, (W,H))
    cv2.putText(annotatedFrame, "FPS: "+str(round(fps,1)), (int(W*.01), int(H*.075)), 
            cv2.FONT_HERSHEY_SIMPLEX, H*.002, (0, 0, 255), 3)
    cv2.imshow("IP Camera", annotatedFrame)
    #cv2.moveWindow("IP Camera",100,100)
    if cv2.waitKey(1)==ord('q'):
        break
running=False
thread.join()  # Wait for thread to fully exit
time.sleep(1)
cv2.destroyAllWindows()
import gc
gc.collect()  # Force garbage collection to reclaim memory/connections
#Make Extra sure all processes are killed
import os
os._exit(0)
print("Program Terminated")

import cv2

from ultralytics import YOLO

#import secret

import threading

import time

W=1280

H=720

#Now, as eplained in the video, edit the line below where

#you put your wifi user name in place of user, and your

#wifi password in place of password, and you put in your

#IP address

RTSP_URL = "rtsp://user:password@192.168.88.44:554/cam/realmonitor?channel=1&subtype=0"

#RTSP_URL = secret.RTSP_URL1

#I have put my wifi credentials in a secret file so no

#one sees my credentials. You will just use the RTSP_URL line above.

# Load the exported NCNN model (replace with your model path)

model = YOLO("/home/pjm/yolo11n_ncnn_model/", task="detect")

lock = threading.Lock()

running = True

def frameGrabber(url):

global ipFrame, running

cap = cv2.VideoCapture(url, cv2.CAP_FFMPEG)

cap.set(cv2.CAP_PROP_BUFFERSIZE, 1)

cap.set(cv2.CAP_PROP_POS_FRAMES, 0) # Reset frame position

while running:

ret, frame = cap.read()

if ret:

with lock:

#frame = cv2.resize(frame, (W, H))

ipFrame = frame.copy()

cap.release()

print("Thread Terminated")

thread = threading.Thread(target=frameGrabber, args=(RTSP_URL,), daemon=True)

thread.start()

tStart=time.time()

time.sleep(2)

fps=0

cnt=0

# Set resolution for faster processing (optional, adjust based on your needs)

while True:

deltaT=time.time()-tStart

fps= fps*.9 + .1/deltaT

tStart=time.time()

with lock:

ipFrameShow=ipFrame.copy()

results = model(ipFrameShow, conf=0.25, verbose=False)[0] # conf: confidence threshold; adjust as needed

# Annotate the frame with detections (boxes, labels, scores)

annotatedFrame = results.plot()

annotatedFrame=cv2.resize(annotatedFrame, (W,H))

cv2.putText(annotatedFrame, "FPS: "+str(round(fps,1)), (int(W*.01), int(H*.075)),

cv2.FONT_HERSHEY_SIMPLEX, H*.002, (0, 0, 255), 3)

cv2.imshow("IP Camera", annotatedFrame)

#cv2.moveWindow("IP Camera",100,100)

if cv2.waitKey(1)==ord('q'):

break

running=False

thread.join() # Wait for thread to fully exit

time.sleep(1)

cv2.destroyAllWindows()

import gc

gc.collect() # Force garbage collection to reclaim memory/connections

#Make Extra sure all processes are killed

import os

os._exit(0)

print("Program Terminated")

The video explains everything, please watch it!

Technology Tutorials

Tag Archives: Artificial intelligence

No Cloud. No Internet. No Problem. Two Commands for Local LLM on Jetson Orin Nano

The Hardware Architecture

The Two-Command Installation

Command 1: Install the Ollama Engine

Command 2: Fire Up the Local Model

Choosing the Right Mind for Your Machine

⚠️ Paul’s Engineering Note on Headroom

Homework Assignment

AI on the Edge LESSON 4: Python Averaging Grades Homework Solution

AI on the Edge LESSON 2: Raspberry Pi Operating System for Artificial Intelligence

AI on the Edge LESSON 1: Introduction and Class Overview

Object Detection Using YOLO and RTSP Camera on Raspberry Pi 5

Making The World a Better Place One High Tech Project at a Time. Enjoy!