Hey guys, Paul McWhorter here from toptechboy.com. Today, we are going to look at how to stop fighting your hardware and start running local AI like an absolute boss.

The NVIDIA Jetson Orin Nano is an absolute masterpiece of edge-compute hardware. But it has one major design constraint that catches almost every beginner off guard: Unified Memory. On the Orin, your CPU and your GPU share the exact same physical pool of 8GB of LPDDR5 RAM. When you boot into that pretty Ubuntu GNOME desktop, the system instantly steals over 1.5 GB of your precious VRAM just to draw a GUI you aren’t even looking at while your code is running.

In this lesson, we are going to reclaim that stolen memory, optimize our storage, and run a massive 8-billion parameter model (LLaMA 3.1 8B) smoothly on the Orin Nano by learning how to properly run a clean, headless configuration.

---

Step 1: Find Your Jetson’s IP Address

Before we turn off the monitor, we need to know how to talk to the Orin over the network. If you don’t know its IP address, you can’t SSH in once the screen goes dark. While you are still in the graphical terminal, run this command:

Look under your active connection interface (usually eth0 for Ethernet or wlan0 for Wi-Fi) for the inet address. It will look something like 192.168.1.15. Write this down! You will need it to remote in later.

---

Step 2: Disable and Remove the Default Swap File

By default, JetPack configures a slow, disk-based swap file on your NVMe drive. While swap space is great for general computing, it is an absolute performance killer for LLMs. If your model spillover starts paging to a disk-based swap file, your tokens-per-second will drop to a crawl, and the high-frequency writes will prematurely wear out your SSD.

We want our models running purely in ultra-fast LPDDR5 RAM. Let’s cleanly turn off and remove the swap file:

---

Step 3: Configure a Clean Boot into the Terminal

Many people will tell you to run sudo systemctl isolate multi-user.target to turn off the GUI. Do not do this! That command aggressively tears down active background services (including Ollama, network managers, and local development scripts) because it forces a state isolation.

Instead, we want to tell the Orin’s bootloader to cleanly start up in command-line mode from a fresh boot. This allows all your network drivers, background scripts, and Ollama to initialize perfectly without a display manager eating your memory:

Once you run this, restart your Orin to let the changes take effect cleanly:

---

How to Boot Back to the GUI (If Needed)

We are developers, which means we want to write and debug our scripts comfortably under the graphical desktop, and then deploy them headlessly. If you ever need to turn your monitor back on and return to the GNOME desktop, simply run this command over SSH:

Followed by a quick reboot (sudo reboot), and your desktop interface will return exactly as it was.

---

Step 5: The Test — Running LLaMA 3.1 8B in the GUI

To prove why this matters, let’s look at what happens when you try to force a large model to run while your monitor is plugged in and the graphical desktop is active. Open your terminal in the GUI and run:

The Result: The model will either completely crash with an “Out of Memory” (OOM) error, or it will run painfully slow, chugging out less than 2 tokens per second.

The “Why”: Where Did Your Memory Go?

An 8-billion parameter model quantized to 4-bits requires roughly 4.7 GB of static memory just to fit its weights. When you add the Context Window (KV Cache), that memory requirement quickly balloons to over 5.5 GB.

Here is exactly how your 8GB Orin Nano’s memory is divided when you run a GUI:

System State	Memory Allocation (Approximate)
OS Kernel & System Daemons	~1.2 GB
GNOME Desktop GUI (Monitor Active)	~1.6 GB
Available VRAM for AI	~5.2 GB (Not enough for 8B models + Context!)

Because the GUI steals 1.6 GB, your available memory drops below the critical threshold required to run LLaMA 3.1 8B. The moment your context grows, the system runs out of room, hits a bottleneck, or crashes.

---

Step 6: Reclaiming the Hardware (Headless Memory Profile)

Now let’s look at the memory profile when we boot the Orin Nano cleanly into the terminal without GDM3 starting up. If you SSH in and run free -h or check jtop, this is what you get:

System State	Memory Allocation (Approximate)
OS Kernel & System Daemons	~1.2 GB
GNOME Desktop GUI	0.0 GB (COMPLETELY RECLAIMED!)
Available VRAM for AI	~6.8 GB (Plenty of headroom for 8B models!)

By going headless, we instantly reclaimed **1.6 GB of ultra-fast VRAM**. That is the difference between night and day when deploying edge AI models.

---

Step 7: Connect from Windows PowerShell

Now that your Orin is booted headlessly, unplug the monitor, keyboard, and mouse. Walk back to your main Windows development machine, open up **PowerShell**, and SSH directly into the Orin over your local network using the IP address you saved in Step 1:

(Be sure to replace “pjm” with your actual Orin username and use your specific IP address!)

---

Step 8: Run LLaMA 3.1 8B Like a Boss

With your GUI safely dead and your memory completely optimized, run the exact same model command inside your PowerShell session:

The Payoff: Because the system now has a massive 6.8 GB of free, continuous VRAM, the model loads entirely into the Orin’s hardware engines. You will see prompt evaluations complete instantly, and the text will output at an extremely usable speed without a single memory warning or system hiccup.

That is how you cleanly manage your hardware resources, develop efficiently, and run large local LLMs on the edge like an absolute boss.

If you enjoyed this write-up, leave a comment below, subscribe to the channel, and I will see you guys in the next lesson!