Tag Archives: Fusion AI Lab Kit

AI on the Edge LESSON 53: Gesture Recognition on the Pi 5 with Training File

 Welcome back, everyone! In this lesson, we are taking our custom MediaPipe hand gesture recognition system to the next level. Up until now, every time you stopped your Python script, all of your hard-earned gesture training data vanished into thin air. If you wanted to run your project again, you had to re-train every single hand sign from scratch.

That ends today. We are going to make our AI remember what it learned by saving our gesture feature vectors to a local file using Python’s pickle library.

What You Will Learn in This Lesson

  • Data Persistence: How to serialize and save Python dictionaries containing landmark distance vectors directly to disk (gesture_data.pkl).

  • Automated Setup: How to check for existing training files at boot so your program automatically skips the initial training prompt if data already exists.

  • Dynamic Training Management: How to interactively train new gestures on the fly ('t' key) or delete outdated gestures ('d' key) during runtime while automatically updating the saved file.

  • Robust Error Handling: Ensuring index tracking (trainIdx) safely syncs with live gesture arrays so pressing the spacebar never crashes your video feed.

Key Concepts Explained

1. Pickling Data Objects In MediaPipe gesture recognition, a trained gesture isn’t a complex neural network weight file; it’s simply a dictionary mapping a gesture name (string) to a list of normalized keypoint ratios (floating-point numbers). Python’s pickle module allows us to convert this dictionary into a byte stream and write it directly to a file (.pkl). When the program launches, pickle.load() reconstructs the dictionary in milliseconds.

2. Managing Execution Flow on Startup When gesture_data.pkl exists, our program automatically populates gestureNames with the keys from trainedData and sets trainIdx past the end of the array. This keeps the camera pipeline running in Inference Mode immediately. If no file exists, the script seamlessly drops into Training Setup Mode to prompt the user for initial inputs.

This is the code we develop in this video lesson:

Homework Assignment

Now that your Pi 5 can remember hand gestures across reboots, it’s time to bridge software recognition with real-world hardware control!

The Challenge: Integrate an SPI NeoPixel Ring (WS2812B/SK6812) with your gesture recognition code.

  • Map distinct hand signs to dynamic LED patterns (for example: Fist = Red Pulse, Open Palm = Bright White Beacon, Peace Sign = Dual Spinner, OK Sign = Rainbow Wheel).

  • Make sure your animation updates are smooth and non-blocking so your OpenCV camera frame rate doesn’t drop!

We want to see who in the community can build the most impressive physical light demonstration controlled entirely by hand gestures. Share your video results and code modifications in the comments below!

AI on the Edge LESSON 52: Gesture Recognition on the Raspberry Pi 5

Welcome back, everyone! In today’s lesson, AI on the Edge LESSON 52: Gesture Recognition on the Raspberry Pi 5, we are building a custom, real-time hand gesture recognition system from scratch. Up until now, we have been using MediaPipe to track keypoints, identify individual hand joints, and extract raw 3D coordinate data. But raw coordinates alone don’t tell the computer what your hand is actually doing. Today, we bridge that gap by teaching our Raspberry Pi 5 how to train on custom hand positions and dynamically classify live hand gestures using pure mathematics and vector geometry—no heavy neural network retraining required.

The core secret to making gesture recognition work reliably across different camera distances is scale-invariant distance normalization. If you simply measure the pixel distance between your fingertips and your wrist, bringing your hand closer to the camera lens will blow out those numbers and break your classifier. To solve this, our script establishes a baseline unit of measurement for every single frame: the distance between the wrist (landmark 0) and the index finger base joint (landmark 5). By dividing all measured finger-tip-to-wrist and tip-to-tip distances by this dynamic scale factor, the resulting feature vector remains identical whether your hand is two feet away or five feet away from the lens.

Once we extract this normalized 15-element feature vector—representing the spatial ratios between all five fingertips relative to the wrist and each other—we construct an interactive training phase directly inside the live video loop. The program prompts you in the terminal for the number of custom gestures you wish to record, along with their labels (such as “Peace”, “Thumbs Up”, or “Fist”). As you present each gesture to your camera feed and press the spacebar, the algorithm captures the instantaneous geometric signature of your hand and stores it in a runtime training dictionary.

During live inference, our classifier calculates the Sum of Absolute Errors (SAE), or Manhattan distance, between the live incoming feature vector and every stored profile in our training dataset. The function loops through all saved gestures to find the candidate with the absolute lowest cumulative error score. To prevent false positives when an untrained or messy hand pose is shown, we compare that lowest error score against a strict threshold value (maxErrorThreshold = 3.5). If the error falls within the allowable boundary, the matched gesture label is instantly displayed on the live OpenCV HUD; otherwise, the system safely defaults to “Unknown”.

We manage our camera feed using high-speed frame capture with Picamera2 running at 60 frames per second at 1280×720 resolution, processing hand detection with MediaPipe Hands, and rendering real-time performance diagnostics—including an exponential moving average FPS counter—directly onto the video output. Go ahead, study the methodology, load the concept onto your Raspberry Pi 5, grab a hot cup of coffee, and let’s get building!

 

AI on the Edge LESSON 49: Live Animated Avatars on the SSD1306 OLED With OpenCV

Hey everybody! Welcome back to the AI on the Edge series!

In Lesson 49, we’re doing something that’s just plain cool — we’re turning that tiny little SSD1306 OLED into a live animated avatar that follows your face and hands in real time!

That’s right — no more static images, and no more being tethered to your desktop and monitor. This time your OLED is alive! Move your head, smile, raise your eyebrows, wave your hands… and the little display instantly mirrors your movements like a miniature digital twin. It’s like having a tiny version of yourself living on that 128×64 screen!

What You’ll Learn in This Lesson:

  • How to use MediaPipe Face Mesh to track 468 facial landmarks
  • How to combine it with Hand Tracking at the same time
  • The trick to projecting those landmarks onto a tiny OLED in real time
  • How to draw both points and connecting lines to create smooth, animated avatars
  • How to keep everything running fast on the Raspberry Pi

This is one of those projects that makes people stop and say “Wait… how is that even possible on that little screen?!”

By the end of this lesson, you’ll have a foundation for building all kinds of fun real-time avatar projects — everything from simple face mirrors to gesture-controlled characters and beyond.

Why This Is Awesome:

We’re not just displaying video on the big screen anymore — we’re compressing all that AI power down onto a cheap little OLED. This is real AI on the Edge stuff, and it looks incredibly impressive for how simple the hardware is.

Whether you want to build interactive displays, robot faces, wearable tech, or just blow your friends’ minds, this lesson gives you the core technique to make it happen.


Ready to make your OLED come alive?

Fire up the code, run it, and watch your tiny digital self start moving with you. Then start experimenting! Try drawing just the eyes and mouth, or just the hands, or maybe even turn it into a little stick figure avatar.

As always, I want to see what you create with this! Drop your versions in the comments or tag me — I love seeing the creative stuff you guys come up with. Don’t just copy and paste my code. Show you really understand the video by making the code yout own.

Enough talk, lets get this party started!

 

AI on the Edge LESSON 48: Hand Detection in OpenCV and MediaPipe on the Raspberry Pi

In Lesson 48 we’re taking a huge step forward — we’re bringing real-time hand detection to the Raspberry Pi using the incredible power of MediaPipe combined with OpenCV and the Pi Camera. This is one of those projects that feels like pure magic when you see it working: the camera picks up your hands instantly, tracks all 21 landmarks on each hand, draws beautiful connections in real time, and does it all right on the edge with no cloud required.
You’ll learn how to set up MediaPipe’s Hands solution for reliable multi-hand tracking, how to efficiently process frames from the Picamera2 library, flip and convert images for proper display, and draw professional-looking landmarks and connections with custom styling. We also keep a smooth FPS counter running so you can see exactly how well your Pi is performing.
This lesson is exciting because hand tracking is the foundation for so many advanced gesture-control projects — think touchless interfaces, sign language recognition, robotic control, virtual instruments, and interactive art installations. Once you have solid hand detection running smoothly on your Raspberry Pi, the creative possibilities are almost endless.
So grab your Raspberry Pi, fire up the camera, and let’s get those hands dancing on the screen! As always, the full code is waiting for you below. Watch the video, follow along, and then start experimenting — I can’t wait to see what you build with this!
Let’s make something awesome!

 

AI on the Edge LESSON 46: Ultimate Dazzleing Running Rainbow On a NeoPixel Ring

In this lesson we create one of the most beautiful and satisfying NeoPixel effects — a smooth, continuous, running rainbow on a 12-LED ring. We affectionately have named this pattern the “Runbow”. This is the “ultimate” version because it is both visually stunning and highly educational, as we explore two fundamentally different programming approaches the Runbow.
The first method is the most intuitive: we place a fixed rainbow across the ring (each LED gets a different hue equally spaced around the color wheel) and then simply rotate the entire pattern one position at a time. This approach is easy to understand because it mimics physically moving a colorful wheel. We save the color of the first pixel, shift all the other pixels one place, and then place the saved color at the end. It feels like a conveyor belt of color circling around the ring.
The second method is more elegant and mathematically pure. Instead of storing and shifting colors, we recalculate the color of every LED on every frame using a moving offset. For each LED we compute its hue as (i / LED_COUNT + offset) % 1.0, where i is the LED’s position and offset is a value that slowly increases over time. This creates a perfectly smooth rainbow that flows around the ring without ever shifting raw RGB values. Because we regenerate the pattern fresh each cycle, there is no risk of color corruption or accumulated errors.
Both techniques produce a gorgeous running rainbow, but they teach very different programming mindsets. The first method helps you deeply understand array manipulation and data movement. The second method introduces the powerful concept of using mathematics and offsets to create motion — a technique used frequently in advanced LED animations, games, and visual effects. In the end, the offset method tends to look smoother and is easier to extend with additional effects (such as brightness pulsing), but both approaches are valuable skills for any embedded AI or IoT developer working with addressable LEDs.You can adjust the speed of the rainbow by changing how much you increment the offset each loop or by modifying the delay. Once you master these two methods, you will be able to create almost any animated pattern you can imagine on your NeoPixel ring. Lets get this party started!
Here is the code we developed in this video:

This is the schematic we are using to connect the NeoPixel ring:

NeoPixel
NeoPixel Schematic