Category Archives: AI On the Edge

AI On the Edge, Python

AI on the Edge LESSON 28: Use Pan Tilt Camera to Track Object of Interest in OpenCV

In this video lesson we learn how to track our object of interest using the Pan Tilt camera. The camera will constantly adjust position to keep the Object of Interest in the center of the camera frame. We identify the Object of Interest based on Color, as we learned in earlier lessons. In this video, we developed the code below:

 

AI On the Edge, Raspberry Pi

AI on the Edge LESSON 27: Track Objects of Interest in OpenCV Using Contours

In this lesson we show you how to box your object of interest in openCV.  We do this by finding outline contours around the object we tune in based on color in the HSV color space. Then we create a bounding rectangle around the contour. Then we end up having a rectangular bounding box that tracks the object of interest as it moves. This is the code we developed in the video. Note you will have to tune the LC and UC parameters for your object of interest, as we showed last week.

 

AI On the Edge, Raspberry Pi

AI on the Edge LESSON 26: Understanding the HSV Color Space in OpenCV

Hey guys, welcome back to the channel. If you’ve been following along, you know we’ve been pushing our hardware absolutely down into the dirt. We’ve been running large language models right on the edge, pushing our boards hot and heavy until the silicon is screaming and the thermal throttling flags are popping up all over the place.

But today, we are stepping away from the heavy-compute server terminals, and we are getting back to our roots: Real-Time Computer Vision and Embedded Control. In our previous lessons, we learned how to hook up our high-speed camera, capture raw frames, and interact with individual pixels using standard RGB/BGR math. But today, we are going to look under the hood of a completely different way of representing color: The HSV Color Space (Hue, Saturation, Value).

If you try to track objects or isolate specific colors in the traditional RGB world, you are going to pull your hair out. The moment a shadow hits your object or the room lighting changes, your Red, Green, and Blue values completely collapse. By shifting our mathematics into the HSV space, we can lock onto a color’s pure identity regardless of whether it is sitting under a bright laboratory spotlight or a dim shadow.

Not only are we going to capture and process these video streams at a smooth-as-silk 60 frames per second, but we are also going to translate that raw visual math directly into the physical world. We are using our trusty SunFounder Fusion HAT+ to dynamically pulse an external RGB LED, matching its brightness and color hue perfectly to whatever pixel your mouse is clicking on in real-time.

Let’s look at the blueprint to make this happen.

The Complete Python Code

Here is the clean, un-guardrailed Python script for today’s lesson. Paste this directly into your local terminal workspace. No bloated libraries, no unnecessary frameworks—just pure, deliberate engineering.

Under the Hood: How the Code Works

1. The Real-Time Telemetry Smooth Filter

Look closely at how we calculate our frames-per-second metric inside the main processing loop:

If you simply print out the raw math of 1 / deltaT, your numbers on the screen are going to jump all over the place like a wild animal. By applying a 95% historical weight and a 5% instant weight, we create a low-pass software filter that smoothly tracks our true hardware operational speed without erratic layout jitter.

2. The Mouse Vector and BGR Array Sequence

When your mouse triggers an event over the window, OpenCV passes us the standard coordinate pairs (x, y). But remember: inside a NumPy data structure, images are structured as Rows first, then Columns. That means when you slice into your image array to read a pixel’s color values, you must pass the parameters as frame[y, x]. If you pass it as [x, y], your program is going to index out of bounds and crash hard.

Furthermore, always remember that OpenCV handles colors in a BGR (Blue, Green, Red) sequence, not RGB. When we extract those elements, they unpack straight into valB, valG, valR.

3. Masking and Bitwise Isolation

To lock onto our target color, we use cv2.inRange() to look at our HSV frame and check it against our lower constraint (LC) and upper constraint (UC). This generates a Mask—a pure black-and-white image where pixels within the target color space are completely white (255), and everything else is completely black (0).

By taking that mask and running a fast bitwise operation

We force the computer to evaluate every single pixel. If the mask is zero, the output is blacked out. If the mask is active, the original, rich color information passes through perfectly, isolating our target object from the background noise instantly.

Get your circuits wired up, get this script running on your machine, and let me know in the comments section below what kind of performance numbers you are pulling on your local workbench. I’ll catch you guys in the next lesson!

Remember we are still setting the LED color to the color that cursor is pointing at. This is the circuit for connecting the RGB LED.

Fusion Hat Circuit Diagram
This is the circuit we will use moving forward in the class
AI On the Edge, Raspberry Pi

AI on the Edge LESSON 25: Create Region of Interest (ROI) in openCV Using the Mouse

Well, hello there! I’m absolutely delighted you could join me today. If you’ve been following along with our journey into AI on the Edge, you know that we are getting closer and closer to building some truly powerful, real-world computer vision applications. But before we can get to the fancy AI stuff, we have to master the fundamentals. Today, we’re tackling something that is going to make your projects look—and feel—a whole lot more professional: creating a Region of Interest (ROI) using the mouse.

Why Do We Need an ROI?

Think about it. When you’re processing a video feed, you’re usually wasting a ton of compute power looking at things that don’t matter. Maybe you’re tracking a ball on a table, but your camera is seeing the whole room. Why process the walls and the ceiling when you only care about the table? By defining an ROI, we tell our code: “Ignore everything else. Only look here.” It saves processing time, it reduces noise, and it makes your AI much more accurate.

Interacting with OpenCV

In this lesson, we’re going to step beyond simple static code. I’m going to show you how to use OpenCV’s callback functions to make your program “live.” We’ll use the mouse to click and drag a rectangle directly on the video feed to define our ROI in real-time. It’s interactive, it’s intuitive, and it’s a vital skill for anyone building real-world vision systems.

The Code

Now, I’ve put a lot of work into making this code clean and easy to follow. You’ll see exactly how we capture those mouse events—cv2.EVENT_LBUTTONDOWN, cv2.EVENT_MOUSEMOVE, and cv2.EVENT_LBUTTONUP—to create that bounding box dynamically.

Putting It to the Test

I want you to take this code, run it on your Jetson, and play around with it. Try defining different regions. Notice how the frame rate stays steady because we aren’t bogging down the CPU with unnecessary pixels. This is the “Edge” part of “AI on the Edge”—making smart, efficient decisions right where the data is being captured.

I can’t wait to see what you build with this. As always, keep those questions coming, stay curious, and most importantly—don’t get discouraged! We’re doing hard things, and you are doing a great job.

I’ll see you in the next lesson!

What questions do you have about implementing ROI in your own computer vision projects? Post them in comments on the video! Thanks for learning.

We will be using the circuit used in the earlier lessons:

Fusion Hat Circuit Diagram
This is the circuit we will use moving forward in the class

 

AI On the Edge, Raspberry Pi

AI on the Edge LESSON 24: Processing Mouse Events in OpenCV on Pi 5

Welcome back, everyone! In our last lesson, we learned how to use matrix slicing to hardcode a Region of Interest (ROI) into our frames. That was a great static approach, but today we are taking interactivity to a whole new level.

In this lesson, you are going to learn how to catch Mouse Events inside your OpenCV windows. Instead of guess-and-checking coordinates in your code, you will be able to click anywhere on your live video stream to instantly grab the precise (x, y) pixel coordinates and read the exact color value of the pixel right under your mouse pointer. This is the foundational mechanic you need to build interactive, point-and-click AI applications.

The Core Concept: Mouse Callbacks and Global Frames

To listen for mouse clicks or movement, OpenCV uses what is called a Callback Function. You tell OpenCV: “Hey, keep an eye on this specific window. If the user does anything with the mouse inside it, instantly jump over to my custom function and tell me what happened.”

We set this up using:

cv2.setMouseCallback(‘Camera’, mouseAction)

The [y, x] Matrix Inversion Trap

There is a massive mathematical trap that catches almost every beginner when they start mapping mouse clicks to image matrices:

  • OpenCV Mouse Coordinates: When you move your mouse, OpenCV tracks position using standard Cartesian geometry: (x, y), where x is the column (horizontal distance from the left) and y is the row (vertical distance from the top).

  • NumPy Array Coordinates: When you plug those numbers into your image array to inspect a pixel, NumPy expects matrix indexing: [row, column].

Because rows correspond to the height (y) and columns correspond to the width (x), you must always invert the coordinates when accessing the frame array:

If you try to pass frame[x, y], your program will either crash with an “index out of bounds” error or return data from the completely wrong part of the image!

The Python Code Developed in This Lesson

Here is the complete, streamlined script we built during today’s tutorial. Copy this code into your workspace on your Raspberry Pi 5, fire it up, and watch your terminal output as you click around the video window.

We first developed this program as a simple example of processing mouse clicks, and print the detected event:

In order to make the program more useful, we developed this code that monitors the position of the mouse cursor, and reports the color of the pixel the mouse points at. The values are printed as labels on the openCV frame:

We can now take the project to the next level by setting the LED color to the color pointed at by the cursor in the openCV window. We will be using our standard circuit we have used in the earlier lessons.

Fusion Hat Circuit Diagram
This is the circuit we will use moving forward in the class

This is the code we developed to set the LED color based on the pixel position of the cursor in the openCV window.

Homework Assignment

 

Alright, it’s time to put this knowledge to work. Your homework assignment is to turn this simple reporting tool into an interactive, dynamic ROI selector. The homework is to  first create a text display under the FPS on the frame that show RGB value at the pixel position the mouse is pointing at, and the pixel location.

 Your homework assignment is to turn this simple reporting tool into an interactive, dynamic ROI selector.

  1. Start with your clean 1280×720 live camera stream.

  2. Modify your mouseAction callback function to look for specific mouse clicks.

  3. The Target Mechanic: When you Left-Click on the video window, store those specific coordinates as your upper-left corner. When you release the click, store those coordinates as your lower-right corner. As you are selecting, draw a live box outline over your ROI

  4. Using those two dynamic coordinate sets, use matrix slicing to pull a clean Region of Interest (ROI) out of the frame and instantly display it in a completely separate, standalone window called “Target ROI”.

  5. Safety Requirement: Make sure your code can handle clicks in any order without crashing (e.g., if a user right-clicks higher or further left than their left-click, write the conditional logic to sort the indices properly before slicing).

Get your black coffee ready, write your logic step-by-step from scratch, and do not copy code you can’t explain. Post your homework solution video on YouTube and drop a link in the comments section below so I can see who is running with the big dogs!