Tag Archives: Raspberry Pi 5

AI on the Edge LESSON 28: Use Pan Tilt Camera to Track Object of Interest in OpenCV

In this video lesson we learn how to track our object of interest using the Pan Tilt camera. The camera will constantly adjust position to keep the Object of Interest in the center of the camera frame. We identify the Object of Interest based on Color, as we learned in earlier lessons. In this video, we developed the code below:

 

AI on the Edge LESSON 26: Understanding the HSV Color Space in OpenCV

Hey guys, welcome back to the channel. If you’ve been following along, you know we’ve been pushing our hardware absolutely down into the dirt. We’ve been running large language models right on the edge, pushing our boards hot and heavy until the silicon is screaming and the thermal throttling flags are popping up all over the place.

But today, we are stepping away from the heavy-compute server terminals, and we are getting back to our roots: Real-Time Computer Vision and Embedded Control. In our previous lessons, we learned how to hook up our high-speed camera, capture raw frames, and interact with individual pixels using standard RGB/BGR math. But today, we are going to look under the hood of a completely different way of representing color: The HSV Color Space (Hue, Saturation, Value).

If you try to track objects or isolate specific colors in the traditional RGB world, you are going to pull your hair out. The moment a shadow hits your object or the room lighting changes, your Red, Green, and Blue values completely collapse. By shifting our mathematics into the HSV space, we can lock onto a color’s pure identity regardless of whether it is sitting under a bright laboratory spotlight or a dim shadow.

Not only are we going to capture and process these video streams at a smooth-as-silk 60 frames per second, but we are also going to translate that raw visual math directly into the physical world. We are using our trusty SunFounder Fusion HAT+ to dynamically pulse an external RGB LED, matching its brightness and color hue perfectly to whatever pixel your mouse is clicking on in real-time.

Let’s look at the blueprint to make this happen.

The Complete Python Code

Here is the clean, un-guardrailed Python script for today’s lesson. Paste this directly into your local terminal workspace. No bloated libraries, no unnecessary frameworks—just pure, deliberate engineering.

Under the Hood: How the Code Works

1. The Real-Time Telemetry Smooth Filter

Look closely at how we calculate our frames-per-second metric inside the main processing loop:

If you simply print out the raw math of 1 / deltaT, your numbers on the screen are going to jump all over the place like a wild animal. By applying a 95% historical weight and a 5% instant weight, we create a low-pass software filter that smoothly tracks our true hardware operational speed without erratic layout jitter.

2. The Mouse Vector and BGR Array Sequence

When your mouse triggers an event over the window, OpenCV passes us the standard coordinate pairs (x, y). But remember: inside a NumPy data structure, images are structured as Rows first, then Columns. That means when you slice into your image array to read a pixel’s color values, you must pass the parameters as frame[y, x]. If you pass it as [x, y], your program is going to index out of bounds and crash hard.

Furthermore, always remember that OpenCV handles colors in a BGR (Blue, Green, Red) sequence, not RGB. When we extract those elements, they unpack straight into valB, valG, valR.

3. Masking and Bitwise Isolation

To lock onto our target color, we use cv2.inRange() to look at our HSV frame and check it against our lower constraint (LC) and upper constraint (UC). This generates a Mask—a pure black-and-white image where pixels within the target color space are completely white (255), and everything else is completely black (0).

By taking that mask and running a fast bitwise operation

We force the computer to evaluate every single pixel. If the mask is zero, the output is blacked out. If the mask is active, the original, rich color information passes through perfectly, isolating our target object from the background noise instantly.

Get your circuits wired up, get this script running on your machine, and let me know in the comments section below what kind of performance numbers you are pulling on your local workbench. I’ll catch you guys in the next lesson!

Remember we are still setting the LED color to the color that cursor is pointing at. This is the circuit for connecting the RGB LED.

Fusion Hat Circuit Diagram
This is the circuit we will use moving forward in the class

AI on the Edge LESSON 20: Resizing, Moving, Converting and Tiling Video frames in OpenCV

Welcome back to the AI on the Edge class series! In this lesson, we are diving deep into some of the most critical foundational skills you need when working with video streams on edge devices: Resizing, Moving, Converting, and Tiling video frames using OpenCV.

When you are developing real-world AI applications on the edge, you rarely just display a single camera feed. You often need to manipulate frames to feed them into your AI models, look at grayscale versions for edge detection, or arrange multiple windows on your desktop neatly so you can monitor your data visually.

If you want to follow along exactly as we do in the video, make sure you have your Raspberry Pi 5 set up with your Camera Module.

What We Cover in This Lesson

  • Fixed FPS Estimation: We continue using our robust low-pass filter formula to track smooth, non-jittery frames-per-second data directly on the video frame.

  • Creating Named Windows: Understanding how cv2.namedWindow() combined with cv2.WINDOW_GUI_NORMAL gives you absolute programmatic control over the placement of your displays.

  • Resizing & Moving Windows: How to accurately position multiple OpenCV windows on your screen using specific coordinates while accounting for operating system taskbars and window decorative margins.

  • Frame Manipulation: Using cv2.resize() to scale down video frames and cv2.cvtColor() to transform the color space from BGR to grayscale.

  • Window Tiling: Arranging a main camera view, a scaled-down color view, and a scaled-down grayscale view in a perfect grid layout on your desktop.

The Complete Lesson 20 Code

Below is the complete Python code we developed during this lesson. It sets up your hardware camera stream, calculates running performance metrics, processes three distinct variations of the video feed, and tiles them cleanly on your screen.

 

AI on the Edge LESSON 19: Create a Bouncing Box in OpenCV On Raspberry Pi

Hey everyone, Paul McWhorter here!

Welcome back to the AI on the Edge series! In today’s lesson, we’re going to have some fun and take our first real steps into computer vision animation.

We’re going to create a colorful box that bounces around the screen like an old-school screensaver, while displaying a live FPS counter so we can see how well our Raspberry Pi is handling the workload.

Even though it looks simple, this project teaches you several foundational skills you’ll use again and again in computer vision:

  • Working with coordinates and drawing shapes in OpenCV
  • Creating smooth real-time animation
  • Detecting boundaries and reversing direction
  • Calculating and displaying live FPS

These are the same techniques you’ll build on later when we start doing object tracking, collision detection, and more advanced AI vision projects.


What You Learned in This Lesson

  • How to draw filled rectangles on a live video stream
  • How to move objects smoothly frame by frame
  • How to make objects “bounce” realistically off screen edges
  • A clean method for calculating and displaying FPS
  • Using variables to easily control size, position, speed, and color

This bouncing box may look basic, but once you understand how to do this, you can create all kinds of animated graphics that interact with what the camera sees.


Pro Tip: After you get it working, play around with the speed, box size, and colors. Try making multiple bouncing boxes with different speeds and directions — it’s a great way to experiment!


Ready for more? In the next lesson, we’re going to kick things up a notch and start working with multiple objects and more complex interactions.

Keep building, keep learning, and I’ll see you in the next video!

Paul McWhorter

For your convenience, this is the code we developed in the video.

 

AI on the Edge LESSON 18: Display Frames Per Second (FPS) on openCV Video Window

In today’s lesson, we add a clean, real-time Frames Per Second (FPS) counter directly onto our live OpenCV video window. Displaying FPS on screen is an essential tool for anyone working with camera-based AI projects on the Raspberry Pi. It gives you immediate feedback on your actual processing performance, helps with optimization, and makes your projects look more professional and polished.

In this lesson, we configure the Picamera2 library to run at 1280×720 resolution with a target of 60 frames per second. We then implement a smoothed FPS calculation using a weighted rolling average, which prevents the displayed value from jumping around wildly. Finally, we overlay the FPS text in the lower-left corner of the video frame using OpenCV’s putText() function, with font size and thickness that scale appropriately with the resolution.

This technique forms an important foundation for future lessons, as we will continue adding more information and graphics directly onto the live video stream. Understanding how to efficiently display performance metrics is key to developing responsive and practical edge AI applications.

In this lesson, this is the code which we develop: