Uncategorized

test

import cv2
import time
from picamera2 import Picamera2
import mediapipe as mp
piCam = Picamera2(1)
W=1280
H=720
tStart = time.time()
fps = 0
RES = (W,H)
piCam.preview_configuration.main.size = RES
piCam.preview_configuration.main.format = “RGB888”
piCam.preview_configuration.controls.FrameRate=60
piCam.preview_configuration.align()
piCam.configure(“preview”)
piCam.start()

textLowerLeft = (int(W*.01),int(H*.05))
fontFace = cv2.FONT_HERSHEY_SIMPLEX
fontThickness = int(W/425)
fontScale = H*.0015
fontColor = (0,0,255)

faceFinder = mp.solutions.face_detection.FaceDetection(
model_selection = 0,
min_detection_confidence = .5)
mpDraw = mp.solutions.drawing_utils

while True:
deltaT = time.time() – tStart
tStart=time.time()
fps = fps*.95 + (1/deltaT)*.05
frame= piCam.capture_array()
frame=cv2.flip(frame,-1)

rgbFrame = cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)
results = faceFinder.process(rgbFrame)
if results.detections:
for detection in results.detections:
#print(detection.location_data.relative_bounding_box)
#print(detection)
bbox =detection.location_data.relative_bounding_box
x = int(bbox.xmin*W)
y = int(bbox.ymin*H)
w = int(bbox.width*W)
h = int(bbox.height*H)
cv2.rectangle(frame,
(x,y),
(x+w,y+h),
(0,255,255),
3)
keypoints =detection.location_data.relative_keypoints
#print(keypoints)
for kp in keypoints:
kpX = int(kp.x*W)
kpY = int(kp.y*H)
cv2.circle(frame,
(kpX,kpY),
15,
(255,0,0),
-1)
myText = “FPS: “+str(round(fps,1))
cv2.putText(frame,myText,textLowerLeft,fontFace,fontScale,fontColor,fontThickness)
cv2.imshow(“Camera”, frame)
cv2.moveWindow(“Camera”,0,60)
if cv2.waitKey(1)==ord(‘q’):
break
cv2.destroyAllWindows()
print(‘Program Terminated’)

AI On the Edge, Raspberry Pi

AI on the Edge LESSON 39: Understanding MediaPipe Data Structures

In this video lesson I show you how to understand the data structures returned by MediaPipe. I show you how to peel the data structure back, to get at the useful information.

When you run face detection with MediaPipe, the results object it returns is not a normal dictionary or list. It is a special custom object called SolutionOutputs. The easiest way to explore it is to start by checking the main attribute: results.detections. This is a Python list that contains one entry for every face detected in the current frame. If no faces are found, results.detections will be None or an empty list.

To extract useful information, you loop through results.detections. Each item in that list is a Detection object. From this object, you can access two main things: the confidence score using detection.score[0], and the location data using detection.location_data. Inside location_data, you will find relative_bounding_box (which gives you xmin, ymin, width, and height as values between 0 and 1) and relative_keypoints (a list of 6 facial points such as eyes, nose, and mouth).

The standard method is to first get the frame’s height and width, then multiply the normalized values (like xmin and width) by the actual pixel dimensions of the image to convert them into usable pixel coordinates. You can then use these coordinates with OpenCV functions such as cv2.rectangle() for the box or cv2.circle() for the keypoints.

By using simple print(type()), print(dir()), and print() statements on results, results.detections, and individual detection objects, you can quickly discover the full structure. This step-by-step approach — starting from results → detections → individual detection → location_data — lets you reliably reach all the useful information MediaPipe provides.

Below is the code we developed in the video.

 

 

AI On the Edge, Raspberry Pi

AI on the Edge 38: Using MediaPipe for Face Recognition on the Raspberry Pi 5

In this video lesson we introduce you to MediaPipe. The OS we had you flash in LESSON 1 already has the MediaPipe framework installed, and all the needed and working dependencies. If you have installed that OS and not modified it, this and future lessons will work. If you find dependency errors, you might need to reflash the original OS.

MediaPipe is a free, open-source framework developed by Google that makes it much easier to add advanced computer vision and AI features to your Python programs. It is especially popular among developers who use OpenCV because it works seamlessly with it and delivers excellent real-time performance, even on devices like the Raspberry Pi 5.

With MediaPipe, you can quickly add powerful capabilities such as face detection, face mesh (detailed facial landmarks), hand tracking, body pose estimation, and more — all without having to write complex deep learning code from scratch. It comes with pre-trained machine learning models that are optimized for speed, allowing your programs to run smoothly at 30 frames per second or higher.

The biggest advantage for Python + OpenCV users is its simplicity. You capture video frames using OpenCV or picamera2, pass them to MediaPipe for processing, and then draw the results (such as bounding boxes or landmarks) back onto your image using normal OpenCV functions. This combination gives developers a fast and straightforward way to build interactive computer vision projects like face trackers, gesture-controlled robots, or smart camera applications.

In short, MediaPipe acts as a powerful, easy-to-use toolkit that bridges the gap between OpenCV and modern AI vision technology.

We introduce you to MediaPipe using a simple example where we create a faceFinder on our Raspberry Pi 5.

 

AI On the Edge, Raspberry Pi

AI on the Edge LESSON 37: Using RTSP and IP Cameras in OpenCV on Raspberry Pi 5

The code below shows the work we did in this lesson.

AI on the Edge Lesson 37: Using RTSP and IP Cameras in OpenCV on Raspberry Pi 5

Hey guys! Welcome back to our AI on the Edge series. In our previous lessons, we’ve had a blast working with standard USB webcams, but if you are building a real-world computer vision application, an automation rig, or a security monitoring setup around your home or farm, USB cables just aren’t going to cut it. You need to pull video feeds from remote IP cameras using the Real-Time Streaming Protocol (RTSP).

Today, we are taking that exact step on the Raspberry Pi 5, connecting to an IP camera, streaming the feed smoothly into OpenCV, and—most importantly—solving the dreaded latency problem that plagues RTSP feeds.

The Big Challenge: Conquering RTSP Latency

If you’ve ever tried pulling an RTSP stream into OpenCV straight out of the box, you’ve probably noticed something frustrating: the video lags behind real-time, sometimes by several seconds or even tens of seconds.

Why does that happen? Because by default, FFmpeg and OpenCV buffer incoming frames to ensure smooth playback. But when you are doing computer vision, AI inferencing, or real-time tracking on the edge, you don’t want old history—you want right now.

To fix that, we pass the cv2.CAP_FFMPEG backend flag and immediately flush the buffer by setting the property to 0. This forces OpenCV to drop the backlog and grab the absolute newest frame available from the camera stream, keeping your Pi 5 processing live data in real-time.

Understanding the Script Structure

Let’s break down the key parts of today’s implementation:

  • Credentials & Resolution: We import a separate secret file to keep our camera IP addresses, usernames, and passwords safe and out of public repositories. We lock our resolution at 1280×720 to balance crisp detail with the Pi 5’s processing overhead.
  • Smooth FPS Calculation: Instead of a jittery raw frame-rate readout, we use an exponential moving average to give us a stable, readable performance metric on screen.
  • The Display Window: We configure a GUI window using OpenCV’s window flags so we can easily position and resize our output feed on the desktop.

Drop Your Questions Below

Working with network streams can sometimes be tricky depending on your specific camera’s firmware, codec settings, and network stability. If you run into any connection drops or lag spikes on your Raspberry Pi 5, drop a comment on the video!

Keep building, stay creative, and I will see you guys in Lesson 38!

Here is the code developed in the video

 

AI On the Edge, Raspberry Pi

AI on the Edge LESSON 36: Select Active Camera in OpenCV With Voice Commands

Welcome back, makers, engineers, and AI enthusiasts! In our previous lessons, we built out robust multi-camera setups, streaming feeds from both Raspberry Pi cameras and high-definition USB cameras. But watching four feeds in separate tiles is only half the battle. What happens when you want to interact with your system hands-free?

In this lesson, we are taking our edge vision projects to the next level by integrating voice commands. You will learn how to run a dedicated speech-to-text thread in the background, catch voice triggers safely using a thread-safe queue, and dynamically switch your active main camera feed in OpenCV on the fly—just by speaking.

What You Will Learn in This Lesson

  • Multi-Threaded Speech Recognition: How to run the stt listener in a separate daemon thread so it never blocks or stutters your high-FPS video processing loop.

  • Thread-Safe Communication: Using Python’s Queue module to pass voice commands seamlessly from the background listening thread into your main application loop.

  • Handling STT Variations: Accounting for common speech-to-text homophone variations (like “two”, “to”, and “too”, or “four” and “for”) to make your voice control robust and reliable.

  • Dynamic Frame Routing: Mapping spoken commands to specific camera streams (piCam1, piCam2, usbCam1, and usbCam2) and updating your primary display window instantly.

  • Organized Window Layouts: Positioning and sizing multiple OpenCV windows on your desktop workspace for a clean, professional multi-camera dashboard.

Step-by-Step Breakdown of the Script

1. Setting Up the Background Voice Thread

When working with real-time video processing in OpenCV, blocking functions are your worst enemy. If you call a speech recognition listening function directly inside your main while loop, your video frames will freeze while waiting for audio input.

To prevent this, we initialize a background worker function (getCamera) and launch it as a daemon thread:

  • The thread continuously listens for your voice commands using the Fusion Hat STT module.

  • Once a command is captured, it strips whitespace, checks for exit triggers (like saying 'quit'), and pushes valid commands directly into our commandQ.

2. Initializing Multiple Cameras

Our setup harnesses the full power of the hardware by combining native Raspberry Pi camera interfaces with standard USB webcams:

  • Pi Cameras: Configured via Picamera2 at a crisp 1280x720 resolution with RGB888 formatting running smoothly at 60 frames per second.

  • USB Cameras: Initialized through OpenCV’s VideoCapture class, set to target resolution and optimized for 30 FPS.

3. Processing and Routing Commands in the Main Loop

Inside the primary application loop, our script handles three core tasks simultaneously:

  1. Calculate Performance: Continuously tracks and smooths out the frames-per-second (FPS) metric so you can monitor system load.

  2. Check the Command Queue: Non-blockingly checks if the background thread has dropped a new camera selection into commandQ. If a new command is waiting, it updates the mainCam variable.

  3. Route the Main Frame: Evaluates the active camera string—including clever fallback checks for common voice misinterpretations like 'camera to' or 'camera for'—and assigns the corresponding video stream to mainFrame.

4. Managing Your OpenCV Windows

To give you a complete command-center experience, the script generates a large primary display window for your active camera view, accompanied by a clean row of smaller preview tiles across the bottom of your screen for all four connected feeds.

 

Making The World a Better Place One High Tech Project at a Time. Enjoy!