AI on the Edge LESSON 59: Object Detection with Active Scanning in MediaPipe with Pi 5

Hello everyone! Paul McWhorter here from toptechboy.com, and welcome back to our thrilling adventure into computer vision and edge AI. Today, we are dropping Lesson 59 of our AI on the Edge series.

In our previous sessions, we got our Raspberry Pi 5 looking at the world using MediaPipe object detection, we learned how to lock onto targets using pan-and-tilt servos, and we even brought in speech-to-text and text-to-speech capabilities via the Fusion Hat so our system could actually listen and talk back to us.

But there has been one nagging limitation: Static tracking. Up until now, if your object of interest wasn’t sitting right inside the camera’s initial field of view, your Pi just sat there blindly staring straight ahead, waiting for magic to happen.

Well, no more! Today, we are leveling up our system to feature Active Scanning.

The Motivation: From Passive Waiting to Active Hunting

Think about how human vision works. If you are looking for your coffee mug in a messy room, you don’t just stare at the wall and hope it walks past your nose. You move your head, you scan the environment, and your eyes sweep back and forth across different zones until you spot what you are looking for.

That is exactly what we are building in Lesson 59. We are combining three powerful pillars:

  1. Computer Vision (MediaPipe & OpenCV): Running blazing-fast object detection on the Raspberry Pi 5 using the PiCamera2 module.

  2. Robotic Actuation (Pan and Tilt Servos): Giving our camera a physical body with a dynamic range of motion.

  3. Conversational AI (Speech-to-Text & Text-to-Speech): Enabling voice command processing so you can simply tell your Pi what to find, while the system talks back to confirm its state.

How the Active Scanning System Works

In this lesson, our program operates across a few distinct operational modes managed seamlessly through threaded execution:

  • The Voice Listener Thread: Running independently in the background, this thread listens for your spoken commands using the speech-to-text engine. You can say things like “find”, “track”, “scan”, or name a specific object of interest (like “cup”, “person”, or “cell phone”). Saying “quit” gracefully terminates the application with a polite audio farewell.

  • Find Mode: The system rests or holds position while waiting for a target instruction or manual queue update.

  • Scan Mode: If the camera doesn’t immediately see your requested object of interest, it enters active scan mode. The pan servo sweeps smoothly back and forth across its horizontal range, stepping up or down the tilt angle after each pass. It systematically sweeps the room like a radar system.

  • Track Mode: The moment MediaPipe detects the target object with high confidence during a scan, the system instantly halts the scanning pattern, announces “Target Found” via text-to-speech, and switches over to active track mode. Using error correction based on the center of the bounding box versus the center of the screen, the servos continuously adjust to keep the target dead-centered in the frame.

What You Will Learn in the Video

In the accompanying video lesson, we break down the implementation step-by-step:

  • How to manage multi-threaded voice queues so your UI and camera feed don’t stutter while listening for audio.

  • Implementing state-machine logic to toggle cleanly between find, scan, and track modes.

  • Tuning servo gains, deadbands, and sweep step-sizes to achieve smooth, organic robotic movement without jitter.

  • Integrating real-time bounding box feedback, frames-per-second (FPS) monitoring, and dynamic visual overlays using OpenCV.

Grab your Raspberry Pi 5, fire up your Fusion Hat, and let’s write some code that brings hardware to life! Watch the full video lesson above, and let’s dive straight in!