This is Getting Crazy Cool!Hey guys, welcome back to the Fusion AI Lab series, AI on the Edge! In Lesson 44 we just took things up another notch. We’re now running MediaPipe Face Mesh on our Raspberry Pi 5, pulling all 468 facial landmarks in real time, and then drawing a clean, smooth face contour directly onto a tiny 128×64 OLED display!

That’s right — your face is now living on that little monochrome OLED in real time! Every eyebrow raise, smile, head tilt, and eye movement gets faithfully reproduced on the display. We’re pulling the camera feed with picamera2, processing it with MediaPipe’s powerful face mesh, scaling the landmarks down to OLED resolution, and then drawing the official FACEMESH_CONTOURS connections using PIL. The result is surprisingly smooth and fun to watch!This project really shows the power of combining modern AI vision tools with simple embedded hardware. Watching your own face rendered in real time on a tiny OLED is just plain awesome — it feels like sci-fi stuff from just a few years ago, and now we’re doing it on the Edge with our Fusion AI Lab kit!

If you’ve been following along, you’re starting to see how powerful these tools are becoming. We’re no longer just detecting faces — we’re understanding the structure of the face and visualizing it however we want. And we’re just getting started!

So fire up your Raspberry Pi, grab that OLED, and let’s keep pushing the limits. You’re doing some seriously cool stuff!

# ====================================================================
# DISCLAIMER:
# This code is provided as-is for educational and experimental 
# purposes only. The author makes no representations or warranties of 
# any kind concerning the safety, suitability, or accuracy of this 
# code. Use at your own risk. The author assumes no liability for any 
# damages, system failures, security breaches, or network issues 
# resulting from the use or implementation of this script.
# ====================================================================
 
import cv2
import time
from picamera2 import Picamera2
import mediapipe as mp
import numpy as np
#################################
import board
import adafruit_ssd1306
from PIL import Image, ImageDraw, ImageFont
import time

wOLED = 128
hOLED = 64
I2C_ADDRESS = 0x3C
i2c = board.I2C()
oled = adafruit_ssd1306.SSD1306_I2C(wOLED, hOLED, i2c, addr = I2C_ADDRESS)

#################################
W=1280
H=720
tStart = time.time()
fps = 0

piCam = Picamera2(1)
piCam.preview_configuration.main.size = (W, H)
piCam.preview_configuration.main.format = "RGB888"
piCam.preview_configuration.controls.FrameRate = 60
piCam.preview_configuration.align()
piCam.configure("preview")
piCam.start()

textLowerLeft = (int(W*.01),int(H*.05))
fontFace = cv2.FONT_HERSHEY_SIMPLEX
fontThickness = int(W/425)
fontScale = H*.0015
fontColor = (0,0,255)

faceMesh = mp.solutions.face_mesh.FaceMesh(
    
    max_num_faces=1,
    refine_landmarks=True,
    min_detection_confidence=0.5,
    min_tracking_confidence=0.5)

cv2.namedWindow('Camera',cv2.WINDOW_GUI_NORMAL)
cv2.moveWindow('Camera',0,65)
cv2.resizeWindow('Camera',W,H)
while True:
    deltaT = time.time() - tStart
    tStart=time.time()
    fps = fps*.95 + (1/deltaT)*.05
    frame = piCam.capture_array()
    frame = cv2.flip(frame, -1)

    rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
    results = faceMesh.process(rgb)
    
    if results.multi_face_landmarks:
        for faceLandMarks in results.multi_face_landmarks:
            #print(faceLandmarks)
            image = Image.new("1",(wOLED,hOLED))
            draw = ImageDraw.Draw(image)
            lmAll = []
            for lm in faceLandMarks.landmark:
                xOLED = int(lm.x*wOLED)
                yOLED = int(lm.y*hOLED)
                draw.point((xOLED,yOLED), fill = 255)
                #print(xOLED,yOLED)
                lmAll.append((xOLED,yOLED))
            for idx in mp.solutions.face_mesh.FACEMESH_CONTOURS:
                #print(idx)
                lineStart = idx[0]
                lineEnd = idx[1]
                pt1 = lmAll[lineStart]
                pt2 = lmAll[lineEnd]
                draw.line((pt1[0],pt1[1],pt2[0],pt2[1]), width=1, fill = 255)


            #print('XXXXXXXXXXXXXXXXXXx')
            oled.image(image)
            oled.show()
        #print()

#     if results.multi_face_landmarks:
#         for faceLandmarks in results.multi_face_landmarks:
#             # Optional second pass for glow effect
#             mp.solutions.drawing_utils.draw_landmarks(
#                 image=frame,
#                 landmark_list=faceLandmarks,
#                 connections=mp.solutions.face_mesh.FACEMESH_TESSELATION,
#                 landmark_drawing_spec=None,
#                 connection_drawing_spec=mp.solutions.drawing_utils.DrawingSpec(
#                     color=(100, 100, 100),
#                     thickness=1
#                 )
#             )
#             # === Best looking translucent mesh ===
#             mp.solutions.drawing_utils.draw_landmarks(
#                 image=frame,
#                 landmark_list=faceLandmarks,
#                 connections=mp.solutions.face_mesh.FACEMESH_CONTOURS,
#                 landmark_drawing_spec=None,                    # Hide dots
#                 connection_drawing_spec=mp.solutions.drawing_utils.DrawingSpec(
#                     color=(0, 0, 180),      # Bright cyan/teal
#                     thickness=7
#                 )
#             )
# 
#             # Optional: Extra detail on irises (makes eyes look better)
#             mp.solutions.drawing_utils.draw_landmarks(
#                 image=frame,
#                 landmark_list=faceLandmarks,
#                 connections=mp.solutions.face_mesh.FACEMESH_IRISES,
#                 landmark_drawing_spec=None,
#                 connection_drawing_spec=mp.solutions.drawing_utils.DrawingSpec(
#                     color=(255, 0, 0),
#                     thickness=7
#                 )
#             )

    myText = "FPS: "+str(round(fps,1))
    cv2.putText(frame,myText,textLowerLeft,fontFace,fontScale,fontColor,fontThickness)

    cv2.imshow("Camera", frame)

    if cv2.waitKey(1) == ord('q'):
        break

cv2.destroyAllWindows()
piCam.stop()
oled.fill(0)
oled.show()

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

# ====================================================================

# DISCLAIMER:

# This code is provided as-is for educational and experimental

# purposes only. The author makes no representations or warranties of

# any kind concerning the safety, suitability, or accuracy of this

# code. Use at your own risk. The author assumes no liability for any

# damages, system failures, security breaches, or network issues

# resulting from the use or implementation of this script.

# ====================================================================

import cv2

import time

from picamera2 import Picamera2

import mediapipe as mp

import numpy as np

#################################

import board

import adafruit_ssd1306

from PIL import Image, ImageDraw, ImageFont

import time

wOLED = 128

hOLED = 64

I2C_ADDRESS = 0x3C

i2c = board.I2C()

oled = adafruit_ssd1306.SSD1306_I2C(wOLED, hOLED, i2c, addr = I2C_ADDRESS)

#################################

W=1280

H=720

tStart = time.time()

fps = 0

piCam = Picamera2(1)

piCam.preview_configuration.main.size = (W, H)

piCam.preview_configuration.main.format = "RGB888"

piCam.preview_configuration.controls.FrameRate = 60

piCam.preview_configuration.align()

piCam.configure("preview")

piCam.start()

textLowerLeft = (int(W*.01),int(H*.05))

fontFace = cv2.FONT_HERSHEY_SIMPLEX

fontThickness = int(W/425)

fontScale = H*.0015

fontColor = (0,0,255)

faceMesh = mp.solutions.face_mesh.FaceMesh(

max_num_faces=1,

refine_landmarks=True,

min_detection_confidence=0.5,

min_tracking_confidence=0.5)

cv2.namedWindow('Camera',cv2.WINDOW_GUI_NORMAL)

cv2.moveWindow('Camera',0,65)

cv2.resizeWindow('Camera',W,H)

while True:

deltaT = time.time() - tStart

tStart=time.time()

fps = fps*.95 + (1/deltaT)*.05

frame = piCam.capture_array()

frame = cv2.flip(frame, -1)

rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)

results = faceMesh.process(rgb)

if results.multi_face_landmarks:

for faceLandMarks in results.multi_face_landmarks:

#print(faceLandmarks)

image = Image.new("1",(wOLED,hOLED))

draw = ImageDraw.Draw(image)

lmAll = []

for lm in faceLandMarks.landmark:

xOLED = int(lm.x*wOLED)

yOLED = int(lm.y*hOLED)

draw.point((xOLED,yOLED), fill = 255)

#print(xOLED,yOLED)

lmAll.append((xOLED,yOLED))

for idx in mp.solutions.face_mesh.FACEMESH_CONTOURS:

#print(idx)

lineStart = idx[0]

lineEnd = idx[1]

pt1 = lmAll[lineStart]

pt2 = lmAll[lineEnd]

draw.line((pt1[0],pt1[1],pt2[0],pt2[1]), width=1, fill = 255)

#print('XXXXXXXXXXXXXXXXXXx')

oled.image(image)

oled.show()

#print()

# if results.multi_face_landmarks:

# for faceLandmarks in results.multi_face_landmarks:

# # Optional second pass for glow effect

# mp.solutions.drawing_utils.draw_landmarks(

# image=frame,

# landmark_list=faceLandmarks,

# connections=mp.solutions.face_mesh.FACEMESH_TESSELATION,

# landmark_drawing_spec=None,

# connection_drawing_spec=mp.solutions.drawing_utils.DrawingSpec(

# color=(100, 100, 100),

# thickness=1

# )

# # === Best looking translucent mesh ===

# mp.solutions.drawing_utils.draw_landmarks(

# image=frame,

# landmark_list=faceLandmarks,

# connections=mp.solutions.face_mesh.FACEMESH_CONTOURS,

# landmark_drawing_spec=None, # Hide dots

# connection_drawing_spec=mp.solutions.drawing_utils.DrawingSpec(

# color=(0, 0, 180), # Bright cyan/teal

# thickness=7

# )

# # Optional: Extra detail on irises (makes eyes look better)

# mp.solutions.drawing_utils.draw_landmarks(

# image=frame,

# landmark_list=faceLandmarks,

# connections=mp.solutions.face_mesh.FACEMESH_IRISES,

# landmark_drawing_spec=None,

# connection_drawing_spec=mp.solutions.drawing_utils.DrawingSpec(

# color=(255, 0, 0),

# thickness=7

# )

myText = "FPS: "+str(round(fps,1))

cv2.putText(frame,myText,textLowerLeft,fontFace,fontScale,fontColor,fontThickness)

cv2.imshow("Camera", frame)

if cv2.waitKey(1) == ord('q'):

break

cv2.destroyAllWindows()

piCam.stop()

oled.fill(0)

oled.show()

In this lesson we are still using our standard Fusion AI Lab Kit cirtuit from the earlier lessons:

Fusion Hat Circuit Diagram — This is the circuit we will use moving forward in the class

We have also connected the OLED to the above circuit according to this diagram:

AI on the Edge LESSON 43: Adding SSD1306 OLED to Your Raspberry Pi Projects

August 19, 2026 admin

In today’s lesson in the AI on the Edge series, we’re adding a very important piece to our growing collection of skills — learning how to use the SSD1306 OLED display with the SunFounder Fusion AI Lab kit.

One of the key goals of this class is to build intelligent systems that can run completely on the edge — without needing to rely on the cloud. A big part of that is giving our edge devices the ability to communicate clearly with us. That’s exactly where the little SSD1306 OLED really shines. Whether you’re building a smart robot, an autonomous sensor node, or an AI-powered monitoring system, having a crisp, low-power display to show status, results, or even fun animations makes your project feel complete and professional.

In this video, we start with the basics and gradually build up. You’ll learn how to connect the OLED using I2C, initialize it with CircuitPython, and then use the PIL library to create images in memory before sending them to the screen. We explore loading different font sizes so you can create nice, readable layouts. Then we move into drawing shapes — rectangles, circles, and borders — before finishing with a fun bouncing ball animation that brings the whole thing to life.

This lesson is particularly important in our AI on the Edge journey because the SSD1306 is extremely lightweight and uses almost no processing power, making it perfect for running alongside face detection, object tracking, speech recognition, and other AI tasks without slowing down your Raspberry Pi. You’ll see how easy it is to display messages like detection results, confidence scores, system status, or even playful personality messages from your AI creations.

By the end of this lesson, you’ll have the confidence to add a real display to any project in this class. Whether you want to show live face tracking data, sensor readings, or just give your robot a fun way to “talk” to the world, the skills you learn here will be used again and again in future projects.

So if you’re following along with the AI on the Edge series, this is another big step forward. Grab your Fusion AI Hat, open up Thonny, and let’s get that OLED screen lighting up with some personality!

As always, I strongly encourage you to take the code and make it your own. Change the messages, create new animations, and think about how you can use this display in your own AI projects. That’s where the real learning and creativity happens

So far in this class we have been using this schematic:

In this lesson, we will be adding the SSD1306 OLED display. Keep the schatic above, but now add the OLED display to the breadboard. It should be connected to the Fusion Hat as follows:

Connect the VCC pin of OLED display to 3.3V on Fusion HAT+
Connect the GND pin of OLED display to GND on Fusion HAT+
Connect the SCL pin of OLED display to SCL (GPIO 3) on Fusion HAT+
Connect the SDA pin of OLED display to SDA (GPIO 2) on Fusion HAT+

While we are updating our project components, go ahead and connect the neoPixel ring to yout Fusion Hat. Here is the schematic to add the neopixel ring, and we will be using it in future lessons.

In the video, we developed the following code to show you how to put the SSD1306 OLED through its paces using the Fusion AI Hat on the Raspberry Pi 5.

# ====================================================================
# DISCLAIMER:
# This code is provided as-is for educational and experimental 
# purposes only. The author makes no representations or warranties of 
# any kind concerning the safety, suitability, or accuracy of this 
# code. Use at your own risk. The author assumes no liability for any 
# damages, system failures, security breaches, or network issues 
# resulting from the use or implementation of this script.
# ====================================================================
 
import board
import adafruit_ssd1306
from PIL import Image, ImageDraw, ImageFont
import time

W =128
H = 64
I2C_ADDRESS = 0x3C

i2c = board.I2C()
oled = adafruit_ssd1306.SSD1306_I2C(W, H, i2c, addr=I2C_ADDRESS)

fontLarge = ImageFont.truetype("/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf",14)
fontMedium = ImageFont.truetype("/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf",10)
fontSmall = ImageFont.truetype("/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf",8)

image = Image.new("1", (W,H))
draw = ImageDraw.Draw(image)
draw.point((int(W/2),int(H/2)), fill =255)
oled.image(image)
oled.show()
ts=1
time.sleep(ts)

image = Image.new("1", (W,H))
draw = ImageDraw.Draw(image)
draw.text( (0,0), "TopTechBoy.com",font=fontLarge, fill=255)
draw.text( (0,16), "SSD1306 OLED",font=fontMedium, fill=255)
draw.text( (0,26), "Fusion AI Hat",font=fontMedium, fill=255)
draw.text( (0,36), "Hello Makers",font=fontMedium, fill=255)
draw.text( (0,46), "Enjoy!",font=fontSmall, fill=255)
oled.image(image)
oled.show()
time.sleep(ts)

image = Image.new("1", (W,H))
draw = ImageDraw.Draw(image)
draw.rectangle((int(.1*W),int(.1*H),int(.9*W),int(.9*H)),
               width=5, outline=255, fill=0)
draw.text((int(W/4),int(H/2.5)),
          "Hello World",font=fontMedium, fill=255)

oled.image(image)
oled.show()
time.sleep(ts)

image = Image.new("1", (W,H))
draw = ImageDraw.Draw(image)
draw.rectangle((int(.1*W),int(.1*H),int(.45*W),int(.45*H)),width=1, outline=255, fill=0)
draw.rectangle((int(.55*W),int(.1*H),int(.9*W),int(.45*H)),width=1, outline=255, fill=255)
d=25
draw.ellipse((int(.1*W),int(.55*H),int(.1*W)+d,int(.55*H)+d),width=1, outline=255, fill=0)
draw.ellipse((int(.55*W),int(.55*H),int(.55*W)+d,int(.55*H)+d),width=1, outline=255, fill=255)
oled.image(image)
oled.show()
time.sleep(ts)

xCenter=40
yCenter=25
radius=7

dx = 1
dy = 1

for frame in range(500):
    image = Image.new("1", (W,H))
    draw = ImageDraw.Draw(image)
    draw.rectangle((0,0,W-1,H-1), width=1, outline = 255, fill=0) 
    draw.ellipse((xCenter-radius,yCenter-radius,xCenter+radius,yCenter+radius), width=1,outline=255,fill=255)
    #draw.rectangle((0,0,W-1,H-1), width=1, outline = 255, fill=0)    
    oled.image(image)
    oled.show()
    
    xCenter = xCenter + dx
    yCenter = yCenter + dy

    if xCenter <= radius or xCenter >= W-1-radius:
        dx = -dx
    if yCenter <= radius or yCenter >= H-1-radius:
        dy = -dy

# ====================================================================

# DISCLAIMER:

# This code is provided as-is for educational and experimental

# purposes only. The author makes no representations or warranties of

# any kind concerning the safety, suitability, or accuracy of this

# code. Use at your own risk. The author assumes no liability for any

# damages, system failures, security breaches, or network issues

# resulting from the use or implementation of this script.

# ====================================================================

import board

import adafruit_ssd1306

from PIL import Image, ImageDraw, ImageFont

import time

W =128

H = 64

I2C_ADDRESS = 0x3C

i2c = board.I2C()

oled = adafruit_ssd1306.SSD1306_I2C(W, H, i2c, addr=I2C_ADDRESS)

fontLarge = ImageFont.truetype("/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf",14)

fontMedium = ImageFont.truetype("/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf",10)

fontSmall = ImageFont.truetype("/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf",8)

image = Image.new("1", (W,H))

draw = ImageDraw.Draw(image)

draw.point((int(W/2),int(H/2)), fill =255)

oled.image(image)

oled.show()

ts=1

time.sleep(ts)

image = Image.new("1", (W,H))

draw = ImageDraw.Draw(image)

draw.text( (0,0), "TopTechBoy.com",font=fontLarge, fill=255)

draw.text( (0,16), "SSD1306 OLED",font=fontMedium, fill=255)

draw.text( (0,26), "Fusion AI Hat",font=fontMedium, fill=255)

draw.text( (0,36), "Hello Makers",font=fontMedium, fill=255)

draw.text( (0,46), "Enjoy!",font=fontSmall, fill=255)

oled.image(image)

oled.show()

time.sleep(ts)

image = Image.new("1", (W,H))

draw = ImageDraw.Draw(image)

draw.rectangle((int(.1*W),int(.1*H),int(.9*W),int(.9*H)),

width=5, outline=255, fill=0)

draw.text((int(W/4),int(H/2.5)),

"Hello World",font=fontMedium, fill=255)

oled.image(image)

oled.show()

time.sleep(ts)

image = Image.new("1", (W,H))

draw = ImageDraw.Draw(image)

draw.rectangle((int(.1*W),int(.1*H),int(.45*W),int(.45*H)),width=1, outline=255, fill=0)

draw.rectangle((int(.55*W),int(.1*H),int(.9*W),int(.45*H)),width=1, outline=255, fill=255)

d=25

draw.ellipse((int(.1*W),int(.55*H),int(.1*W)+d,int(.55*H)+d),width=1, outline=255, fill=0)

draw.ellipse((int(.55*W),int(.55*H),int(.55*W)+d,int(.55*H)+d),width=1, outline=255, fill=255)

oled.image(image)

oled.show()

time.sleep(ts)

xCenter=40

yCenter=25

radius=7

dx = 1

dy = 1

for frame in range(500):

image = Image.new("1", (W,H))

draw = ImageDraw.Draw(image)

draw.rectangle((0,0,W-1,H-1), width=1, outline = 255, fill=0)

draw.ellipse((xCenter-radius,yCenter-radius,xCenter+radius,yCenter+radius), width=1,outline=255,fill=255)

#draw.rectangle((0,0,W-1,H-1), width=1, outline = 255, fill=0)

oled.image(image)

oled.show()

xCenter = xCenter + dx

yCenter = yCenter + dy

if xCenter <= radius or xCenter >= W-1-radius:

dx = -dx

if yCenter <= radius or yCenter >= H-1-radius:

dy = -dy

AI On the Edge, Raspberry Pi

AI on the Edge LESSON 42: Create Composite Images Using Masks in OpenCV and MediaPipe

August 16, 2026 admin

In this exciting project, we combine a Raspberry Pi 5, the Fusion AI Lab Kit, a Pi Camera, and a remote IP camera to generate a stunning real-time composite video. Watch as a glowing, translucent MediaPipe face mesh of my face hovers magically over live video of the Mighty River Nice scenery captured by an IP camera. The effect looks futuristic and professional — perfect for creative video effects, interactive installations, or just blowing your mind with computer vision! Using Picamera2 for high-frame-rate local capture and OpenCV with an RTSP stream from the river camera, we process everything in real time. MediaPipe’s Face Mesh detects and tracks facial landmarks, which we draw as beautiful cyan/teal contours with glowing irises. Then we create a clean mask, separate the mesh foreground from the river background, and blend them seamlessly into one composite frame. You’ll see every debugging layer live on screen too — meshLayer, mask, inverted mask, riverBG, and meshFG — so you can understand exactly how the magic happens.This tutorial is beginner-to-intermediate friendly and packed with practical OpenCV + MediaPipe techniques you can adapt for your own augmented reality projects. Whether you’re a longtime follower of the Paul McWhorter channel or new to the Fusion AI Kit, you’ll walk away inspired and ready to build your own hovering effects, overlays, or interactive displays.Grab the full code from the video description, fire up your Pi 5, and start creating jaw-dropping computer vision projects today. Drop a comment and let me know what you’d like to overlay next — another face mesh, hand tracking, or something completely different? Let’s keep pushing the limits of what we can do with affordable AI hardware!

# ====================================================================
# DISCLAIMER:
# This code is provided as-is for educational and experimental 
# purposes only. The author makes no representations or warranties of 
# any kind concerning the safety, suitability, or accuracy of this 
# code. Use at your own risk. The author assumes no liability for any 
# damages, system failures, security breaches, or network issues 
# resulting from the use or implementation of this script.
# ====================================================================
 
import cv2
import time
from picamera2 import Picamera2
import mediapipe as mp
import numpy as np
import secret
W=1280
H=720
tStart = time.time()
fps = 0

piCam = Picamera2(1)
piCam.preview_configuration.main.size = (W, H)
piCam.preview_configuration.main.format = "RGB888"
piCam.preview_configuration.controls.FrameRate = 60
piCam.preview_configuration.align()
piCam.configure("preview")
piCam.start()

cam = cv2.VideoCapture(secret.RTSP1,cv2.CAP_FFMPEG)
cam.set(cv2.CAP_PROP_BUFFERSIZE,0)
cam.set(cv2.CAP_PROP_POS_FRAMES,0)

textLowerLeft = (int(W*.01),int(H*.05))
fontFace = cv2.FONT_HERSHEY_SIMPLEX
fontThickness = int(W/425)
fontScale = H*.0015
fontColor = (0,0,255)

faceMesh = mp.solutions.face_mesh.FaceMesh(
    
    max_num_faces=1,
    refine_landmarks=True,
    min_detection_confidence=0.5,
    min_tracking_confidence=0.5)

cv2.namedWindow('Composite',cv2.WINDOW_GUI_NORMAL)
cv2.resizeWindow('Composite',W,H)
cv2.moveWindow('Composite',0,65)

cv2.namedWindow('meshLayer',cv2.WINDOW_GUI_NORMAL)
cv2.resizeWindow('meshLayer',320,180)
cv2.moveWindow('meshLayer',0,H+65)

cv2.namedWindow('grayLayer',cv2.WINDOW_GUI_NORMAL)
cv2.resizeWindow('grayLayer',320,180)
cv2.moveWindow('grayLayer',320,H+65)

cv2.namedWindow('maskLayer',cv2.WINDOW_GUI_NORMAL)
cv2.resizeWindow('maskLayer',320,180)
cv2.moveWindow('maskLayer',320*2,H+65)

cv2.namedWindow('invLayer',cv2.WINDOW_GUI_NORMAL)
cv2.resizeWindow('invLayer',320,180)
cv2.moveWindow('invLayer',320*3,H+65)

cv2.namedWindow('riverBG',cv2.WINDOW_GUI_NORMAL)
cv2.resizeWindow('riverBG',320,180)
cv2.moveWindow('riverBG',W,65)

cv2.namedWindow('meshFG',cv2.WINDOW_GUI_NORMAL)
cv2.resizeWindow('meshFG',320,180)
cv2.moveWindow('meshFG',W,180+65+20)
while True:
    deltaT = time.time() - tStart
    tStart=time.time()
    fps = fps*.95 + (1/deltaT)*.05
    frame = piCam.capture_array()
    frame = cv2.flip(frame, -1)
    
    _, river = cam.read()
    river = cv2.resize(river,(W,H))
    meshLayer = frame.copy()
    meshLayer[0:H-1,0:W-1] = [255,255,255]
    rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
    results = faceMesh.process(rgb)

    if results.multi_face_landmarks:
        for faceLandmarks in results.multi_face_landmarks:
            # Optional second pass for glow effect
            mp.solutions.drawing_utils.draw_landmarks(
                image=meshLayer,
                landmark_list=faceLandmarks,
                connections=mp.solutions.face_mesh.FACEMESH_TESSELATION,
                landmark_drawing_spec=None,
                connection_drawing_spec=mp.solutions.drawing_utils.DrawingSpec(
                    color=(0, 150, 200),
                    thickness=2
                )
            )
            # === Best looking translucent mesh ===
            mp.solutions.drawing_utils.draw_landmarks(
                image=meshLayer,
                landmark_list=faceLandmarks,
                connections=mp.solutions.face_mesh.FACEMESH_CONTOURS,
                landmark_drawing_spec=None,                    # Hide dots
                connection_drawing_spec=mp.solutions.drawing_utils.DrawingSpec(
                    color=(0, 0, 180),      # Bright cyan/teal
                    thickness=7
                )
            )

            # Optional: Extra detail on irises (makes eyes look better)
            mp.solutions.drawing_utils.draw_landmarks(
                image=meshLayer,
                landmark_list=faceLandmarks,
                connections=mp.solutions.face_mesh.FACEMESH_IRISES,
                landmark_drawing_spec=None,
                connection_drawing_spec=mp.solutions.drawing_utils.DrawingSpec(
                    color=(255, 0, 0),
                    thickness=7
                )
            )
    gray = cv2.cvtColor(meshLayer,cv2.COLOR_BGR2GRAY)

    _ , mask =cv2.threshold(gray, 250, 255, cv2.THRESH_BINARY)
    invLayer = cv2.bitwise_not(mask)
    riverBG =cv2.bitwise_and(river, river, mask = mask)
    meshFG = cv2.bitwise_and(meshLayer,meshLayer, mask=invLayer)
    composite = cv2.add(riverBG,meshFG)
    
    myText = "FPS: "+str(round(fps,1))
    cv2.putText(composite,myText,textLowerLeft,fontFace,fontScale,fontColor,fontThickness)

    cv2.imshow("Composite", composite)
    cv2.imshow("meshLayer",meshLayer)
    cv2.imshow("grayLayer",gray)
    cv2.imshow("maskLayer",mask)
    cv2.imshow("invLayer",invLayer)
    cv2.imshow("riverBG",riverBG)
    cv2.imshow("meshFG",meshFG)
    if cv2.waitKey(1) == ord('q'):
        break

cv2.destroyAllWindows()
piCam.stop()

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

# ====================================================================

# DISCLAIMER:

# This code is provided as-is for educational and experimental

# purposes only. The author makes no representations or warranties of

# any kind concerning the safety, suitability, or accuracy of this

# code. Use at your own risk. The author assumes no liability for any

# damages, system failures, security breaches, or network issues

# resulting from the use or implementation of this script.

# ====================================================================

import cv2

import time

from picamera2 import Picamera2

import mediapipe as mp

import numpy as np

import secret

W=1280

H=720

tStart = time.time()

fps = 0

piCam = Picamera2(1)

piCam.preview_configuration.main.size = (W, H)

piCam.preview_configuration.main.format = "RGB888"

piCam.preview_configuration.controls.FrameRate = 60

piCam.preview_configuration.align()

piCam.configure("preview")

piCam.start()

cam = cv2.VideoCapture(secret.RTSP1,cv2.CAP_FFMPEG)

cam.set(cv2.CAP_PROP_BUFFERSIZE,0)

cam.set(cv2.CAP_PROP_POS_FRAMES,0)

textLowerLeft = (int(W*.01),int(H*.05))

fontFace = cv2.FONT_HERSHEY_SIMPLEX

fontThickness = int(W/425)

fontScale = H*.0015

fontColor = (0,0,255)

faceMesh = mp.solutions.face_mesh.FaceMesh(

max_num_faces=1,

refine_landmarks=True,

min_detection_confidence=0.5,

min_tracking_confidence=0.5)

cv2.namedWindow('Composite',cv2.WINDOW_GUI_NORMAL)

cv2.resizeWindow('Composite',W,H)

cv2.moveWindow('Composite',0,65)

cv2.namedWindow('meshLayer',cv2.WINDOW_GUI_NORMAL)

cv2.resizeWindow('meshLayer',320,180)

cv2.moveWindow('meshLayer',0,H+65)

cv2.namedWindow('grayLayer',cv2.WINDOW_GUI_NORMAL)

cv2.resizeWindow('grayLayer',320,180)

cv2.moveWindow('grayLayer',320,H+65)

cv2.namedWindow('maskLayer',cv2.WINDOW_GUI_NORMAL)

cv2.resizeWindow('maskLayer',320,180)

cv2.moveWindow('maskLayer',320*2,H+65)

cv2.namedWindow('invLayer',cv2.WINDOW_GUI_NORMAL)

cv2.resizeWindow('invLayer',320,180)

cv2.moveWindow('invLayer',320*3,H+65)

cv2.namedWindow('riverBG',cv2.WINDOW_GUI_NORMAL)

cv2.resizeWindow('riverBG',320,180)

cv2.moveWindow('riverBG',W,65)

cv2.namedWindow('meshFG',cv2.WINDOW_GUI_NORMAL)

cv2.resizeWindow('meshFG',320,180)

cv2.moveWindow('meshFG',W,180+65+20)

while True:

deltaT = time.time() - tStart

tStart=time.time()

fps = fps*.95 + (1/deltaT)*.05

frame = piCam.capture_array()

frame = cv2.flip(frame, -1)

_, river = cam.read()

river = cv2.resize(river,(W,H))

meshLayer = frame.copy()

meshLayer[0:H-1,0:W-1] = [255,255,255]

rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)

results = faceMesh.process(rgb)

if results.multi_face_landmarks:

for faceLandmarks in results.multi_face_landmarks:

# Optional second pass for glow effect

mp.solutions.drawing_utils.draw_landmarks(

image=meshLayer,

landmark_list=faceLandmarks,

connections=mp.solutions.face_mesh.FACEMESH_TESSELATION,

landmark_drawing_spec=None,

connection_drawing_spec=mp.solutions.drawing_utils.DrawingSpec(

color=(0, 150, 200),

thickness=2

)

# === Best looking translucent mesh ===

mp.solutions.drawing_utils.draw_landmarks(

image=meshLayer,

landmark_list=faceLandmarks,

connections=mp.solutions.face_mesh.FACEMESH_CONTOURS,

landmark_drawing_spec=None, # Hide dots

connection_drawing_spec=mp.solutions.drawing_utils.DrawingSpec(

color=(0, 0, 180), # Bright cyan/teal

thickness=7

)

# Optional: Extra detail on irises (makes eyes look better)

mp.solutions.drawing_utils.draw_landmarks(

image=meshLayer,

landmark_list=faceLandmarks,

connections=mp.solutions.face_mesh.FACEMESH_IRISES,

landmark_drawing_spec=None,

connection_drawing_spec=mp.solutions.drawing_utils.DrawingSpec(

color=(255, 0, 0),

thickness=7

)

gray = cv2.cvtColor(meshLayer,cv2.COLOR_BGR2GRAY)

_ , mask =cv2.threshold(gray, 250, 255, cv2.THRESH_BINARY)

invLayer = cv2.bitwise_not(mask)

riverBG =cv2.bitwise_and(river, river, mask = mask)

meshFG = cv2.bitwise_and(meshLayer,meshLayer, mask=invLayer)

composite = cv2.add(riverBG,meshFG)

myText = "FPS: "+str(round(fps,1))

cv2.putText(composite,myText,textLowerLeft,fontFace,fontScale,fontColor,fontThickness)

cv2.imshow("Composite", composite)

cv2.imshow("meshLayer",meshLayer)

cv2.imshow("grayLayer",gray)

cv2.imshow("maskLayer",mask)

cv2.imshow("invLayer",invLayer)

cv2.imshow("riverBG",riverBG)

cv2.imshow("meshFG",meshFG)

if cv2.waitKey(1) == ord('q'):

break

cv2.destroyAllWindows()

piCam.stop()

AI On the Edge, Raspberry Pi

AI on the Edge LESSON 41: Creating FaceMesh Using MediaPipe in OpenCV

August 13, 2026 admin

In this project, I demonstrate how to create a smooth, real-time face mesh overlay using the Raspberry Pi 5, the official Pi Camera, MediaPipe, and OpenCV. The program captures live video from the camera and draws a detailed, colorful mesh that follows every movement of the face with high accuracy. The result is a visually appealing augmented reality-style effect that runs efficiently even on a single-board computer.

The goal of this project is to build a responsive face tracking system that detects and draws 468 facial landmarks in real time. This creates a striking mesh that highlights the contours of the face, eyes, lips, and jawline, making it an excellent foundation for more advanced computer vision projects like virtual filters, AR effects, or interactive installations.

The program follows a straightforward but efficient real-time vision pipeline. First, it initializes the Raspberry Pi Camera using the modern picamera2 library, configured for 1280×720 resolution at 60 frames per second. It then sets up MediaPipe’s Face Mesh solution with landmark refinement enabled for better eye tracking.

In the main loop, the program continuously grabs a frame from the camera, corrects its orientation, and converts it from BGR to RGB format since MediaPipe expects RGB input. The frame is then passed to the Face Mesh model for processing. When a face is detected, the program draws multiple layers of graphics on top of the image: a fine tesselation mesh across the entire face, thick and vibrant contours around the major facial features, and special highlighting on the irises. Finally, the processed frame is displayed in an OpenCV window, creating a smooth and engaging real-time visualization.

This approach works particularly well on the Raspberry Pi 5 because it balances visual quality with performance. By limiting detection to a single face and using efficient drawing methods, the application maintains high frame rates while producing a professional-looking result. The multi-layer drawing technique (tesselation + contours + irises) gives the mesh depth and visual appeal that single-pass drawings often lack.

The project makes use of several powerful technologies: picamera2 for fast camera access, Google’s MediaPipe for high-speed machine learning-based landmark detection, OpenCV for image handling and display, and NumPy for efficient array operations.

This face mesh project serves as an excellent stepping stone into real-time AI and computer vision on embedded hardware. Once you have the basic mesh working, it becomes much easier to expand into creative applications such as face filters, gesture recognition, or overlaying the mesh onto other video sources.

The code developed in the video lesson is presented below:

# ====================================================================
# DISCLAIMER:
# This code is provided as-is for educational and experimental 
# purposes only. The author makes no representations or warranties of 
# any kind concerning the safety, suitability, or accuracy of this 
# code. Use at your own risk. The author assumes no liability for any 
# damages, system failures, security breaches, or network issues 
# resulting from the use or implementation of this script.
# ====================================================================
 
import cv2
import time
from picamera2 import Picamera2
import mediapipe as mp
import numpy as np
W=1280
H=720
tStart = time.time()
fps = 0

piCam = Picamera2(1)
piCam.preview_configuration.main.size = (W, H)
piCam.preview_configuration.main.format = "RGB888"
piCam.preview_configuration.controls.FrameRate = 60
piCam.preview_configuration.align()
piCam.configure("preview")
piCam.start()

textLowerLeft = (int(W*.01),int(H*.05))
fontFace = cv2.FONT_HERSHEY_SIMPLEX
fontThickness = int(W/425)
fontScale = H*.0015
fontColor = (0,0,255)

faceMesh = mp.solutions.face_mesh.FaceMesh(
    
    max_num_faces=1,
    refine_landmarks=True,
    min_detection_confidence=0.5,
    min_tracking_confidence=0.5)

cv2.namedWindow('Camera',cv2.WINDOW_GUI_NORMAL)
cv2.moveWindow('Camera',0,65)
cv2.resizeWindow('Camera',W,H)
while True:
    deltaT = time.time() - tStart
    tStart=time.time()
    fps = fps*.95 + (1/deltaT)*.05
    frame = piCam.capture_array()
    frame = cv2.flip(frame, -1)

    rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
    results = faceMesh.process(rgb)

    if results.multi_face_landmarks:
        for faceLandmarks in results.multi_face_landmarks:
            # Optional second pass for glow effect
            mp.solutions.drawing_utils.draw_landmarks(
                image=frame,
                landmark_list=faceLandmarks,
                connections=mp.solutions.face_mesh.FACEMESH_TESSELATION,
                landmark_drawing_spec=None,
                connection_drawing_spec=mp.solutions.drawing_utils.DrawingSpec(
                    color=(100, 100, 100),
                    thickness=1
                )
            )
            # === Best looking translucent mesh ===
            mp.solutions.drawing_utils.draw_landmarks(
                image=frame,
                landmark_list=faceLandmarks,
                connections=mp.solutions.face_mesh.FACEMESH_CONTOURS,
                landmark_drawing_spec=None,                    # Hide dots
                connection_drawing_spec=mp.solutions.drawing_utils.DrawingSpec(
                    color=(0, 0, 180),      # Bright cyan/teal
                    thickness=7
                )
            )

            # Optional: Extra detail on irises (makes eyes look better)
            mp.solutions.drawing_utils.draw_landmarks(
                image=frame,
                landmark_list=faceLandmarks,
                connections=mp.solutions.face_mesh.FACEMESH_IRISES,
                landmark_drawing_spec=None,
                connection_drawing_spec=mp.solutions.drawing_utils.DrawingSpec(
                    color=(255, 0, 0),
                    thickness=7
                )
            )

    myText = "FPS: "+str(round(fps,1))
    cv2.putText(frame,myText,textLowerLeft,fontFace,fontScale,fontColor,fontThickness)

    cv2.imshow("Camera", frame)

    if cv2.waitKey(1) == ord('q'):
        break

cv2.destroyAllWindows()
piCam.stop()

100

101

# ====================================================================

# DISCLAIMER:

# This code is provided as-is for educational and experimental

# purposes only. The author makes no representations or warranties of

# any kind concerning the safety, suitability, or accuracy of this

# code. Use at your own risk. The author assumes no liability for any

# damages, system failures, security breaches, or network issues

# resulting from the use or implementation of this script.

# ====================================================================

import cv2

import time

from picamera2 import Picamera2

import mediapipe as mp

import numpy as np

W=1280

H=720

tStart = time.time()

fps = 0

piCam = Picamera2(1)

piCam.preview_configuration.main.size = (W, H)

piCam.preview_configuration.main.format = "RGB888"

piCam.preview_configuration.controls.FrameRate = 60

piCam.preview_configuration.align()

piCam.configure("preview")

piCam.start()

textLowerLeft = (int(W*.01),int(H*.05))

fontFace = cv2.FONT_HERSHEY_SIMPLEX

fontThickness = int(W/425)

fontScale = H*.0015

fontColor = (0,0,255)

faceMesh = mp.solutions.face_mesh.FaceMesh(

max_num_faces=1,

refine_landmarks=True,

min_detection_confidence=0.5,

min_tracking_confidence=0.5)

cv2.namedWindow('Camera',cv2.WINDOW_GUI_NORMAL)

cv2.moveWindow('Camera',0,65)

cv2.resizeWindow('Camera',W,H)

while True:

deltaT = time.time() - tStart

tStart=time.time()

fps = fps*.95 + (1/deltaT)*.05

frame = piCam.capture_array()

frame = cv2.flip(frame, -1)

rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)

results = faceMesh.process(rgb)

if results.multi_face_landmarks:

for faceLandmarks in results.multi_face_landmarks:

# Optional second pass for glow effect

mp.solutions.drawing_utils.draw_landmarks(

image=frame,

landmark_list=faceLandmarks,

connections=mp.solutions.face_mesh.FACEMESH_TESSELATION,

landmark_drawing_spec=None,

connection_drawing_spec=mp.solutions.drawing_utils.DrawingSpec(

color=(100, 100, 100),

thickness=1

)

# === Best looking translucent mesh ===

mp.solutions.drawing_utils.draw_landmarks(

image=frame,

landmark_list=faceLandmarks,

connections=mp.solutions.face_mesh.FACEMESH_CONTOURS,

landmark_drawing_spec=None, # Hide dots

connection_drawing_spec=mp.solutions.drawing_utils.DrawingSpec(

color=(0, 0, 180), # Bright cyan/teal

thickness=7

)

# Optional: Extra detail on irises (makes eyes look better)

mp.solutions.drawing_utils.draw_landmarks(

image=frame,

landmark_list=faceLandmarks,

connections=mp.solutions.face_mesh.FACEMESH_IRISES,

landmark_drawing_spec=None,

connection_drawing_spec=mp.solutions.drawing_utils.DrawingSpec(

color=(255, 0, 0),

thickness=7

)

myText = "FPS: "+str(round(fps,1))

cv2.putText(frame,myText,textLowerLeft,fontFace,fontScale,fontColor,fontThickness)

cv2.imshow("Camera", frame)

if cv2.waitKey(1) == ord('q'):

break

cv2.destroyAllWindows()

piCam.stop()

AI On the Edge, Raspberry Pi

AI on the Edge LESSON 40: Active Face Tracker with Pan Tilt Camera and MediaPipe on Pi 5

August 8, 2026 admin

Boys and girls, welcome back! In today’s lesson, we are going to tie together everything we’ve been building in the AI on the Edge series and construct something truly interactive: a fully autonomous, voice-controlled, pan-tilt face tracking robot running locally right on your Raspberry Pi 5!

In our previous lessons, we learned how to detect faces using MediaPipe and how to drive physical servos to point a camera. Today, we step up our game. We are bringing in multithreading, Speech-to-Text (STT) using the Fusion Hat, and Text-to-Speech (TTS) with Piper to give our Pi a voice, a personality, and the physical ability to track down humanoids in real time.

What We Are Building in This Lesson

Imagine setting up a camera system that constantly scans its environment. The moment a human face enters the frame, the system locks on and speaks up: “Humanoid Detected, Shall I track?”

Using real-time voice commands, you can issue directions straight to the Pi without touching a keyboard:

“Track” — Activates proportional control on the pan-tilt kit. The servos will calculate pixel error relative to the center of the frame and smoothly adjust their angles to keep your face dead center.
“Release” — Disables active tracking, letting the servos hold their position while the vision loop continues monitoring.
“Blind” — Isolates the facial keypoints for the subject’s eyes and draws solid black circles over them in real time, causing the robot to announce: “Subject Has Been Blinded, Shall I Vaporize?”
“Restore” — Removes the eye overlay and brings vision back to normal.
“Quit” — Safely terminates all background threads, announces shutdown, and closes down the application gracefully.

Key Technical Concepts Covered

1. Multi-Threaded Architecture & Thread-Safe Queues

Audio processing—both listening for voice input and generating spoken speech—is computationally heavy and blocking by nature. If you run speech recognition directly inside your primary video processing loop, your frame rate will plummet from a smooth 60 FPS down to a complete crawl.

To solve this, we spin up two independent background threads using Python’s threading module:

Speech Thread: Monitors a thread-safe speakQ (Queue) and handles text-to-speech output using Piper without stalling the main loop.
Command Thread: Continuously listens to the microphone via Speech-to-Text, strips and parses incoming voice triggers, and pushes valid commands into a commandQ.

2. MediaPipe Facial Landmark Detection

We leverage MediaPipe’s high-speed face detection solution running at 1280×720 resolution on the Raspberry Pi 5. By calculating relative bounding boxes and keypoint coordinate matrices (x, y), the system identifies both face centroids and precise feature locations like eye coordinates.

3. Proportional Servo Error Correction

To keep the camera centered on a moving subject, the script computes positional error delta values between the center of the bounding box and the exact midpoint of the camera frame:

xError = xBoxCenter – xFrameCenter

yError = yBoxCenter – yFrameCenter

These error values are scaled down and applied directly to update the current pan and tilt servo angles, ensuring smooth, continuous tracking movement without jarring overshoots.

Your Homework Assignment

Get your Raspberry Pi 5, mount your pan-tilt camera assembly with the Fusion Hat, and implement the multithreaded architecture outlined in this lesson. Tune your servo scaling factors to ensure your tracking motion is fluid and responsive at 60 FPS. Have fun!

# ====================================================================
# DISCLAIMER:
# This code is provided as-is for educational and experimental 
# purposes only. The author makes no representations or warranties of 
# any kind concerning the safety, suitability, or accuracy of this 
# code. Use at your own risk. The author assumes no liability for any 
# damages, system failures, security breaches, or network issues 
# resulting from the use or implementation of this script.
# ====================================================================
 
import cv2
import time
from picamera2 import Picamera2
import mediapipe as mp
#########################################
import threading
from queue import Queue
from fusion_hat.tts import Piper
tts = Piper()
tts.set_model('en_US-kristin-medium')
from fusion_hat.stt import STT
stt = STT('en-us')
 
cnt = 0
running = True
commandQ = Queue()
speakQ = Queue()
###########################################
from fusion_hat.servo import Servo
panPin = 2
tiltPin = 3
panServo = Servo(panPin)
tiltServo = Servo(tiltPin)
panAngle = -10
tiltAngle =-20
panServo.angle(panAngle)
tiltServo.angle(tiltAngle)
piCam = Picamera2()
W=1280
H=720
tStart = time.time()
fps = 0
RES = (W,H)
piCam.preview_configuration.main.size = RES
piCam.preview_configuration.main.format = "RGB888"
piCam.preview_configuration.controls.FrameRate=60
piCam.preview_configuration.align()
piCam.configure("preview")
piCam.start()
textLowerLeft = (int(W*.01),int(H*.05))
fontFace = cv2.FONT_HERSHEY_SIMPLEX
fontThickness = int(W/425)
fontScale = H*.0015
fontColor = (0,0,255)
 
faceFinder = mp.solutions.face_detection.FaceDetection(
    model_selection = 0,
    min_detection_confidence = .5)
 
track = False
xFrameC = int((W-1)/2)
yFrameC = int((H-1)/2)
xBoxC = xFrameC
yBoxC = yFrameC
##########################################
def speak():
    global running
    print('Speak Thread Running')
    while running:
        if speakQ.empty() == False:
            txt = speakQ.get()
            print(txt)
            tts.say(txt)
        time.sleep(.1)
speakThread = threading.Thread(target=speak,daemon=True)
speakThread.start()

def getCommand():
    print('Get Command Thread is Running')
    global running
    while running:
        myCommand = stt.listen(stream=False)
        myCommand = myCommand.strip()
        print(myCommand)
        if myCommand == 'quit':
            msg = 'Project Humanoid Elimination Terminated. Have a Nice Day.'
            speakQ.put(msg)
            running = False
            break
        commandQ.put(myCommand)
    print('Get Command Thread Terminated\n')
commandThread = threading.Thread(target=getCommand,daemon=True)
commandThread.start()
print('Main Program Has Commenced')
msg = 'Hello, I am Ready to Eliminate Humanoids'
speakQ.put(msg)
msgOld = 'Hello, I am Ready to Eliminate Humanoids'
cv2.namedWindow('Camera',cv2.WINDOW_GUI_NORMAL)
cv2.moveWindow('Camera',0,65)
cv2.resizeWindow('Camera',W,H)
################################
msg2=''
msg2Old=''
blind = False
while running:
    deltaT = time.time() - tStart
    tStart=time.time()
    fps = fps*.95 + (1/deltaT)*.05
    frame= piCam.capture_array()
    frame=cv2.flip(frame,-1)
    
    if commandQ.empty() == False:
        myCommand = commandQ.get()
        if myCommand == 'track':
            track = True
        if myCommand == 'release':
            track = False
        if myCommand == 'blind':
            blind = True
            msg2="Subject Has Been Blinded, Shall I Vaporize?"
        if myCommand == 'restore':
            blind = False
            msg2 = "Subject Sight Has Been Restored"
        if msg2!= msg2Old:
            speakQ.put(msg2)
        msg2Old = msg2
    rgbFrame = cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)
    results = faceFinder.process(rgbFrame)
    if results.detections:
        msg = "Humanoid Detected, Shall I track"
        for detection in results.detections:
            bbox = detection.location_data.relative_bounding_box
            x = int(bbox.xmin*W)
            y = int(bbox.ymin*H)
            w = int(bbox.width*W)
            h = int(bbox.height*H)
            xBoxC = x + int(w/2)
            yBoxC = y + int(h/2)
            xError = xBoxC - xFrameC
            yError = yBoxC - yFrameC
            cv2.rectangle(frame,
                          (x,y),
                          (x+w,y+h),
                          (0,255,255),
                          3)
            keypoints = detection.location_data.relative_keypoints
            i=0
            for kp in keypoints:
                kpX = int(kp.x*W)
                kpY = int(kp.y*H)
                #cv2.putText(frame,str(i),(kpX,kpY),fontFace,fontScale,fontColor,fontThickness)
                
                if blind == True:
                    if i==0 or i== 1:
                        cv2.circle(frame, (kpX,kpY),
                                   20,
                                   (0,0,0),
                                   -1)
                i=i+1       
    if not results.detections:
        msg = "No Humanoids Detected, Continuing to Scan"
        track = False
    if msgOld != msg:
        speakQ.put(msg)
    msgOld = msg
   
    if track == True and results.detections:
        panAngle = panAngle - xError/50/2
        panServo.angle(int(panAngle))
        #time.sleep(.02)
        tiltAngle = tiltAngle + yError/50/2
        tiltServo.angle(int(tiltAngle))
        #time.sleep(.02) 
    myText = "FPS: "+str(round(fps,1))
    cv2.putText(frame,myText,textLowerLeft,fontFace,fontScale,fontColor,fontThickness)
    cv2.imshow("Camera", frame)
    cv2.moveWindow("Camera",0,60)
    if cv2.waitKey(1)==ord('q'):
        break
cv2.destroyAllWindows()
print('')
print('Program Terminated')

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

# ====================================================================

# DISCLAIMER:

# This code is provided as-is for educational and experimental

# purposes only. The author makes no representations or warranties of

# any kind concerning the safety, suitability, or accuracy of this

# code. Use at your own risk. The author assumes no liability for any

# damages, system failures, security breaches, or network issues

# resulting from the use or implementation of this script.

# ====================================================================

import cv2

import time

from picamera2 import Picamera2

import mediapipe as mp

#########################################

import threading

from queue import Queue

from fusion_hat.tts import Piper

tts = Piper()

tts.set_model('en_US-kristin-medium')

from fusion_hat.stt import STT

stt = STT('en-us')

cnt = 0

running = True

commandQ = Queue()

speakQ = Queue()

###########################################

from fusion_hat.servo import Servo

panPin = 2

tiltPin = 3

panServo = Servo(panPin)

tiltServo = Servo(tiltPin)

panAngle = -10

tiltAngle =-20

panServo.angle(panAngle)

tiltServo.angle(tiltAngle)

piCam = Picamera2()

W=1280

H=720

tStart = time.time()

fps = 0

RES = (W,H)

piCam.preview_configuration.main.size = RES

piCam.preview_configuration.main.format = "RGB888"

piCam.preview_configuration.controls.FrameRate=60

piCam.preview_configuration.align()

piCam.configure("preview")

piCam.start()

textLowerLeft = (int(W*.01),int(H*.05))

fontFace = cv2.FONT_HERSHEY_SIMPLEX

fontThickness = int(W/425)

fontScale = H*.0015

fontColor = (0,0,255)

faceFinder = mp.solutions.face_detection.FaceDetection(

model_selection = 0,

min_detection_confidence = .5)

track = False

xFrameC = int((W-1)/2)

yFrameC = int((H-1)/2)

xBoxC = xFrameC

yBoxC = yFrameC

##########################################

def speak():

global running

print('Speak Thread Running')

while running:

if speakQ.empty() == False:

txt = speakQ.get()

print(txt)

tts.say(txt)

time.sleep(.1)

speakThread = threading.Thread(target=speak,daemon=True)

speakThread.start()

def getCommand():

print('Get Command Thread is Running')

global running

while running:

myCommand = stt.listen(stream=False)

myCommand = myCommand.strip()

print(myCommand)

if myCommand == 'quit':

msg = 'Project Humanoid Elimination Terminated. Have a Nice Day.'

speakQ.put(msg)

running = False

break

commandQ.put(myCommand)

print('Get Command Thread Terminated\n')

commandThread = threading.Thread(target=getCommand,daemon=True)

commandThread.start()

print('Main Program Has Commenced')

msg = 'Hello, I am Ready to Eliminate Humanoids'

speakQ.put(msg)

msgOld = 'Hello, I am Ready to Eliminate Humanoids'

cv2.namedWindow('Camera',cv2.WINDOW_GUI_NORMAL)

cv2.moveWindow('Camera',0,65)

cv2.resizeWindow('Camera',W,H)

################################

msg2=''

msg2Old=''

blind = False

while running:

deltaT = time.time() - tStart

tStart=time.time()

fps = fps*.95 + (1/deltaT)*.05

frame= piCam.capture_array()

frame=cv2.flip(frame,-1)

if commandQ.empty() == False:

myCommand = commandQ.get()

if myCommand == 'track':

track = True

if myCommand == 'release':

track = False

if myCommand == 'blind':

blind = True

msg2="Subject Has Been Blinded, Shall I Vaporize?"

if myCommand == 'restore':

blind = False

msg2 = "Subject Sight Has Been Restored"

if msg2!= msg2Old:

speakQ.put(msg2)

msg2Old = msg2

rgbFrame = cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)

results = faceFinder.process(rgbFrame)

if results.detections:

msg = "Humanoid Detected, Shall I track"

for detection in results.detections:

bbox = detection.location_data.relative_bounding_box

x = int(bbox.xmin*W)

y = int(bbox.ymin*H)

w = int(bbox.width*W)

h = int(bbox.height*H)

xBoxC = x + int(w/2)

yBoxC = y + int(h/2)

xError = xBoxC - xFrameC

yError = yBoxC - yFrameC

cv2.rectangle(frame,

(x,y),

(x+w,y+h),

(0,255,255),

keypoints = detection.location_data.relative_keypoints

i=0

for kp in keypoints:

kpX = int(kp.x*W)

kpY = int(kp.y*H)

#cv2.putText(frame,str(i),(kpX,kpY),fontFace,fontScale,fontColor,fontThickness)

if blind == True:

if i==0 or i== 1:

cv2.circle(frame, (kpX,kpY),

20,

(0,0,0),

-1)

i=i+1

if not results.detections:

msg = "No Humanoids Detected, Continuing to Scan"

track = False

if msgOld != msg:

speakQ.put(msg)

msgOld = msg

if track == True and results.detections:

panAngle = panAngle - xError/50/2

panServo.angle(int(panAngle))

#time.sleep(.02)

tiltAngle = tiltAngle + yError/50/2

tiltServo.angle(int(tiltAngle))

#time.sleep(.02)

myText = "FPS: "+str(round(fps,1))

cv2.putText(frame,myText,textLowerLeft,fontFace,fontScale,fontColor,fontThickness)

cv2.imshow("Camera", frame)

cv2.moveWindow("Camera",0,60)

if cv2.waitKey(1)==ord('q'):

break

cv2.destroyAllWindows()

print('')

print('Program Terminated')

Technology Tutorials

AI on the Edge LESSON 44: Displaying Live MediaPipe FaceMesh Avatar on SSD1306 OLED in OpenCV

AI on the Edge LESSON 43: Adding SSD1306 OLED to Your Raspberry Pi Projects

AI on the Edge LESSON 42: Create Composite Images Using Masks in OpenCV and MediaPipe

AI on the Edge LESSON 41: Creating FaceMesh Using MediaPipe in OpenCV

AI on the Edge LESSON 40: Active Face Tracker with Pan Tilt Camera and MediaPipe on Pi 5

What We Are Building in This Lesson

Key Technical Concepts Covered

1. Multi-Threaded Architecture & Thread-Safe Queues

2. MediaPipe Facial Landmark Detection

3. Proportional Servo Error Correction

Your Homework Assignment

Making The World a Better Place One High Tech Project at a Time. Enjoy!