Tag Archives: TTS

Hello everybody! Paul McWhorter here from toptechboy.com, welcoming you back to another thrilling session of our AI on the Edge series. Today, we are taking our hardware vision capabilities to the next level. If you have ever wondered how to scale up your Raspberry Pi 5 setup beyond just a single lens, this lesson is for you. We are going to write a Python script that pulls live feeds from multiple Raspberry Pi cameras and multiple USB webcams simultaneously, displaying all four streams cleanly in real time with an on-screen frames-per-second (FPS) tracker.

Codex Knowledge: Multi-Camera Architecture on the Pi 5

When working with edge hardware like the Raspberry Pi 5, managing multiple high-bandwidth video streams requires understanding how the underlying libraries interact with the Linux kernel and system memory. In this lesson, we leverage two distinct hardware interfaces:

Picamera2 API: We initialize two separate native Pi camera instances using Picamera2(0) and Picamera2(1). By explicitly configuring the preview size to 640×360, setting the format to RGB888, locking the frame rate to 60 FPS, and calling align() before starting, we ensure the hardware pipelines are optimized for low-latency streaming without frame-drop bottlenecks.
OpenCV VideoCapture: For our USB webcams, we use OpenCV’s cv2.VideoCapture() mapped to specific device indices (in our setup, indices 16 and 18). We explicitly set the frame width, height, and target frame rate properties to keep the data flow synchronized with our Pi camera streams.
Window Layout Management: Using OpenCV highgui functions like cv2.namedWindow, cv2.moveWindow, and cv2.resizeWindow, we programmatically arrange all four camera feeds into a neat 2×2 grid on your desktop workspace, preventing windows from stacking blindly on top of each other.

General Knowledge: The Evolution of Multi-Stream Machine Vision

In industrial automation, robotics, and advanced edge AI deployments, relying on a single camera angle is rarely enough. Multi-camera systems are the gold standard for comprehensive spatial awareness, 3D depth estimation, object tracking across wide fields of view, and panoramic monitoring. Historically, running multiple high-resolution video streams required bulky, power-hungry desktop workstations equipped with expensive capture cards. Today, single-board computers like the Raspberry Pi 5—combined with optimized kernel drivers and efficient software wrappers like Picamera2—allow engineers and creators to build robust, multi-sensor vision arrays right at the edge at a fraction of the cost and power consumption.

Python Source Code

Here is the complete, production-ready script for Lesson 35. Make sure your cameras are securely connected and properly indexed before running the program.

import cv2
import time
from picamera2 import Picamera2

W = 640
H = 360
tStart = time.time()
fps = 0
RES = (W, H)

# Initialize Pi Camera 1
piCam1 = Picamera2(0)
piCam1.preview_configuration.main.size = RES
piCam1.preview_configuration.main.format = "RGB888"
piCam1.preview_configuration.controls.FrameRate = 60
piCam1.preview_configuration.align()
piCam1.configure("preview")
piCam1.start()

# Initialize Pi Camera 2
piCam2 = Picamera2(1)
piCam2.preview_configuration.main.size = RES
piCam2.preview_configuration.main.format = "RGB888"
piCam2.preview_configuration.controls.FrameRate = 60
piCam2.preview_configuration.align()
piCam2.configure("preview")
piCam2.start()

# Initialize USB Camera 1
usbCam1 = cv2.VideoCapture(16)
usbCam1.set(cv2.CAP_PROP_FRAME_WIDTH, W)
usbCam1.set(cv2.CAP_PROP_FRAME_HEIGHT, H)
usbCam1.set(cv2.CAP_PROP_FPS, 30)

# Initialize USB Camera 2
usbCam2 = cv2.VideoCapture(18)
usbCam2.set(cv2.CAP_PROP_FRAME_WIDTH, W)
usbCam2.set(cv2.CAP_PROP_FRAME_HEIGHT, H)
usbCam2.set(cv2.CAP_PROP_FPS, 30)

textLowerLeft = (int(W * 0.01), int(H * 0.05))
fontFace = cv2.FONT_HERSHEY_SIMPLEX
fontThickness = int(W / 425)
fontScale = H * 0.0015
fontColor = (0, 0, 255)

# Setup Window Grid Layout
cv2.namedWindow('Camera1', cv2.WINDOW_GUI_NORMAL)
cv2.moveWindow('Camera1', 0, 65)
cv2.resizeWindow('Camera1', W, H)

cv2.namedWindow('Camera2', cv2.WINDOW_GUI_NORMAL)
cv2.moveWindow('Camera2', W, 65)
cv2.resizeWindow('Camera2', W, H)

cv2.namedWindow('Camera3', cv2.WINDOW_GUI_NORMAL)
cv2.moveWindow('Camera3', 0, 65 + H + 20)
cv2.resizeWindow('Camera3', W, H)

cv2.namedWindow('Camera4', cv2.WINDOW_GUI_NORMAL)
cv2.moveWindow('Camera4', W, 65 + H + 20)
cv2.resizeWindow('Camera4', W, H)

while True:
    deltaT = time.time() - tStart
    tStart = time.time()
    fps = fps * 0.95 + (1 / deltaT) * 0.05
    
    frame1 = piCam1.capture_array()
    # frame1 = cv2.flip(frame1, -1)
    
    frame2 = piCam2.capture_array()
    frame2 = cv2.flip(frame2, -1)
    
    _ , frame3 = usbCam1.read()
    _ , frame4 = usbCam2.read()
    
    myText = "FPS: " + str(round(fps, 1))
    cv2.putText(frame1, myText, textLowerLeft, fontFace, fontScale, fontColor, fontThickness)
    
    cv2.imshow("Camera1", frame1)
    cv2.imshow("Camera2", frame2)
    cv2.imshow("Camera3", frame3)
    cv2.imshow("Camera4", frame4)
    
    if cv2.waitKey(1) == ord('q'):
        break

cv2.destroyAllWindows()
print('Program Terminated')

import cv2

import time

from picamera2 import Picamera2

W = 640

H = 360

tStart = time.time()

fps = 0

RES = (W, H)

# Initialize Pi Camera 1

piCam1 = Picamera2(0)

piCam1.preview_configuration.main.size = RES

piCam1.preview_configuration.main.format = "RGB888"

piCam1.preview_configuration.controls.FrameRate = 60

piCam1.preview_configuration.align()

piCam1.configure("preview")

piCam1.start()

# Initialize Pi Camera 2

piCam2 = Picamera2(1)

piCam2.preview_configuration.main.size = RES

piCam2.preview_configuration.main.format = "RGB888"

piCam2.preview_configuration.controls.FrameRate = 60

piCam2.preview_configuration.align()

piCam2.configure("preview")

piCam2.start()

# Initialize USB Camera 1

usbCam1 = cv2.VideoCapture(16)

usbCam1.set(cv2.CAP_PROP_FRAME_WIDTH, W)

usbCam1.set(cv2.CAP_PROP_FRAME_HEIGHT, H)

usbCam1.set(cv2.CAP_PROP_FPS, 30)

# Initialize USB Camera 2

usbCam2 = cv2.VideoCapture(18)

usbCam2.set(cv2.CAP_PROP_FRAME_WIDTH, W)

usbCam2.set(cv2.CAP_PROP_FRAME_HEIGHT, H)

usbCam2.set(cv2.CAP_PROP_FPS, 30)

textLowerLeft = (int(W * 0.01), int(H * 0.05))

fontFace = cv2.FONT_HERSHEY_SIMPLEX

fontThickness = int(W / 425)

fontScale = H * 0.0015

fontColor = (0, 0, 255)

# Setup Window Grid Layout

cv2.namedWindow('Camera1', cv2.WINDOW_GUI_NORMAL)

cv2.moveWindow('Camera1', 0, 65)

cv2.resizeWindow('Camera1', W, H)

cv2.namedWindow('Camera2', cv2.WINDOW_GUI_NORMAL)

cv2.moveWindow('Camera2', W, 65)

cv2.resizeWindow('Camera2', W, H)

cv2.namedWindow('Camera3', cv2.WINDOW_GUI_NORMAL)

cv2.moveWindow('Camera3', 0, 65 + H + 20)

cv2.resizeWindow('Camera3', W, H)

cv2.namedWindow('Camera4', cv2.WINDOW_GUI_NORMAL)

cv2.moveWindow('Camera4', W, 65 + H + 20)

cv2.resizeWindow('Camera4', W, H)

while True:

deltaT = time.time() - tStart

tStart = time.time()

fps = fps * 0.95 + (1 / deltaT) * 0.05

frame1 = piCam1.capture_array()

# frame1 = cv2.flip(frame1, -1)

frame2 = piCam2.capture_array()

frame2 = cv2.flip(frame2, -1)

_ , frame3 = usbCam1.read()

_ , frame4 = usbCam2.read()

myText = "FPS: " + str(round(fps, 1))

cv2.putText(frame1, myText, textLowerLeft, fontFace, fontScale, fontColor, fontThickness)

cv2.imshow("Camera1", frame1)

cv2.imshow("Camera2", frame2)

cv2.imshow("Camera3", frame3)

cv2.imshow("Camera4", frame4)

if cv2.waitKey(1) == ord('q'):

break

cv2.destroyAllWindows()

print('Program Terminated')

Conclusion

There you have it! You are now successfully driving a multi-camera computer vision array right off your Raspberry Pi 5. Play around with the window positioning, check your device indices if your USB cameras don’t immediately pop up, and get ready because in our next lesson we will start piping these multi-source frames directly into our neural network inference models. Keep tinkering, stay curious, and I will see you in the next lesson!

NVIDIA, NVIDIA Jetson Orin Nano

AI on the NVIDIA Jetson Orin Nano: Adding Text-to-Speech (TTS) with Piper

July 20, 2026 admin

In this lesson, we are building a local, offline voice pipeline using Piper. This engine runs natively on our Jetson Orin Nano hardware, providing fast and natural speech without needing an internet connection. To keep this simple, we will install everything into one specific folder so the software can easily find its own files.

Step 1: System Prep & Piper Installation

Open your terminal and run these commands one by one to create the workspace and download the required files. We are placing everything into the same directory to ensure the AI engine can always find the voice model.

# 1. Update and install dependencies
sudo apt update
sudo apt install -y wget tar libasound2-dev

# 2. Create the main project folder
mkdir -p ~/voiceAssistant/piper
cd ~/voiceAssistant/piper

# 3. Download and extract the Piper binary
wget https://github.com/rhasspy/piper/releases/download/2023.11.14-2/piper_linux_aarch64.tar.gz
tar -xvzf piper_linux_aarch64.tar.gz

# 4. Enter the folder where Piper was extracted
cd piper

# 5. Download the voice model files into this same folder
wget -O en_US-lessac-medium.onnx 'https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/lessac/medium/en_US-lessac-medium.onnx?download=true'
wget -O en_US-lessac-medium.onnx.json 'https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/lessac/medium/en_US-lessac-medium.onnx.json?download=true'

# 6. Grant execution permissions to the Piper engine
chmod +x piper

# 1. Update and install dependencies

sudo apt update

sudo apt install -y wget tar libasound2-dev

# 2. Create the main project folder

mkdir -p ~/voiceAssistant/piper

cd ~/voiceAssistant/piper

# 3. Download and extract the Piper binary

wget https://github.com/rhasspy/piper/releases/download/2023.11.14-2/piper_linux_aarch64.tar.gz

tar -xvzf piper_linux_aarch64.tar.gz

# 4. Enter the folder where Piper was extracted

cd piper

# 5. Download the voice model files into this same folder

wget -O en_US-lessac-medium.onnx 'https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/lessac/medium/en_US-lessac-medium.onnx?download=true'

wget -O en_US-lessac-medium.onnx.json 'https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/lessac/medium/en_US-lessac-medium.onnx.json?download=true'

# 6. Grant execution permissions to the Piper engine

chmod +x piper

Step 2: Identify Your Audio Device

Because every setup is different, we need to tell the system which speaker to use. Run the following command in your terminal:

aplay -l

aplay -l

Look through the list for your speaker. You will see something like card 0 and device 0. If your card is 0 and device is 0, your identifier is plughw:0,0. You will use these numbers in the Python script below.

Step 3: The Python Pipeline Script

Create a new Python file and paste the code below. Because we installed everything into the same folder, this script will find the files immediately.

# ====================================================================
# DISCLAIMER:
# This code is provided as-is for educational and experimental 
# purposes only. The author makes no representations or warranties of 
# any kind concerning the safety, suitability, or accuracy of this 
# code. Use at your own risk. The author assumes no liability for any 
# damages, system failures, security breaches, or network issues 
# resulting from the use or implementation of this script.
# ====================================================================
 
import subprocess

# --- CONFIGURATION ---
# piperPath: The location of the downloaded Piper executable file.
# modelPath: The path to the downloaded AI voice model file (.onnx).
# audioDevice: The specific hardware address of the speaker (e.g., plughw:0,0).
piperPath = "/home/pjm/voiceAssistant/piper/piper/piper"
modelPath = "/home/pjm/voiceAssistant/piper/piper/en_US-lessac-medium.onnx"
audioDevice = "plughw:0,0"

# --- BUILD THE COMMANDS ---
# piperCommand: This runs the AI engine. 
# --output-raw tells Piper to send out raw audio data instead of a .wav file.
piperCommand = piperPath + " --model " + modelPath + " --output-raw"

# aplayCommand: This is the Linux sound player that pipes audio to your speakers.
# -D: Specifies the device (hardware).
# -r 22050: Sets the sample rate. Piper requires 22050Hz for this model.
# -f S16_LE: Sets the format to 16-bit Signed Little Endian (the raw digital format).
# -t raw: Tells aplay that the incoming data has no header and is pure audio.
# -: The hyphen tells aplay to read input from 'stdin' (the data flowing from the pipe).
aplayCommand = "aplay -D " + audioDevice + " -r 22050 -f S16_LE -t raw -"

# fullCommand: We connect the two commands using a pipe '|'. 
# This takes the output of the AI (piperCommand) and feeds it directly 
# into the speaker input (aplayCommand).
fullCommand = piperCommand + " | " + aplayCommand

# --- RUN THE PIPELINE ---
# subprocess.Popen allows us to run the command in the background while
# letting our Python code continue to run.
speechProcess = subprocess.Popen(fullCommand, shell=True, stdin=subprocess.PIPE)

# speechMessage: The text we want the AI to read.
# We must encode it as bytes (b"...") to send it to the system.
speechMessage = "Hello students. The voice pipeline is verified and operational."
speakable=speechMessage.encode('utf-8')

# communicate: This sends the text message into the running Piper process.
speechProcess.communicate(input=speakable)

# ====================================================================

# DISCLAIMER:

# This code is provided as-is for educational and experimental

# purposes only. The author makes no representations or warranties of

# any kind concerning the safety, suitability, or accuracy of this

# code. Use at your own risk. The author assumes no liability for any

# damages, system failures, security breaches, or network issues

# resulting from the use or implementation of this script.

# ====================================================================

import subprocess

# --- CONFIGURATION ---

# piperPath: The location of the downloaded Piper executable file.

# modelPath: The path to the downloaded AI voice model file (.onnx).

# audioDevice: The specific hardware address of the speaker (e.g., plughw:0,0).

piperPath = "/home/pjm/voiceAssistant/piper/piper/piper"

modelPath = "/home/pjm/voiceAssistant/piper/piper/en_US-lessac-medium.onnx"

audioDevice = "plughw:0,0"

# --- BUILD THE COMMANDS ---

# piperCommand: This runs the AI engine.

# --output-raw tells Piper to send out raw audio data instead of a .wav file.

piperCommand = piperPath + " --model " + modelPath + " --output-raw"

# aplayCommand: This is the Linux sound player that pipes audio to your speakers.

# -D: Specifies the device (hardware).

# -r 22050: Sets the sample rate. Piper requires 22050Hz for this model.

# -f S16_LE: Sets the format to 16-bit Signed Little Endian (the raw digital format).

# -t raw: Tells aplay that the incoming data has no header and is pure audio.

# -: The hyphen tells aplay to read input from 'stdin' (the data flowing from the pipe).

aplayCommand = "aplay -D " + audioDevice + " -r 22050 -f S16_LE -t raw -"

# fullCommand: We connect the two commands using a pipe '|'.

# This takes the output of the AI (piperCommand) and feeds it directly

# into the speaker input (aplayCommand).

fullCommand = piperCommand + " | " + aplayCommand

# --- RUN THE PIPELINE ---

# subprocess.Popen allows us to run the command in the background while

# letting our Python code continue to run.

speechProcess = subprocess.Popen(fullCommand, shell=True, stdin=subprocess.PIPE)

# speechMessage: The text we want the AI to read.

# We must encode it as bytes (b"...") to send it to the system.

speechMessage = "Hello students. The voice pipeline is verified and operational."

speakable=speechMessage.encode('utf-8')

# communicate: This sends the text message into the running Piper process.

speechProcess.communicate(input=speakable)

Step 4: Customizing Your Voice

Piper has dozens of voices available. To see the full library, visit the Piper Voices Repository. Download the .onnx and .onnx.json files for your preferred voice, place them in the ~/voiceAssistant/piper/piper/ folder, and update the modelPath variable in your script.

Homework: The Talking Echo Bot

Your assignment is to play around with different voice models, and choose several that you like the best. Then modify the Python script so it becomes an interactive “Echo Bot.” Instead of hardcoding the message, use Python’s input() function to ask the user what to say. When the user types a sentence and presses Enter, your script should pipe that text into Piper and speak it back to you. Use a while True: loop to keep the program running so you can continue talking to your computer. Make a video of your working project, and the voices you chose. In the description in your video, make sure to leave a link back to the video above. That way users can easily click between my video lesson over to your video, and then back to the class video.

AI On the Edge, Raspberry Pi

AI on the Edge LESSON 34: Project Combining TTS, STT, Face Recognition and Servos on Pi 5

July 18, 2026 admin

In our previous lessons, we built individual components. We got our machine to listen locally using Speech-to-Text (STT), we got it to talk back cleanly with Text-to-Speech (TTS), and we learned how to manipulate physical hardware using precision servos. Today, we bridge the gap between software and the physical world. We are building a multi-threaded, autonomous edge system that tracks humanoids in real time.

The Engineering Mindset: Concurrency & Threading

If you try to build a system like this sequentially—running camera capture, object detection, audio listening, and audio speaking in a single while True: loop—your project will fail completely. Why? Because tasks like waiting for a voice command (stt.listen()) or synthesizing a voice line are blocking operations. If your code is stuck waiting for you to finish speaking a command, your camera frames freeze, your servo adjustments drop to zero, and your humanoid target escapes.

To solve this, we architect our software using Multi-threading. We spawn independent execution paths that run concurrently, passing data back and forth safely using thread-safe Queues:

The Main Thread: Handles the high-speed Picamera2 video capture loop, executes Haar Cascade face/eye calculations, and updates the servo angles via mathematical error tracking.
The Speak Thread: Idles quietly in the background until a message lands in the speakQ, instantly triggering the local Piper TTS engine without stalling our camera frame rate.
The Command Thread: Keeps a continuous ear open via our local STT engine. When it parses a voice command like “track”, “release”, or “quit”, it safely pops that token into the commandQ for the main loop to execute on its next pass.

The Complete Fusion Architecture Code

Below is the complete Python pipeline developed in today’s lesson. Make sure your hardware connections for the pan-and-tilt servos match pins 2 and 3 on your expansion setup, and verify your fusion_hat software stack is completely updated.

# ====================================================================
# DISCLAIMER:
# This code is provided as-is for educational and experimental 
# purposes only. The author makes no representations or warranties of 
# any kind concerning the safety, suitability, or accuracy of this 
# code. Use at your own risk. The author assumes no liability for any 
# damages, system failures, security breaches, or network issues 
# resulting from the use or implementation of this script.
# ====================================================================
 
import cv2
import time
from picamera2 import Picamera2
#########################################
import threading
from queue import Queue
from fusion_hat.tts import Piper
tts = Piper()
tts.set_model('en_US-kristin-medium')

from fusion_hat.stt import STT
stt = STT('en-us')

cnt = 0
running = True
commandQ = Queue()
speakQ = Queue()
###########################################

from fusion_hat.servo import Servo
panPin = 2
tiltPin = 3
panServo = Servo(panPin)
tiltServo = Servo(tiltPin)
panAngle = -10
tiltAngle =-20
panServo.angle(panAngle)
tiltServo.angle(tiltAngle)
piCam = Picamera2()
W=1280
H=720
tStart = time.time()
fps = 0
RES = (W,H)
piCam.preview_configuration.main.size = RES
piCam.preview_configuration.main.format = "RGB888"
piCam.preview_configuration.controls.FrameRate=60
piCam.preview_configuration.align()
piCam.configure("preview")
piCam.start()

textLowerLeft = (int(W*.01),int(H*.05))
fontFace = cv2.FONT_HERSHEY_SIMPLEX
fontThickness = int(W/425)
fontScale = H*.0015
fontColor = (0,0,255)

faceFinder = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')
eyeFinder = cv2.CascadeClassifier(cv2.data.haarcascades + "haarcascade_eye.xml")

track = False
xFrameC = int((W-1)/2)
yFrameC = int((H-1)/2)
xBoxC = xFrameC
yBoxC = yFrameC
##########################################
def speak():
    global running
    print('Speak Thread Running')
    while running:
        if speakQ.empty() == False:
            txt = speakQ.get()
            print(txt)
            tts.say(txt)
    print('Speak Thread is Terminated')
    print('')
def getCommand():
    print('Get Command Thread is Running')
    global running
    while running:
        myCommand = stt.listen(stream = False)
        myCommand = myCommand.strip()
        print(myCommand)
        if myCommand == 'quit':
            msg = 'Project Humanoid Elimination Terminated. Have a Nice Day.'
            speakQ.put(msg)
            running = False
            break
        commandQ.put(myCommand)
    print('Get Command Thread Terminated')
    print('')

speakThread = threading.Thread(target=speak,daemon=True)
speakThread.start()

commandThread = threading.Thread(target=getCommand,daemon=True)
commandThread.start()
print('Main Program Has Commenced')
msg = 'Hello, I am Ready to Eliminate Humanoids'
speakQ.put(msg)
msgOld = 'Hello, I am Ready to Eliminate Humanoids'

cv2.namedWindow('Camera',cv2.WINDOW_GUI_NORMAL)
cv2.moveWindow('Camera',0,65)
cv2.resizeWindow('Camera',W,H)
############################################
while running:
    deltaT = time.time() - tStart
    tStart=time.time()
    fps = fps*.95 + (1/deltaT)*.05
    frame= piCam.capture_array()
    frame=cv2.flip(frame,-1)
    
    if commandQ.empty() == False:
        myCommand = commandQ.get()
        if myCommand == 'track':
            track = True
        if myCommand == 'release':
            track = False

    gray = cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY)
    faces = faceFinder.detectMultiScale(gray,scaleFactor=1.1, minNeighbors =5, minSize=(100,100))
    if len(faces) != 0:
        cnt = 0 
        msg = "Humanoid Detected, Shall I track?"
    if len(faces) == 0:
        cnt = cnt + 1
        if cnt == 25:
            msg = "No Humanoid Detected, Continuing to Scan"
    if msgOld != msg:
        speakQ.put(msg)
    msgOld = msg
    for (x,y,w,h) in faces:
        cv2.rectangle(frame, (x,y),(x+w,y+h),(0,255,255),3)
        
        xBoxC = x + int(w/2)
        yBoxC = y + int(h/2)
        xError = xBoxC - xFrameC
        yError = yBoxC - yFrameC
        
        eyes =eyeFinder.detectMultiScale(gray[y:y+h,x:x+w],scaleFactor=1.1,minNeighbors=5,minSize=(15,15))
        for (i,j,w,h) in eyes:
            cv2.rectangle(frame,(x+i,y+j),(x+i+w,y+j+h),(255,0,0),3)
    
    if track == True and len(faces)&gt;0:
        panAngle = panAngle - xError/50/2
        panServo.angle(int(panAngle))
        #time.sleep(.02)
        tiltAngle = tiltAngle + yError/50/2
        tiltServo.angle(int(tiltAngle))
        #time.sleep(.02)
                        
    
    myText = "FPS: "+str(round(fps,1))
    cv2.putText(frame,myText,textLowerLeft,fontFace,fontScale,fontColor,fontThickness)
    cv2.imshow("Camera", frame)
    cv2.moveWindow("Camera",0,60)
    if cv2.waitKey(1)==ord('q'):
        break
cv2.destroyAllWindows()
print('')
print('Program Terminated')

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

# ====================================================================

# DISCLAIMER:

# This code is provided as-is for educational and experimental

# purposes only. The author makes no representations or warranties of

# any kind concerning the safety, suitability, or accuracy of this

# code. Use at your own risk. The author assumes no liability for any

# damages, system failures, security breaches, or network issues

# resulting from the use or implementation of this script.

# ====================================================================

import cv2

import time

from picamera2 import Picamera2

#########################################

import threading

from queue import Queue

from fusion_hat.tts import Piper

tts = Piper()

tts.set_model('en_US-kristin-medium')

from fusion_hat.stt import STT

stt = STT('en-us')

cnt = 0

running = True

commandQ = Queue()

speakQ = Queue()

###########################################

from fusion_hat.servo import Servo

panPin = 2

tiltPin = 3

panServo = Servo(panPin)

tiltServo = Servo(tiltPin)

panAngle = -10

tiltAngle =-20

panServo.angle(panAngle)

tiltServo.angle(tiltAngle)

piCam = Picamera2()

W=1280

H=720

tStart = time.time()

fps = 0

RES = (W,H)

piCam.preview_configuration.main.size = RES

piCam.preview_configuration.main.format = "RGB888"

piCam.preview_configuration.controls.FrameRate=60

piCam.preview_configuration.align()

piCam.configure("preview")

piCam.start()

textLowerLeft = (int(W*.01),int(H*.05))

fontFace = cv2.FONT_HERSHEY_SIMPLEX

fontThickness = int(W/425)

fontScale = H*.0015

fontColor = (0,0,255)

faceFinder = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')

eyeFinder = cv2.CascadeClassifier(cv2.data.haarcascades + "haarcascade_eye.xml")

track = False

xFrameC = int((W-1)/2)

yFrameC = int((H-1)/2)

xBoxC = xFrameC

yBoxC = yFrameC

##########################################

def speak():

global running

print('Speak Thread Running')

while running:

if speakQ.empty() == False:

txt = speakQ.get()

print(txt)

tts.say(txt)

print('Speak Thread is Terminated')

print('')

def getCommand():

print('Get Command Thread is Running')

global running

while running:

myCommand = stt.listen(stream = False)

myCommand = myCommand.strip()

print(myCommand)

if myCommand == 'quit':

msg = 'Project Humanoid Elimination Terminated. Have a Nice Day.'

speakQ.put(msg)

running = False

break

commandQ.put(myCommand)

print('Get Command Thread Terminated')

print('')

speakThread = threading.Thread(target=speak,daemon=True)

speakThread.start()

commandThread = threading.Thread(target=getCommand,daemon=True)

commandThread.start()

print('Main Program Has Commenced')

msg = 'Hello, I am Ready to Eliminate Humanoids'

speakQ.put(msg)

msgOld = 'Hello, I am Ready to Eliminate Humanoids'

cv2.namedWindow('Camera',cv2.WINDOW_GUI_NORMAL)

cv2.moveWindow('Camera',0,65)

cv2.resizeWindow('Camera',W,H)

############################################

while running:

deltaT = time.time() - tStart

tStart=time.time()

fps = fps*.95 + (1/deltaT)*.05

frame= piCam.capture_array()

frame=cv2.flip(frame,-1)

if commandQ.empty() == False:

myCommand = commandQ.get()

if myCommand == 'track':

track = True

if myCommand == 'release':

track = False

gray = cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY)

faces = faceFinder.detectMultiScale(gray,scaleFactor=1.1, minNeighbors =5, minSize=(100,100))

if len(faces) != 0:

cnt = 0

msg = "Humanoid Detected, Shall I track?"

if len(faces) == 0:

cnt = cnt + 1

if cnt == 25:

msg = "No Humanoid Detected, Continuing to Scan"

if msgOld != msg:

speakQ.put(msg)

msgOld = msg

for (x,y,w,h) in faces:

cv2.rectangle(frame, (x,y),(x+w,y+h),(0,255,255),3)

xBoxC = x + int(w/2)

yBoxC = y + int(h/2)

xError = xBoxC - xFrameC

yError = yBoxC - yFrameC

eyes =eyeFinder.detectMultiScale(gray[y:y+h,x:x+w],scaleFactor=1.1,minNeighbors=5,minSize=(15,15))

for (i,j,w,h) in eyes:

cv2.rectangle(frame,(x+i,y+j),(x+i+w,y+j+h),(255,0,0),3)

if track == True and len(faces)>0:

panAngle = panAngle - xError/50/2

panServo.angle(int(panAngle))

#time.sleep(.02)

tiltAngle = tiltAngle + yError/50/2

tiltServo.angle(int(tiltAngle))

#time.sleep(.02)

myText = "FPS: "+str(round(fps,1))

cv2.putText(frame,myText,textLowerLeft,fontFace,fontScale,fontColor,fontThickness)

cv2.imshow("Camera", frame)

cv2.moveWindow("Camera",0,60)

if cv2.waitKey(1)==ord('q'):

break

cv2.destroyAllWindows()

print('')

print('Program Terminated')

Breaking Down the Math & Logic

Pro-Tip on State Debouncing: Notice how we handle messages using the cnt variable and the msgOld buffer. If we simply asked the system to say “Humanoid Detected” every single time the frame loops, the TTS engine would crash from queue overflow, stuttering continuously. By checking if msgOld != msg, we ensure the system speaks a notification exactly once upon transition. The 25-frame hysteresis buffer ensures that a single missed frame doesn’t cause a false “No Humanoid” trigger.

Proportional Error Tracking

Look carefully at how the servo angles are adjusted:

panAngle = panAngle - xError/50/2

1	panAngle = panAngle - xError/50/2

We are calculating the error vector—how many pixels the center of the bounding box (xBoxC) is from the absolute center of our camera stream (xFrameC). Instead of moving the servo by a fixed step, we move it proportionally to the size of the error. Big error? Fast movement. Small error? Gentle, microscopic correction. Dividing by 50 dampens our proportional gain loop so our servos don’t violently oscillate and throw our camera system completely out of alignment.

Your Homework Assignment

You have the system reading frames, listening asynchronously, and adjusting angles proportionally. Now it’s time to earn your stripes as an edge systems designer.

Your Assignment: Look at our msg tracking logic. Right now, if a humanoid is detected, it simply asks, “Shall I track?”. Your task is to tie our speech input directly to this state. Modify the control flow so that the tracking logic requires verification. If a humanoid is detected, the system must wait until the commandQ yields a verified verbal confirmation—like “execute” or “yes”—before the proportional servo tracking begins. If it hears “abort”, it must break off and look away.

Drop your custom implementation script in the comments section below, explain the logic behind your state modifications, and let’s see how you optimize your loop efficiency. Go get ’em!

AI On the Edge, Tutorial

AI on the Edge LESSON 14: Control LED Color With Voice Commands on Raspberry Pi 5

May 12, 2026 admin

In Lesson 14 of AI on the Edge, we’re doing something really fun and powerful — we’re building a voice-controlled RGB LED that listens to you, changes colors on command, and even talks back with some personality! This is true edge AI running 100% locally on your Raspberry Pi with the Fusion HAT. No cloud, no internet, just fast, private, and responsive voice interaction right on your desk.

You simply speak a color — red, green, blue, cyan, magenta, yellow, off, or even quit — and the RGB LED instantly springs to life with beautiful color. But that’s not all. Every time you give a command, the system replies with a fun, playful spoken response using the Piper text-to-speech engine. It turns your Raspberry Pi into a charming little LED companion that feels alive and interactive.In this lesson, you’ll learn how to combine local Speech-to-Text with the STT library and natural-sounding Text-to-Speech with Piper. You’ll master PWM control of a full-color RGB LED through the Fusion HAT, and you’ll see how to use Python threading plus a queue to keep the voice listening running smoothly in the background without ever locking up your main program. The code is clean, well-structured, and includes proper startup greetings, graceful shutdown, and excellent resource cleanup — exactly the kind of solid practices we love in this series.What makes this project extra special is how it brings everything together. You get real-time voice recognition, instant hardware response, and spoken feedback — all happening locally on the edge. It’s fast, it’s private, and it’s incredibly satisfying to watch that LED light up exactly as you command while your Pi chats back at you.

Go ahead and watch the full Lesson 14 video, grab the complete code from the description, and build this project step by step with me. Once you have it running, I want you to play with it! Add new colors, create your own funny responses, or start thinking about how you could combine this voice control with sensors or other hardware in future projects.

This is the kind of hands-on, creative AI application that makes learning so exciting. You’re not just watching — you’re building real, useful skills that put you in the driver’s seat with artificial intelligence.

Fire up that Raspberry Pi, get your Fusion HAT ready, and let’s make some colors shine while the Pi talks back. I can’t wait to see what you create with this one!

Happy building, everyone — I’ll see you in the next lesson!

This is the schematic we are using for the project:

Fusion Hat Circuit Diagram — This is the circuit we will use moving forward in the class

This is the code we developed in the video:

# ====================================================================
# DISCLAIMER:
# This code is provided as-is for educational and experimental 
# purposes only. The author makes no representations or warranties of 
# any kind concerning the safety, suitability, or accuracy of this 
# code. Use at your own risk. The author assumes no liability for any 
# damages, system failures, security breaches, or network issues 
# resulting from the use or implementation of this script.
# ====================================================================
 
from fusion_hat.pwm import PWM
from fusion_hat.stt import STT
import threading
from queue import Queue
from time import sleep
from fusion_hat.tts import Piper
tts = Piper()
tts.set_model('en_US-kristin-medium')
msg = 'Speak your favorite color, and your wish is my command'
tts.say(msg,stream=False)

redPin = 5
greenPin = 6
bluePin = 7

redLED= PWM(redPin)
greenLED=PWM(greenPin)
blueLED=PWM(bluePin)

rVal=0
gVal=0
bVal=0

stt = STT('en-us')
running=True
colorQ = Queue()

def getColor():
    print("Input Thread is Running")
    global running
    while running:
        print('What color: red, green, blue, cyan, magenta, yellow, off, quit')
        myColor = stt.listen(stream=False)
        myColor=myColor.strip()
        if myColor == 'quit':
            running = False
            msg = 'So Sorry to See you go, Please Call Me Again Soon'
            tts.say(msg, stream=False)
            break
        colorQ.put(myColor)
    print("Thread is Terminated")
colorThread= threading.Thread(target=getColor,daemon=True)
colorThread.start()
print("Main Program is Started")
try:
    while running:
        if colorQ.empty() == False:
            myColor = colorQ.get()
            print('Color: ',myColor)
            if myColor == 'off':
                rVal = 0
                gVal = 0
                bVal = 0
                msg = 'Please bring back your beautiful colors!'
                tts.say(msg, stream=False)
            if myColor == 'red' or myColor=='read':
                rVal = 100
                gVal = 0
                bVal = 0
                msg = 'I can not get you out of my head, so I will turn it red'
                tts.say(msg, stream=False)
            if myColor == 'green':
                rVal = 0
                gVal = 100
                bVal = 0
                msg = 'Because I want you to be seen, I will turn it green'
                tts.say(msg, stream=False)
            if myColor == 'blue':
                rVal = 0
                gVal = 0
                bVal = 100
                msg = 'Because I love you, I will turn it blue'
                tts.say(msg, stream=False)
            if myColor == 'cyan':
                rVal = 0
                gVal = 100
                bVal = 25
                msg = 'You turn my world cyan and bright, what a beautiful sight!'
                tts.say(msg, stream=False)
            if myColor == 'magenta':
                rVal = 100
                gVal = 0
                bVal = 100
                msg = 'You Light my Magenta Fire, You are my burning Desire'
                tts.say(msg, stream=False)
            if myColor == 'yellow':
                rVal = 100
                gVal = 25
                bVal = 0
                msg = 'You are such a handsome fellow, I will turn it yellow'
                tts.say(msg, stream=False)          
            myColor = 'null'
            redLED.pulse_width_percent(rVal)
            greenLED.pulse_width_percent(gVal)
            blueLED.pulse_width_percent(bVal)
    redLED.pulse_width_percent(0)
    greenLED.pulse_width_percent(0)
    blueLED.pulse_width_percent(0)
    redLED.enable(False)
    greenLED.enable(False)
    blueLED.enable(False)
    print("LEDs are Released")
    print("Program is Terminated")            
         
except KeyboardInterrupt:
    running = False
    redLED.pulse_width_percent(0)
    greenLED.pulse_width_percent(0)
    blueLED.pulse_width_percent(0)
    redLED.enable(False)
    greenLED.enable(False)
    blueLED.enable(False)
    print("LEDs are Released")
    print("Program is Terminated")

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

# ====================================================================

# DISCLAIMER:

# This code is provided as-is for educational and experimental

# purposes only. The author makes no representations or warranties of

# any kind concerning the safety, suitability, or accuracy of this

# code. Use at your own risk. The author assumes no liability for any

# damages, system failures, security breaches, or network issues

# resulting from the use or implementation of this script.

# ====================================================================

from fusion_hat.pwm import PWM

from fusion_hat.stt import STT

import threading

from queue import Queue

from time import sleep

from fusion_hat.tts import Piper

tts = Piper()

tts.set_model('en_US-kristin-medium')

msg = 'Speak your favorite color, and your wish is my command'

tts.say(msg,stream=False)

redPin = 5

greenPin = 6

bluePin = 7

redLED= PWM(redPin)

greenLED=PWM(greenPin)

blueLED=PWM(bluePin)

rVal=0

gVal=0

bVal=0

stt = STT('en-us')

running=True

colorQ = Queue()

def getColor():

print("Input Thread is Running")

global running

while running:

print('What color: red, green, blue, cyan, magenta, yellow, off, quit')

myColor = stt.listen(stream=False)

myColor=myColor.strip()

if myColor == 'quit':

running = False

msg = 'So Sorry to See you go, Please Call Me Again Soon'

tts.say(msg, stream=False)

break

colorQ.put(myColor)

print("Thread is Terminated")

colorThread= threading.Thread(target=getColor,daemon=True)

colorThread.start()

print("Main Program is Started")

try:

while running:

if colorQ.empty() == False:

myColor = colorQ.get()

print('Color: ',myColor)

if myColor == 'off':

rVal = 0

gVal = 0

bVal = 0

msg = 'Please bring back your beautiful colors!'

tts.say(msg, stream=False)

if myColor == 'red' or myColor=='read':

rVal = 100

gVal = 0

bVal = 0

msg = 'I can not get you out of my head, so I will turn it red'

tts.say(msg, stream=False)

if myColor == 'green':

rVal = 0

gVal = 100

bVal = 0

msg = 'Because I want you to be seen, I will turn it green'

tts.say(msg, stream=False)

if myColor == 'blue':

rVal = 0

gVal = 0

bVal = 100

msg = 'Because I love you, I will turn it blue'

tts.say(msg, stream=False)

if myColor == 'cyan':

rVal = 0

gVal = 100

bVal = 25

msg = 'You turn my world cyan and bright, what a beautiful sight!'

tts.say(msg, stream=False)

if myColor == 'magenta':

rVal = 100

gVal = 0

bVal = 100

msg = 'You Light my Magenta Fire, You are my burning Desire'

tts.say(msg, stream=False)

if myColor == 'yellow':

rVal = 100

gVal = 25

bVal = 0

msg = 'You are such a handsome fellow, I will turn it yellow'

tts.say(msg, stream=False)

myColor = 'null'

redLED.pulse_width_percent(rVal)

greenLED.pulse_width_percent(gVal)

blueLED.pulse_width_percent(bVal)

redLED.pulse_width_percent(0)

greenLED.pulse_width_percent(0)

blueLED.pulse_width_percent(0)

redLED.enable(False)

greenLED.enable(False)

blueLED.enable(False)

print("LEDs are Released")

print("Program is Terminated")

except KeyboardInterrupt:

running = False

redLED.pulse_width_percent(0)

greenLED.pulse_width_percent(0)

blueLED.pulse_width_percent(0)

redLED.enable(False)

greenLED.enable(False)

blueLED.enable(False)

print("LEDs are Released")

print("Program is Terminated")

AI On the Edge

AI on the Edge LESSON 9: Make Your Raspberry Pi Talk with TTS

April 26, 2026 admin

In today’s lesson, we’re going to make a huge leap forward in giving our Raspberry Pi some real personality. We’re going to teach it how to talk using Text-to-Speech (TTS). Instead of just blinking LEDs or printing text to the screen, our Pi will now speak out loud with a clear, natural-sounding voice.

This is a really fun and important lesson because one of the main goals of this class is to build intelligent systems that can interact with us in more human ways. Being able to make your Raspberry Pi speak opens up all kinds of exciting possibilities — whether you want your robot to tell you what it sees, have your AI assistant read sensor data out loud, or just add some personality and humor to your projects.

In this video, I show you how to use the TTS capabilities on the SunFounder Fusion AI Hat. You’ll learn how to install and set up the TTS engine, speak simple sentences, change voices, and control when the Pi talks. We’ll also look at how to make the speech sound more natural and how to integrate it smoothly into your programs without freezing everything else.

By the end of this lesson, your Raspberry Pi will be able to speak clearly and confidently — which is going to make the rest of our AI on the Edge journey a lot more exciting. Voice output combined with voice input (which we’ll work on soon) is what turns a simple circuit into a real interactive AI companion.

So go ahead and grab your Fusion AI Hat, plug in a speaker, and let’s give your Raspberry Pi a voice! As always, I encourage you to code along with me in the video and then experiment. Try making it say funny things, read temperatures, announce when it detects a face — whatever sparks your creativity.

This is where your projects start to feel truly alive.

I’m really excited for you on this one — let’s make your Raspberry Pi talk!

This is the schematic we are using on these projects;

This Schematic is explained in detail in LESSON #5.

Then this is the code we developed in today’s lesson.

# ====================================================================
# DISCLAIMER:
# This code is provided as-is for educational and experimental 
# purposes only. The author makes no representations or warranties of 
# any kind concerning the safety, suitability, or accuracy of this 
# code. Use at your own risk. The author assumes no liability for any 
# damages, system failures, security breaches, or network issues 
# resulting from the use or implementation of this script.
# ====================================================================
 
from fusion_hat.adc import ADC
from fusion_hat.pwm import PWM
from fusion_hat.tts import Piper
from time import sleep
tts = Piper()
tts.set_model('en_US-amy-low')
potPin = 0
myPot = ADC(potPin)
redPin = 5
redLED = PWM(redPin)
rangeOld = 100

while True:
    potVal = myPot.read()
    writeVal=100*(potVal/4095) - .005
    if writeVal < 1:
        msg = "Your LED is Off"
        range = 0
    if writeVal >=1 and writeVal<10:
        msg = "Your LED is Brightness 1"
        range =  1
    if writeVal >=10 and writeVal<20:
        msg = "Your LED is Brightness 2"
        range=2
    if writeVal >=20 and writeVal<30:
        msg = "Your LED is Brightness 3"
        range=3
    if writeVal >=30 and writeVal<40:
        msg = "Your LED is Brightness 4"
        range=4
    if writeVal >=40 and writeVal<50:
        msg = "Your LED is Brightness 5"
        range=5
    if writeVal >=50 and writeVal<60:
        msg = "Your LED is Brightness 6"
        range=6
    if writeVal >=60 and writeVal<70:
        msg = "Your LED is Brightness 7"
        range=7
    if writeVal >=70 and writeVal<80:
        msg = "Your LED is Brightness 8"
        range=8
    if writeVal >=80 and writeVal<90:
        msg = "Your LED is Brightness 9"
        range=9
    if writeVal >=90 and writeVal<99:
        msg = "Your LED is Brightness 10"
        range=10
    if writeVal >=99:
        msg = "Your LED is full Brightness"
        range=11
    if rangeOld != range:
        tts.say(msg, stream = False)
    rangeOld=range
    redLED.pulse_width_percent(int(writeVal))
    sleep(.1)

# ====================================================================

# DISCLAIMER:

# This code is provided as-is for educational and experimental

# purposes only. The author makes no representations or warranties of

# any kind concerning the safety, suitability, or accuracy of this

# code. Use at your own risk. The author assumes no liability for any

# damages, system failures, security breaches, or network issues

# resulting from the use or implementation of this script.

# ====================================================================

from fusion_hat.adc import ADC

from fusion_hat.pwm import PWM

from fusion_hat.tts import Piper

from time import sleep

tts = Piper()

tts.set_model('en_US-amy-low')

potPin = 0

myPot = ADC(potPin)

redPin = 5

redLED = PWM(redPin)

rangeOld = 100

while True:

potVal = myPot.read()

writeVal=100*(potVal/4095) - .005

if writeVal < 1:

msg = "Your LED is Off"

range = 0

if writeVal >=1 and writeVal<10:

msg = "Your LED is Brightness 1"

range = 1

if writeVal >=10 and writeVal<20:

msg = "Your LED is Brightness 2"

range=2

if writeVal >=20 and writeVal<30:

msg = "Your LED is Brightness 3"

range=3

if writeVal >=30 and writeVal<40:

msg = "Your LED is Brightness 4"

range=4

if writeVal >=40 and writeVal<50:

msg = "Your LED is Brightness 5"

range=5

if writeVal >=50 and writeVal<60:

msg = "Your LED is Brightness 6"

range=6

if writeVal >=60 and writeVal<70:

msg = "Your LED is Brightness 7"

range=7

if writeVal >=70 and writeVal<80:

msg = "Your LED is Brightness 8"

range=8

if writeVal >=80 and writeVal<90:

msg = "Your LED is Brightness 9"

range=9

if writeVal >=90 and writeVal<99:

msg = "Your LED is Brightness 10"

range=10

if writeVal >=99:

msg = "Your LED is full Brightness"

range=11

if rangeOld != range:

tts.say(msg, stream = False)

rangeOld=range

redLED.pulse_width_percent(int(writeVal))

sleep(.1)

Technology Tutorials

Tag Archives: TTS

AI on the Edge LESSON 35: Running Multiple Pi Cameras and USB Cameras on the Pi 5

Codex Knowledge: Multi-Camera Architecture on the Pi 5

General Knowledge: The Evolution of Multi-Stream Machine Vision

Python Source Code

Conclusion

AI on the NVIDIA Jetson Orin Nano: Adding Text-to-Speech (TTS) with Piper

Step 1: System Prep & Piper Installation

Step 2: Identify Your Audio Device

Step 3: The Python Pipeline Script

Step 4: Customizing Your Voice

Homework: The Talking Echo Bot

AI on the Edge LESSON 34: Project Combining TTS, STT, Face Recognition and Servos on Pi 5

The Engineering Mindset: Concurrency & Threading

The Complete Fusion Architecture Code

Proportional Error Tracking

Your Homework Assignment

AI on the Edge LESSON 14: Control LED Color With Voice Commands on Raspberry Pi 5

AI on the Edge LESSON 9: Make Your Raspberry Pi Talk with TTS

Making The World a Better Place One High Tech Project at a Time. Enjoy!