Tag Archives: Speech Recognition

AI on the Edge LESSON 36: Select Active Camera in OpenCV With Voice Commands

July 24, 2026 admin

Welcome back, makers, engineers, and AI enthusiasts! In our previous lessons, we built out robust multi-camera setups, streaming feeds from both Raspberry Pi cameras and high-definition USB cameras. But watching four feeds in separate tiles is only half the battle. What happens when you want to interact with your system hands-free?

In this lesson, we are taking our edge vision projects to the next level by integrating voice commands. You will learn how to run a dedicated speech-to-text thread in the background, catch voice triggers safely using a thread-safe queue, and dynamically switch your active main camera feed in OpenCV on the fly—just by speaking.

What You Will Learn in This Lesson

Multi-Threaded Speech Recognition: How to run the stt listener in a separate daemon thread so it never blocks or stutters your high-FPS video processing loop.
Thread-Safe Communication: Using Python’s Queue module to pass voice commands seamlessly from the background listening thread into your main application loop.
Handling STT Variations: Accounting for common speech-to-text homophone variations (like “two”, “to”, and “too”, or “four” and “for”) to make your voice control robust and reliable.
Dynamic Frame Routing: Mapping spoken commands to specific camera streams (piCam1, piCam2, usbCam1, and usbCam2) and updating your primary display window instantly.
Organized Window Layouts: Positioning and sizing multiple OpenCV windows on your desktop workspace for a clean, professional multi-camera dashboard.

Step-by-Step Breakdown of the Script

1. Setting Up the Background Voice Thread

When working with real-time video processing in OpenCV, blocking functions are your worst enemy. If you call a speech recognition listening function directly inside your main while loop, your video frames will freeze while waiting for audio input.

To prevent this, we initialize a background worker function (getCamera) and launch it as a daemon thread:

The thread continuously listens for your voice commands using the Fusion Hat STT module.
Once a command is captured, it strips whitespace, checks for exit triggers (like saying 'quit'), and pushes valid commands directly into our commandQ.

2. Initializing Multiple Cameras

Our setup harnesses the full power of the hardware by combining native Raspberry Pi camera interfaces with standard USB webcams:

Pi Cameras: Configured via Picamera2 at a crisp 1280x720 resolution with RGB888 formatting running smoothly at 60 frames per second.
USB Cameras: Initialized through OpenCV’s VideoCapture class, set to target resolution and optimized for 30 FPS.

3. Processing and Routing Commands in the Main Loop

Inside the primary application loop, our script handles three core tasks simultaneously:

Calculate Performance: Continuously tracks and smooths out the frames-per-second (FPS) metric so you can monitor system load.
Check the Command Queue: Non-blockingly checks if the background thread has dropped a new camera selection into commandQ. If a new command is waiting, it updates the mainCam variable.
Route the Main Frame: Evaluates the active camera string—including clever fallback checks for common voice misinterpretations like 'camera to' or 'camera for'—and assigns the corresponding video stream to mainFrame.

4. Managing Your OpenCV Windows

To give you a complete command-center experience, the script generates a large primary display window for your active camera view, accompanied by a clean row of smaller preview tiles across the bottom of your screen for all four connected feeds.

# ====================================================================
# DISCLAIMER:
# This code is provided as-is for educational and experimental 
# purposes only. The author makes no representations or warranties of 
# any kind concerning the safety, suitability, or accuracy of this 
# code. Use at your own risk. The author assumes no liability for any 
# damages, system failures, security breaches, or network issues 
# resulting from the use or implementation of this script.
# ====================================================================
import cv2
import time
from picamera2 import Picamera2
import threading
from queue import Queue
from fusion_hat.stt import STT
stt = STT('en-us')
commandQ = Queue()
mainCam = 'camera one'

W=1280
H=720
tStart = time.time()
fps = 0
RES = (W,H)

piCam1 = Picamera2(0)
piCam1.preview_configuration.main.size = RES
piCam1.preview_configuration.main.format = "RGB888"
piCam1.preview_configuration.controls.FrameRate=60
piCam1.preview_configuration.align()
piCam1.configure("preview")
piCam1.start()

piCam2 = Picamera2(1)
piCam2.preview_configuration.main.size = RES
piCam2.preview_configuration.main.format = "RGB888"
piCam2.preview_configuration.controls.FrameRate=60
piCam2.preview_configuration.align()
piCam2.configure("preview")
piCam2.start()

usbCam1 = cv2.VideoCapture(16)
usbCam1.set(cv2.CAP_PROP_FRAME_WIDTH,W)
usbCam1.set(cv2.CAP_PROP_FRAME_HEIGHT,H)
usbCam1.set(cv2.CAP_PROP_FPS, 30)

usbCam2 = cv2.VideoCapture(18)
usbCam2.set(cv2.CAP_PROP_FRAME_WIDTH,W)
usbCam2.set(cv2.CAP_PROP_FRAME_HEIGHT,H)
usbCam2.set(cv2.CAP_PROP_FPS, 30)

textLowerLeft = (int(W*.01),int(H*.05))
fontFace = cv2.FONT_HERSHEY_SIMPLEX
fontThickness = int(W/425)
fontScale = H*.0015
fontColor = (0,0,255)

running = True
def getCamera():
    print('Voice Thread Started')
    global running
    while running:
        myCamera = stt.listen(stream = False)
        myCamera = myCamera.strip()
        print(myCamera)
        if myCamera == 'quit':
            print('Security Camera Offline')
            running = False
            break
        commandQ.put(myCamera)
    print('Thread Terminated')

cameraThread = threading.Thread(target = getCamera,daemon=True)
cameraThread.start()

cv2.namedWindow('Main Camera',cv2.WINDOW_GUI_NORMAL)
cv2.moveWindow('Main Camera',0,65)
cv2.resizeWindow('Main Camera',W,H)

cv2.namedWindow('Camera1',cv2.WINDOW_GUI_NORMAL)
cv2.moveWindow('Camera1',0,H)
cv2.resizeWindow('Camera1',int(W/4),int(H/4))

cv2.namedWindow('Camera2',cv2.WINDOW_GUI_NORMAL)
cv2.moveWindow('Camera2',int(W/4),H)
cv2.resizeWindow('Camera2',int(W/4),int(H/4))

cv2.namedWindow('Camera3',cv2.WINDOW_GUI_NORMAL)
cv2.moveWindow('Camera3',2*int(W/4),H)
cv2.resizeWindow('Camera3',int(W/4),int(H/4))

cv2.namedWindow('Camera4',cv2.WINDOW_GUI_NORMAL)
cv2.moveWindow('Camera4',3*int(W/4),H)
cv2.resizeWindow('Camera4',int(W/4),int(H/4))

while running:
    deltaT = time.time() - tStart
    tStart=time.time()
    fps = fps*.95 + (1/deltaT)*.05
    frame1= piCam1.capture_array()
    #frame1=cv2.flip(frame1,-1)
    
    frame2= piCam2.capture_array()
    frame2=cv2.flip(frame2,-1)
    
    _ , frame3 = usbCam1.read()
    _ , frame4 = usbCam2.read()
    mainFrame = frame1
    
    if commandQ.empty() == False:
        mainCam = commandQ.get()
        
    if mainCam == 'camera one':
        mainFrame = frame1
    if mainCam == 'camera two' or mainCam == 'camera to' or mainCam == 'camera too':
        mainFrame = frame2
    if mainCam == 'camera three':
        mainFrame = frame3
    if mainCam == 'camera four' or mainCam == 'camera for':
        mainFrame = frame4
    myText = "FPS: "+str(round(fps,1))
    cv2.putText(frame1,myText,textLowerLeft,fontFace,fontScale,fontColor,fontThickness)
    cv2.imshow("Main Camera", mainFrame)
    cv2.imshow("Camera1", frame1)
    cv2.imshow("Camera2", frame2)
    cv2.imshow("Camera3", frame3)
    cv2.imshow("Camera4", frame4)    
    if cv2.waitKey(1)==ord('q'):
        break
cv2.destroyAllWindows()
print('Program Terminated')

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

# ====================================================================

# DISCLAIMER:

# This code is provided as-is for educational and experimental

# purposes only. The author makes no representations or warranties of

# any kind concerning the safety, suitability, or accuracy of this

# code. Use at your own risk. The author assumes no liability for any

# damages, system failures, security breaches, or network issues

# resulting from the use or implementation of this script.

# ====================================================================

import cv2

import time

from picamera2 import Picamera2

import threading

from queue import Queue

from fusion_hat.stt import STT

stt = STT('en-us')

commandQ = Queue()

mainCam = 'camera one'

W=1280

H=720

tStart = time.time()

fps = 0

RES = (W,H)

piCam1 = Picamera2(0)

piCam1.preview_configuration.main.size = RES

piCam1.preview_configuration.main.format = "RGB888"

piCam1.preview_configuration.controls.FrameRate=60

piCam1.preview_configuration.align()

piCam1.configure("preview")

piCam1.start()

piCam2 = Picamera2(1)

piCam2.preview_configuration.main.size = RES

piCam2.preview_configuration.main.format = "RGB888"

piCam2.preview_configuration.controls.FrameRate=60

piCam2.preview_configuration.align()

piCam2.configure("preview")

piCam2.start()

usbCam1 = cv2.VideoCapture(16)

usbCam1.set(cv2.CAP_PROP_FRAME_WIDTH,W)

usbCam1.set(cv2.CAP_PROP_FRAME_HEIGHT,H)

usbCam1.set(cv2.CAP_PROP_FPS, 30)

usbCam2 = cv2.VideoCapture(18)

usbCam2.set(cv2.CAP_PROP_FRAME_WIDTH,W)

usbCam2.set(cv2.CAP_PROP_FRAME_HEIGHT,H)

usbCam2.set(cv2.CAP_PROP_FPS, 30)

textLowerLeft = (int(W*.01),int(H*.05))

fontFace = cv2.FONT_HERSHEY_SIMPLEX

fontThickness = int(W/425)

fontScale = H*.0015

fontColor = (0,0,255)

running = True

def getCamera():

print('Voice Thread Started')

global running

while running:

myCamera = stt.listen(stream = False)

myCamera = myCamera.strip()

print(myCamera)

if myCamera == 'quit':

print('Security Camera Offline')

running = False

break

commandQ.put(myCamera)

print('Thread Terminated')

cameraThread = threading.Thread(target = getCamera,daemon=True)

cameraThread.start()

cv2.namedWindow('Main Camera',cv2.WINDOW_GUI_NORMAL)

cv2.moveWindow('Main Camera',0,65)

cv2.resizeWindow('Main Camera',W,H)

cv2.namedWindow('Camera1',cv2.WINDOW_GUI_NORMAL)

cv2.moveWindow('Camera1',0,H)

cv2.resizeWindow('Camera1',int(W/4),int(H/4))

cv2.namedWindow('Camera2',cv2.WINDOW_GUI_NORMAL)

cv2.moveWindow('Camera2',int(W/4),H)

cv2.resizeWindow('Camera2',int(W/4),int(H/4))

cv2.namedWindow('Camera3',cv2.WINDOW_GUI_NORMAL)

cv2.moveWindow('Camera3',2*int(W/4),H)

cv2.resizeWindow('Camera3',int(W/4),int(H/4))

cv2.namedWindow('Camera4',cv2.WINDOW_GUI_NORMAL)

cv2.moveWindow('Camera4',3*int(W/4),H)

cv2.resizeWindow('Camera4',int(W/4),int(H/4))

while running:

deltaT = time.time() - tStart

tStart=time.time()

fps = fps*.95 + (1/deltaT)*.05

frame1= piCam1.capture_array()

#frame1=cv2.flip(frame1,-1)

frame2= piCam2.capture_array()

frame2=cv2.flip(frame2,-1)

_ , frame3 = usbCam1.read()

_ , frame4 = usbCam2.read()

mainFrame = frame1

if commandQ.empty() == False:

mainCam = commandQ.get()

if mainCam == 'camera one':

mainFrame = frame1

if mainCam == 'camera two' or mainCam == 'camera to' or mainCam == 'camera too':

mainFrame = frame2

if mainCam == 'camera three':

mainFrame = frame3

if mainCam == 'camera four' or mainCam == 'camera for':

mainFrame = frame4

myText = "FPS: "+str(round(fps,1))

cv2.putText(frame1,myText,textLowerLeft,fontFace,fontScale,fontColor,fontThickness)

cv2.imshow("Main Camera", mainFrame)

cv2.imshow("Camera1", frame1)

cv2.imshow("Camera2", frame2)

cv2.imshow("Camera3", frame3)

cv2.imshow("Camera4", frame4)

if cv2.waitKey(1)==ord('q'):

break

cv2.destroyAllWindows()

print('Program Terminated')

AI On the Edge, Tutorial

AI on the Edge LESSON 13: Control LED Brightness with Voice Commands on Raspberry Pi 5

May 9, 2026 admin

Hey everyone, welcome back to the AI on the Edge series!

In today’s lesson, we’re taking another big step forward in building truly interactive AI projects that run right on our Raspberry Pi 5. We’re going to give our hardware a voice — literally. You’ll learn how to control the brightness of an LED using simple voice commands like “low”, “medium”, “high”, “on”, and “off”.

This lesson builds directly on the speech-to-text skills we learned earlier. Using the Fusion Hat’s microphone and the excellent STT library, we create a system where you can speak naturally to your Pi and it responds instantly by changing the LED brightness. We also bring in Python threading so the voice listening doesn’t block the main program — which is a critical skill as our projects get more complex.

One of the things I really like about this project is how it shows the power of combining AI with real hardware. You’re not just making the LED turn on and off anymore — you’re giving it smooth, adjustable brightness control using nothing but your voice. It’s a perfect example of the kind of interactive, intelligent edge computing we’re working toward in this class.

By the end of this lesson, you’ll have a solid understanding of how to use voice commands to control hardware, how to manage multiple things happening at the same time with threading, and how to create a much more natural and user-friendly interface for your projects.

This is the kind of thing that makes your Raspberry Pi projects feel alive and responsive. Whether you eventually want to control motors, lights, robots, or entire systems with your voice, the techniques you learn in this lesson will serve as a strong foundation.

So grab your SunFounder Fusion AI Hat, hook up that red LED, and let’s get your Raspberry Pi listening and responding to your voice commands like a proper smart device!

As always, I encourage you to type the code along with me in the video, then play around with it. Try adding more commands, change the LED color, or combine it with other sensors. That’s where the real learning and creativity happens.

I’m really excited to see what you build with this one!

This is the schematic we are using, from LESSON #5.

Fusion Hat Circuit Diagram — This is the circuit we will use moving forward in the class

In the video, this is the code we developed:

# ====================================================================
# DISCLAIMER:
# This code is provided as-is for educational and experimental 
# purposes only. The author makes no representations or warranties of 
# any kind concerning the safety, suitability, or accuracy of this 
# code. Use at your own risk. The author assumes no liability for any 
# damages, system failures, security breaches, or network issues 
# resulting from the use or implementation of this script.
# ====================================================================
 
from fusion_hat.pwm import PWM
from fusion_hat.stt import STT
import threading
from queue import Queue
from time import sleep

redPin = 5
redLED = PWM(redPin)
redLED.freq(1000)

stt = STT(language ="en-us")

brightness = "off"
bNumb = 0
running = True
brightQ = Queue()
def blinkBright():
    print("Input Thread is Running")
    global running
    while running:
        print("What Brightness: (off, low, medium, high, on or quit: )")
        cmd = stt.listen(stream = False)
        cmd = cmd.strip()
        print(cmd)
        if cmd == "quit":
            running = False
            break
        brightQ.put(cmd)
    print("Thread is Terminated")
brightThread = threading.Thread(target = blinkBright, daemon = True)
brightThread.start()
print("Main Program Has Started")
try:
    while running:
        if brightQ.empty() == False:
            brightness = brightQ.get()
            if brightness == "quit":
                break
        if brightness == "off":
            bNumb = 0
        if brightness == "low" or brightness == "whoa":
            bNumb = 8
        if brightness == "medium":
            bNumb = 25
        if brightness == "high" or brightness == "hi":
            bNumb = 60
        if brightness == "on":
            bNumb = 100
        redLED.pulse_width_percent(bNumb)
    redLED.pulse_width_percent(0)
    redLED.enable(False)
    print("LED Released")
    print("Program is Terminated")
        
except KeyboardInterrupt:
    running = False
    redLED.pulse_width_percent(0)
    redLED.enable(False)
    print("LED Released")
    print("Program Terminated")

# ====================================================================

# DISCLAIMER:

# This code is provided as-is for educational and experimental

# purposes only. The author makes no representations or warranties of

# any kind concerning the safety, suitability, or accuracy of this

# code. Use at your own risk. The author assumes no liability for any

# damages, system failures, security breaches, or network issues

# resulting from the use or implementation of this script.

# ====================================================================

from fusion_hat.pwm import PWM

from fusion_hat.stt import STT

import threading

from queue import Queue

from time import sleep

redPin = 5

redLED = PWM(redPin)

redLED.freq(1000)

stt = STT(language ="en-us")

brightness = "off"

bNumb = 0

running = True

brightQ = Queue()

def blinkBright():

print("Input Thread is Running")

global running

while running:

print("What Brightness: (off, low, medium, high, on or quit: )")

cmd = stt.listen(stream = False)

cmd = cmd.strip()

print(cmd)

if cmd == "quit":

running = False

break

brightQ.put(cmd)

print("Thread is Terminated")

brightThread = threading.Thread(target = blinkBright, daemon = True)

brightThread.start()

print("Main Program Has Started")

try:

while running:

if brightQ.empty() == False:

brightness = brightQ.get()

if brightness == "quit":

break

if brightness == "off":

bNumb = 0

if brightness == "low" or brightness == "whoa":

bNumb = 8

if brightness == "medium":

bNumb = 25

if brightness == "high" or brightness == "hi":

bNumb = 60

if brightness == "on":

bNumb = 100

redLED.pulse_width_percent(bNumb)

redLED.pulse_width_percent(0)

redLED.enable(False)

print("LED Released")

print("Program is Terminated")

except KeyboardInterrupt:

running = False

redLED.pulse_width_percent(0)

redLED.enable(False)

print("LED Released")

print("Program Terminated")

AI On the Edge, Tutorial

AI on the Edge LESSON 11: Control LED on Raspberry Pi With Voice Commands

May 1, 2026 admin

In today’s lesson, we’re taking our first exciting step into giving our Raspberry Pi the ability to understand and respond to our voice. That’s right — we’re going to control a physical LED using nothing but spoken commands! This is a huge milestone in the class because it marks the beginning of building truly interactive AI projects that can listen to us and take action in the real world.

Using the SunFounder Fusion AI Hat’s built-in microphone and the excellent STT (Speech-to-Text) library, we create a simple but powerful voice assistant that can turn an LED on and off with commands like “on”, “off”, and “quit”. I walk you through every single line of the code so you can clearly see how we capture voice input, process the command, and control real hardware.

This lesson is intentionally straightforward because I want you to build a strong foundation. Once you understand how to take a voice command and turn it into physical action, we can start adding more complexity — like controlling multiple devices, adjusting brightness, or even combining voice control with computer vision in future lessons.

One of the things I love most about this project is how it makes the Raspberry Pi feel “alive.” Instead of clicking buttons or typing commands, you can now talk directly to your project. This is the kind of interaction that makes edge AI projects so much fun and so powerful.

By the end of this lesson, you’ll have a working voice-controlled LED and the confidence to start expanding your voice control skills. This is exactly the kind of capability we need as we move forward in the AI on the Edge journey — giving our intelligent systems natural, human-friendly ways to interact with us.

So grab your Fusion AI Hat, hook up that LED, and let’s turn your Raspberry Pi into a voice-controlled device! As always, I strongly encourage you to code along with me in the video and then play around with the program. Try adding more commands, control multiple LEDs, or even have it say something back to you.

This is where things start getting really fun. Let’s get that LED responding to your voice!

This is the schematic of the circuit we are using for our AI class. We go into great detail on this schematic in LESSON #5 if you want to learn more about it.

Now this is the code we developed in this lesson:

# ====================================================================
# DISCLAIMER:
# This code is provided as-is for educational and experimental 
# purposes only. The author makes no representations or warranties of 
# any kind concerning the safety, suitability, or accuracy of this 
# code. Use at your own risk. The author assumes no liability for any 
# damages, system failures, security breaches, or network issues 
# resulting from the use or implementation of this script.
# ====================================================================
 
from fusion_hat.stt import STT
from fusion_hat.pin import Pin, Mode
from time import sleep

redPin = 17
redLED = Pin(redPin,mode=Mode.OUT)
stt = STT(language="en-us")
while True:
    print("Voice Assistant At the READY!, Speak at Any Time: ")
    command = stt.listen(stream=False)
    command = command.strip()
    print(command)
    if command == "on":
        redLED.high()
        print("LED ON")
    if command == "off":
        redLED.low()
        print("LED OFF")
    if command == "quit":
        break
redLED.low()

# ====================================================================

# DISCLAIMER:

# This code is provided as-is for educational and experimental

# purposes only. The author makes no representations or warranties of

# any kind concerning the safety, suitability, or accuracy of this

# code. Use at your own risk. The author assumes no liability for any

# damages, system failures, security breaches, or network issues

# resulting from the use or implementation of this script.

# ====================================================================

from fusion_hat.stt import STT

from fusion_hat.pin import Pin, Mode

from time import sleep

redPin = 17

redLED = Pin(redPin,mode=Mode.OUT)

stt = STT(language="en-us")

while True:

print("Voice Assistant At the READY!, Speak at Any Time: ")

command = stt.listen(stream=False)

command = command.strip()

print(command)

if command == "on":

redLED.high()

print("LED ON")

if command == "off":

redLED.low()

print("LED OFF")

if command == "quit":

break

redLED.low()

Jetson Nano

AI on the Jetson Nano LESSON 61: Image Recognition and Speech (TTS) on the Nano

September 26, 2020 admin

In this video lesson we learn how to add speech to our NVIDIA Jetson Nano we demonstrate how the Jetson can not only recognize an item, but can audibly speak the item it sees. The video takes you through the process step-by-step, and shows you how to make it all work together properly. For your convenience, the code we developed is included below.

import jetson.inference
import jetson.utils
import numpy as np 
import time
import os
from gtts import gTTS
import threading

speak=True
item='Welcome to My Identify. Are you Ready to Rumble?'
confidence=0
itemOld=''


import cv2
print(cv2.__version__)
width=1280
height=720
flip=2
#Uncomment These next Two Line for Pi Camera
#camSet='nvarguscamerasrc !  video/x-raw(memory:NVMM), width=3264, height=2464, format=NV12, framerate=21/1 ! nvvidconv flip-method='+str(flip)+' ! video/x-raw, width='+str(dispW)+', height='+str(dispH)+', format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink'
#cam= cv2.VideoCapture(camSet)

def sayItem():
    global speak
    global item
    while True:
        if speak ==True:
            output=gTTS(text=item, lang='en',slow=False)
            output.save('output.mp3')
            os.system('mpg123 output.mp3')
            speak=False
x=threading.Thread(target=sayItem, daemon=True)
x.start()

#Or, if you have a WEB cam, uncomment the next line
#(If it does not work, try setting to '1' instead of '0')
cam=cv2.VideoCapture('/dev/video1')
cam.set(cv2.CAP_PROP_FRAME_WIDTH,width)
cam.set(cv2.CAP_PROP_FRAME_HEIGHT,height)
net=jetson.inference.imageNet('googlenet')
font=cv2.FONT_HERSHEY_SIMPLEX
timeMark=time.time()
fpsFilter=0

while True:
    ret, frame = cam.read()
    img=cv2.cvtColor(frame,cv2.COLOR_BGR2RGBA).astype(np.float32)
    img=jetson.utils.cudaFromNumpy(img)
    if speak==False:
        classID, confidence = net.Classify(img,width,height)
        if confidence>=.5:
            item=net.GetClassDesc(classID)
            if item!=itemOld:
                speak=True
        if confidence<.5:
            item=''
        itemOld=item
    dt=time.time()-timeMark
    timeMark=time.time()
    fps=1/dt
    fpsFilter=.95*fpsFilter + .05 *fps
    cv2.putText(frame,str(round(fpsFilter,1))+'  fps  '+item+'   '+str(round(confidence,2)),(0,30),font,1,(0,0,255),2)
    cv2.imshow('nanoCam',frame)
    cv2.moveWindow('nanoCam',0,0)
    if cv2.waitKey(1)==ord('q'):
        break
cam.release()
cv2.destroyAllWindows()

import jetson.inference

import jetson.utils

import numpy as np

import time

import os

from gtts import gTTS

import threading

speak=True

item='Welcome to My Identify. Are you Ready to Rumble?'

confidence=0

itemOld=''

import cv2

print(cv2.__version__)

width=1280

height=720

flip=2

#Uncomment These next Two Line for Pi Camera

#camSet='nvarguscamerasrc ! video/x-raw(memory:NVMM), width=3264, height=2464, format=NV12, framerate=21/1 ! nvvidconv flip-method='+str(flip)+' ! video/x-raw, width='+str(dispW)+', height='+str(dispH)+', format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink'

#cam= cv2.VideoCapture(camSet)

def sayItem():

global speak

global item

while True:

if speak ==True:

output=gTTS(text=item, lang='en',slow=False)

output.save('output.mp3')

os.system('mpg123 output.mp3')

speak=False

x=threading.Thread(target=sayItem, daemon=True)

x.start()

#Or, if you have a WEB cam, uncomment the next line

#(If it does not work, try setting to '1' instead of '0')

cam=cv2.VideoCapture('/dev/video1')

cam.set(cv2.CAP_PROP_FRAME_WIDTH,width)

cam.set(cv2.CAP_PROP_FRAME_HEIGHT,height)

net=jetson.inference.imageNet('googlenet')

font=cv2.FONT_HERSHEY_SIMPLEX

timeMark=time.time()

fpsFilter=0

while True:

ret, frame = cam.read()

img=cv2.cvtColor(frame,cv2.COLOR_BGR2RGBA).astype(np.float32)

img=jetson.utils.cudaFromNumpy(img)

if speak==False:

classID, confidence = net.Classify(img,width,height)

if confidence>=.5:

item=net.GetClassDesc(classID)

if item!=itemOld:

speak=True

if confidence<.5:

item=''

itemOld=item

dt=time.time()-timeMark

timeMark=time.time()

fps=1/dt

fpsFilter=.95*fpsFilter + .05 *fps

cv2.putText(frame,str(round(fpsFilter,1))+' fps '+item+' '+str(round(confidence,2)),(0,30),font,1,(0,0,255),2)

cv2.imshow('nanoCam',frame)

cv2.moveWindow('nanoCam',0,0)

if cv2.waitKey(1)==ord('q'):

break

cam.release()

cv2.destroyAllWindows()

Technology Tutorials

Tag Archives: Speech Recognition

AI on the Edge LESSON 36: Select Active Camera in OpenCV With Voice Commands

What You Will Learn in This Lesson

Step-by-Step Breakdown of the Script

1. Setting Up the Background Voice Thread

2. Initializing Multiple Cameras

3. Processing and Routing Commands in the Main Loop

4. Managing Your OpenCV Windows

AI on the Edge LESSON 13: Control LED Brightness with Voice Commands on Raspberry Pi 5

AI on the Edge LESSON 11: Control LED on Raspberry Pi With Voice Commands

AI on the Jetson Nano LESSON 61: Image Recognition and Speech (TTS) on the Nano

Making The World a Better Place One High Tech Project at a Time. Enjoy!