Tag Archives: Pan/Tilt

AI on the Edge LESSON 30: Tune Object Tracker with Mouse Selected ROI

July 3, 2026 admin

AI on the Edge LESSON 30: Tune Object Tracker with Mouse Selected ROI

Welcome, Makers!

Well, hello there! It is absolutely fantastic to have you back. I’m Paul McWhorter, and today, we are taking a massive step forward in our AI on the Edge journey.

Up until now, we’ve been hard-coding our color thresholds (those pesky Lower Color and Upper Color values) to tell our camera what to look for. That’s fine for a science experiment, but it’s not exactly “smart,” is it? If the lighting changes, or if we want to track a different colored object, we have to go back into the code and manually edit those numbers.

Not anymore!

In today’s lesson, we are building a tool that lets us teach the AI. We’re going to use the mouse to draw a Region of Interest (ROI) right on our camera feed. The system will look at the pixels inside that box, calculate the average Hue, Saturation, and Value, and automatically set our tracking range for us.

This is the kind of professional-level functionality that turns a hobby project into a true, intelligent machine.

The Concept: From Hard-Coding to Dynamic Learning

The magic happens in our mouseAction function. Instead of just reading pixel values, we are now implementing a “click-and-drag” system:

Click and Hold: We capture the startX and startY coordinates.
Drag: We draw a rectangle in real-time so we can see exactly what area we are selecting.
Release: We take that specific slice of the image, convert it to the HSV color space, and use the cv2.mean() function to find the average color properties.
Auto-Tune: We set our LC (Lower Color) and UC (Upper Color) based on that average.

By doing this, the system learns what “object” we want to track on the fly. It’s elegant, it’s powerful, and it feels like real magic when you see those servos snap onto your target after a quick mouse drag.

What We’ve Accomplished

By the end of this lesson, you will have a system that:

Visually selects an object using the mouse.
Automatically calculates the optimal HSV thresholds for that specific object.
Updates the tracking behavior immediately without needing to stop or re-run the code.
Maintains that professional “Edge” feel, giving you real-time feedback on your FPS and mouse position data.

A Note on the “Edge”

Remember, we aren’t just running code; we are running on hardware. When we calculate the mean of the ROI, we are doing real image processing on the fly. You’ll notice the Composite and Mask windows updated immediately, giving you a visual confirmation that your “teacher” (you!) has successfully guided the “student” (the AI).

This is the power of working with OpenCV and the Raspberry Pi. You are building a system that observes, thinks, and reacts—all in real-time.

Get Ready to Build

Grab your Pi, make sure your servos are ready to go, and let’s get that camera calibrated. You’ve put in the work to get this far, and today is where all that effort starts to feel really rewarding.

I’m incredibly proud of how far you’ve come. Let’s dive in and start building!

Are you ready to see how accurately your Pi can “see” once you’ve given it the ability to learn from your selections?

In the video lesson we developed the following code.

# ====================================================================
# DISCLAIMER:
# This code is provided as-is for educational and experimental 
# purposes only. The author makes no representations or warranties of 
# any kind concerning the safety, suitability, or accuracy of this 
# code. Use at your own risk. The author assumes no liability for any 
# damages, system failures, security breaches, or network issues 
# resulting from the use or implementation of this script.
# ====================================================================
 
import cv2
import time
from picamera2 import Picamera2
from fusion_hat.pwm import PWM
from fusion_hat.servo import Servo
panPin = 2
tiltPin = 3
panServo = Servo(panPin)
tiltServo = Servo(tiltPin)
panAngle = 0
tiltAngle = 0
panServo.angle(panAngle)
tiltServo.angle(tiltAngle)

piCam = Picamera2()
W=1280
H=720
tStart = time.time()
fps = 0

xFrameC = int((W-1)/2)
yFrameC = int((H-1)/2)
xBoxC = xFrameC
yBoxC = yFrameC


redPin = 5
greenPin = 6
bluePin = 7
redLED = PWM(redPin)
greenLED = PWM(greenPin)
blueLED = PWM(bluePin)

RES = (W,H)
piCam.preview_configuration.main.size = RES
piCam.preview_configuration.main.format = "RGB888"
piCam.preview_configuration.controls.FrameRate=60
piCam.preview_configuration.align()
piCam.configure("preview")
piCam.start()

textLowerLeft = (int(W*.01),int(H*.06))
fontFace = cv2.FONT_HERSHEY_SIMPLEX
fontThickness = int(W/425)
fontScale = H*.0015
fontColor = (0,0,255)
xPos = 0
textLowerLeft1 = (int(W*.01),int(H*.06)*2)
textLowerLeft2 = (int(W*.01),int(H*.06)*3)
yPos = 0
valR = 0
valG = 0
valB = 0

Hue = 0
Sat = 0
Val = 0

LC = (25,100,90)
UC = (35,255,255)

frame = None
track = False
drawing = False
makeROI = False
boxColor = (255,255,0)
boxThick = 3
def mouseAction(event, x, y, flags, param):
    global frame, xPos, yPos, Hue, Sat, Val
    global track,startX,startY,endX,endY,makeROI,drawing
    if event == 0:
        xPos = x
        yPos = y
        if frame is not None:
            valB, valG, valR = frame[y,x]
            redLED.pulse_width_percent(int(valR/255*100))
            greenLED.pulse_width_percent(int(valG/255*100/2))
            blueLED.pulse_width_percent(int(valB/255*100/4))
            frameHSV = cv2.cvtColor(frame,cv2.COLOR_BGR2HSV)
            Hue, Sat, Val =frameHSV[y,x]
            if Hue == 60:
                track = True
            if Hue == 0:
                track = False
        if drawing == True:
            endX = x
            endY = y
    if event == 1:
        drawing = True
        makeROI = False
        startX = x
        startY = y
        endX = x
        endY = y 
    if event == 4:
        drawing = False
        makeROI = True
        endX = x
        endY = y                    
cv2.namedWindow('Camera',cv2.WINDOW_GUI_NORMAL)
cv2.moveWindow('Camera',0,65)
cv2.resizeWindow('Camera',W,H)

cv2.namedWindow('Mask',cv2.WINDOW_GUI_NORMAL)
cv2.moveWindow('Mask',W,65)
cv2.resizeWindow('Mask',int(W/2),int(H/2))

cv2.namedWindow('Composite',cv2.WINDOW_GUI_NORMAL)
cv2.moveWindow('Composite',W,65+int(H/2)+25)
cv2.resizeWindow('Composite',int(W/2),int(H/2))

cv2.setMouseCallback('Camera',mouseAction)

while True:
    deltaT = time.time() - tStart
    tStart=time.time()
    fps = fps*.95 + (1/deltaT)*.05
    frame= piCam.capture_array()
    frame=cv2.flip(frame,-1)
    
    if drawing == True:
        cv2.rectangle(frame,(startX,startY),(endX,endY),boxColor,boxThick)
    
    if makeROI == True:
        ROI = frame[startY:endY,startX:endX]
        ROIhsv = cv2.cvtColor(ROI,cv2.COLOR_BGR2HSV)
        AVGhsv = cv2.mean(ROIhsv)
        print(AVGhsv)
        LC = (AVGhsv[0]-15,AVGhsv[1]-50,AVGhsv[2]-50)
        UC = (AVGhsv[0]+15,255,255)
        cv2.imshow("ROI",ROI)
        cv2.moveWindow("ROI",W,65)
        makeROI = False

    
    cv2.circle(frame,(int(.9*W),int(.1*H)),20,(0,255,0),-1)
    cv2.circle(frame,(int(.95*W),int(.1*H)),20,(0,0,255),-1)    
    frameHSV = cv2.cvtColor(frame,cv2.COLOR_BGR2HSV)
    mask=cv2.inRange(frameHSV,LC,UC)
    composite = cv2.bitwise_and(frame, frame, mask=mask)
    
    contours, _ =cv2.findContours(mask,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
    if contours:
        #cv2.drawContours(frame,contours,-1,(255,0,0),3)
        largestContour = max(contours, key = cv2.contourArea)
        area = cv2.contourArea(largestContour)
        if area>100:
            #cv2.drawContours(frame,largestContour,-1,(255,0,0),3)
            x, y, w, h = cv2.boundingRect(largestContour)
            cv2.rectangle(frame, (x,y),(x+w,y+h),(0,255,255),3)
            xBoxC  =  x + int(w/2)
            yBoxC  =  y + int(h/2)
            
            xError = xBoxC - xFrameC
            yError = yBoxC - yFrameC
            
            cv2.circle(frame,(xFrameC,yFrameC),10,(255,0,0),-1)
            cv2.circle(frame,(xBoxC,yBoxC), 30,(0,0,255),3)
            if track == True:
                panAngle = panAngle - xError/50/2
                panServo.angle(int(panAngle))
                time.sleep(.02)
                tiltAngle=tiltAngle + yError/50/2
                tiltServo.angle(int(tiltAngle))
                time.sleep(.02)
                           
    myText = "FPS: "+str(round(fps,1))
    cv2.putText(frame,myText,textLowerLeft,fontFace,fontScale,fontColor,fontThickness)
    
    text1 = "Mouse Pos: "+str((xPos,yPos))
    text2 = "Pixel Color: "+str((Hue,Sat,Val))
    cv2.putText(frame,text1,textLowerLeft1,fontFace,fontScale,fontColor,fontThickness)    
    cv2.putText(frame,text2,textLowerLeft2,fontFace,fontScale,fontColor,fontThickness)    
    cv2.imshow("Camera", frame)
    cv2.imshow("Composite",composite)
    cv2.imshow("Mask",mask)

    if cv2.waitKey(1)==ord('q'):
        break
cv2.destroyAllWindows()
redLED.pulse_width_percent(0)
greenLED.pulse_width_percent(0)
blueLED.pulse_width_percent(0)
print('Program Terminated')

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

# ====================================================================

# DISCLAIMER:

# This code is provided as-is for educational and experimental

# purposes only. The author makes no representations or warranties of

# any kind concerning the safety, suitability, or accuracy of this

# code. Use at your own risk. The author assumes no liability for any

# damages, system failures, security breaches, or network issues

# resulting from the use or implementation of this script.

# ====================================================================

import cv2

import time

from picamera2 import Picamera2

from fusion_hat.pwm import PWM

from fusion_hat.servo import Servo

panPin = 2

tiltPin = 3

panServo = Servo(panPin)

tiltServo = Servo(tiltPin)

panAngle = 0

tiltAngle = 0

panServo.angle(panAngle)

tiltServo.angle(tiltAngle)

piCam = Picamera2()

W=1280

H=720

tStart = time.time()

fps = 0

xFrameC = int((W-1)/2)

yFrameC = int((H-1)/2)

xBoxC = xFrameC

yBoxC = yFrameC

redPin = 5

greenPin = 6

bluePin = 7

redLED = PWM(redPin)

greenLED = PWM(greenPin)

blueLED = PWM(bluePin)

RES = (W,H)

piCam.preview_configuration.main.size = RES

piCam.preview_configuration.main.format = "RGB888"

piCam.preview_configuration.controls.FrameRate=60

piCam.preview_configuration.align()

piCam.configure("preview")

piCam.start()

textLowerLeft = (int(W*.01),int(H*.06))

fontFace = cv2.FONT_HERSHEY_SIMPLEX

fontThickness = int(W/425)

fontScale = H*.0015

fontColor = (0,0,255)

xPos = 0

textLowerLeft1 = (int(W*.01),int(H*.06)*2)

textLowerLeft2 = (int(W*.01),int(H*.06)*3)

yPos = 0

valR = 0

valG = 0

valB = 0

Hue = 0

Sat = 0

Val = 0

LC = (25,100,90)

UC = (35,255,255)

frame = None

track = False

drawing = False

makeROI = False

boxColor = (255,255,0)

boxThick = 3

def mouseAction(event, x, y, flags, param):

global frame, xPos, yPos, Hue, Sat, Val

global track,startX,startY,endX,endY,makeROI,drawing

if event == 0:

xPos = x

yPos = y

if frame is not None:

valB, valG, valR = frame[y,x]

redLED.pulse_width_percent(int(valR/255*100))

greenLED.pulse_width_percent(int(valG/255*100/2))

blueLED.pulse_width_percent(int(valB/255*100/4))

frameHSV = cv2.cvtColor(frame,cv2.COLOR_BGR2HSV)

Hue, Sat, Val =frameHSV[y,x]

if Hue == 60:

track = True

if Hue == 0:

track = False

if drawing == True:

endX = x

endY = y

if event == 1:

drawing = True

makeROI = False

startX = x

startY = y

endX = x

endY = y

if event == 4:

drawing = False

makeROI = True

endX = x

endY = y

cv2.namedWindow('Camera',cv2.WINDOW_GUI_NORMAL)

cv2.moveWindow('Camera',0,65)

cv2.resizeWindow('Camera',W,H)

cv2.namedWindow('Mask',cv2.WINDOW_GUI_NORMAL)

cv2.moveWindow('Mask',W,65)

cv2.resizeWindow('Mask',int(W/2),int(H/2))

cv2.namedWindow('Composite',cv2.WINDOW_GUI_NORMAL)

cv2.moveWindow('Composite',W,65+int(H/2)+25)

cv2.resizeWindow('Composite',int(W/2),int(H/2))

cv2.setMouseCallback('Camera',mouseAction)

while True:

deltaT = time.time() - tStart

tStart=time.time()

fps = fps*.95 + (1/deltaT)*.05

frame= piCam.capture_array()

frame=cv2.flip(frame,-1)

if drawing == True:

cv2.rectangle(frame,(startX,startY),(endX,endY),boxColor,boxThick)

if makeROI == True:

ROI = frame[startY:endY,startX:endX]

ROIhsv = cv2.cvtColor(ROI,cv2.COLOR_BGR2HSV)

AVGhsv = cv2.mean(ROIhsv)

print(AVGhsv)

LC = (AVGhsv[0]-15,AVGhsv[1]-50,AVGhsv[2]-50)

UC = (AVGhsv[0]+15,255,255)

cv2.imshow("ROI",ROI)

cv2.moveWindow("ROI",W,65)

makeROI = False

cv2.circle(frame,(int(.9*W),int(.1*H)),20,(0,255,0),-1)

cv2.circle(frame,(int(.95*W),int(.1*H)),20,(0,0,255),-1)

frameHSV = cv2.cvtColor(frame,cv2.COLOR_BGR2HSV)

mask=cv2.inRange(frameHSV,LC,UC)

composite = cv2.bitwise_and(frame, frame, mask=mask)

contours, _ =cv2.findContours(mask,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)

if contours:

#cv2.drawContours(frame,contours,-1,(255,0,0),3)

largestContour = max(contours, key = cv2.contourArea)

area = cv2.contourArea(largestContour)

if area>100:

#cv2.drawContours(frame,largestContour,-1,(255,0,0),3)

x, y, w, h = cv2.boundingRect(largestContour)

cv2.rectangle(frame, (x,y),(x+w,y+h),(0,255,255),3)

xBoxC = x + int(w/2)

yBoxC = y + int(h/2)

xError = xBoxC - xFrameC

yError = yBoxC - yFrameC

cv2.circle(frame,(xFrameC,yFrameC),10,(255,0,0),-1)

cv2.circle(frame,(xBoxC,yBoxC), 30,(0,0,255),3)

if track == True:

panAngle = panAngle - xError/50/2

panServo.angle(int(panAngle))

time.sleep(.02)

tiltAngle=tiltAngle + yError/50/2

tiltServo.angle(int(tiltAngle))

time.sleep(.02)

myText = "FPS: "+str(round(fps,1))

cv2.putText(frame,myText,textLowerLeft,fontFace,fontScale,fontColor,fontThickness)

text1 = "Mouse Pos: "+str((xPos,yPos))

text2 = "Pixel Color: "+str((Hue,Sat,Val))

cv2.putText(frame,text1,textLowerLeft1,fontFace,fontScale,fontColor,fontThickness)

cv2.putText(frame,text2,textLowerLeft2,fontFace,fontScale,fontColor,fontThickness)

cv2.imshow("Camera", frame)

cv2.imshow("Composite",composite)

cv2.imshow("Mask",mask)

if cv2.waitKey(1)==ord('q'):

break

cv2.destroyAllWindows()

redLED.pulse_width_percent(0)

greenLED.pulse_width_percent(0)

blueLED.pulse_width_percent(0)

print('Program Terminated')

OpenCV, Python, Raspberry Pi

Raspberry Pi LESSON 59: Improved Pan/Tilt Tracking Control Algorithm

April 27, 2023 admin

In this Video Lesson we show an improved control algorithm for tracking an Object of Interest in OpenCV. We develop a simple example of Proportional control, where the correction signal is proportional to the error signal. We show this is a much improved algorithm over our earlier one, which simply applied 1 degree corrections independent of the size of the error. The code we develop in this lesson is included below for your convenience.

import cv2
from picamera2 import Picamera2
import time
import numpy as np
from servo import Servo
picam2 = Picamera2()

pan=Servo(pin=13)
tilt=Servo(pin=12)

panAngle=0
tiltAngle=0

pan.set_angle(panAngle)
tilt.set_angle(tiltAngle)

dispW=1280
dispH=720
picam2.preview_configuration.main.size = (dispW,dispH)
picam2.preview_configuration.main.format = "RGB888"
picam2.preview_configuration.controls.FrameRate=30
picam2.preview_configuration.align()
picam2.configure("preview")
picam2.start()
fps=0
pos=(30,60)
font=cv2.FONT_HERSHEY_SIMPLEX
height=1.5
weight=3
myColor=(0,0,255)
track=0

def onTrack1(val):
    global hueLow
    hueLow=val
    print('Hue Low',hueLow)
def onTrack2(val):
    global hueHigh
    hueHigh=val
    print('Hue High',hueHigh)
def onTrack3(val):
    global satLow
    satLow=val
    print('Sat Low',satLow)
def onTrack4(val):
    global satHigh
    satHigh=val
    print('Sat High',satHigh)
def onTrack5(val):
    global valLow
    valLow=val
    print('Val Low',valLow)
def onTrack6(val):
    global valHigh
    valHigh=val
    print('Val High',valHigh)
def onTrack7(val):
    global track
    track=val
    print('Track Value',track)

cv2.namedWindow('myTracker')

cv2.createTrackbar('Hue Low','myTracker',10,179,onTrack1)
cv2.createTrackbar('Hue High','myTracker',20,179,onTrack2)
cv2.createTrackbar('Sat Low','myTracker',100,255,onTrack3)
cv2.createTrackbar('Sat High','myTracker',255,255,onTrack4)
cv2.createTrackbar('Val Low','myTracker',100,255,onTrack5)
cv2.createTrackbar('Val High','myTracker',255,255,onTrack6)
cv2.createTrackbar('Train-0 Track-1','myTracker',0,1,onTrack7)


while True:
    tStart=time.time()
    frame= picam2.capture_array()
    frame=cv2.flip(frame,-1)
    frameHSV=cv2.cvtColor(frame,cv2.COLOR_BGR2HSV)
    cv2.putText(frame,str(int(fps))+' FPS',pos,font,height,myColor,weight)
    lowerBound=np.array([hueLow,satLow,valLow])
    upperBound=np.array([hueHigh,satHigh,valHigh])
    myMask=cv2.inRange(frameHSV,lowerBound,upperBound)
    myMaskSmall=cv2.resize(myMask,(int(dispW/2),int(dispH/2)))
    myObject=cv2.bitwise_and(frame,frame, mask=myMask)
    myObjectSmall=cv2.resize(myObject,(int(dispW/2),int(dispH/2)))
    
    contours,junk=cv2.findContours(myMask,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
    if len(contours)>0:
        contours=sorted(contours,key=lambda x:cv2.contourArea(x),reverse=True)
        #cv2.drawContours(frame,contours,-1,(255,0,0),3)
        contour=contours[0]
        x,y,w,h=cv2.boundingRect(contour)
        cv2.rectangle(frame,(x,y),(x+w,y+h),(0,0,255),3)
        if track==1:
            error=(x+w/2)-dispW/2
            panAngle=panAngle-error/75
            if panAngle>90:
                panAngle=90
            if panAngle<-90:
                panAngle=-90
            if abs(error)>35:
                pan.set_angle(panAngle)
            tiltError=(y+h/2)-dispH/2
            tiltAngle=tiltAngle+tiltError/75
            if tiltAngle>40:
                tiltAngle=40
            if tiltAngle<-90:
                tiltAngle=-90
            if abs(tiltError)>35:
                tilt.set_angle(tiltAngle)
        
    cv2.imshow('Camera',frame)
    cv2.imshow('Mask',myMaskSmall)
    cv2.imshow('My Object',myObjectSmall)
    if cv2.waitKey(1)==ord('q'):
        break
    tEnd=time.time()
    loopTime=tEnd-tStart
    fps=.9*fps + .1*(1/loopTime)
cv2.destroyAllWindows()

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

import cv2

from picamera2 import Picamera2

import time

import numpy as np

from servo import Servo

picam2 = Picamera2()

pan=Servo(pin=13)

tilt=Servo(pin=12)

panAngle=0

tiltAngle=0

pan.set_angle(panAngle)

tilt.set_angle(tiltAngle)

dispW=1280

dispH=720

picam2.preview_configuration.main.size = (dispW,dispH)

picam2.preview_configuration.main.format = "RGB888"

picam2.preview_configuration.controls.FrameRate=30

picam2.preview_configuration.align()

picam2.configure("preview")

picam2.start()

fps=0

pos=(30,60)

font=cv2.FONT_HERSHEY_SIMPLEX

height=1.5

weight=3

myColor=(0,0,255)

track=0

def onTrack1(val):

global hueLow

hueLow=val

print('Hue Low',hueLow)

def onTrack2(val):

global hueHigh

hueHigh=val

print('Hue High',hueHigh)

def onTrack3(val):

global satLow

satLow=val

print('Sat Low',satLow)

def onTrack4(val):

global satHigh

satHigh=val

print('Sat High',satHigh)

def onTrack5(val):

global valLow

valLow=val

print('Val Low',valLow)

def onTrack6(val):

global valHigh

valHigh=val

print('Val High',valHigh)

def onTrack7(val):

global track

track=val

print('Track Value',track)

cv2.namedWindow('myTracker')

cv2.createTrackbar('Hue Low','myTracker',10,179,onTrack1)

cv2.createTrackbar('Hue High','myTracker',20,179,onTrack2)

cv2.createTrackbar('Sat Low','myTracker',100,255,onTrack3)

cv2.createTrackbar('Sat High','myTracker',255,255,onTrack4)

cv2.createTrackbar('Val Low','myTracker',100,255,onTrack5)

cv2.createTrackbar('Val High','myTracker',255,255,onTrack6)

cv2.createTrackbar('Train-0 Track-1','myTracker',0,1,onTrack7)

while True:

tStart=time.time()

frame= picam2.capture_array()

frame=cv2.flip(frame,-1)

frameHSV=cv2.cvtColor(frame,cv2.COLOR_BGR2HSV)

cv2.putText(frame,str(int(fps))+' FPS',pos,font,height,myColor,weight)

lowerBound=np.array([hueLow,satLow,valLow])

upperBound=np.array([hueHigh,satHigh,valHigh])

myMask=cv2.inRange(frameHSV,lowerBound,upperBound)

myMaskSmall=cv2.resize(myMask,(int(dispW/2),int(dispH/2)))

myObject=cv2.bitwise_and(frame,frame, mask=myMask)

myObjectSmall=cv2.resize(myObject,(int(dispW/2),int(dispH/2)))

contours,junk=cv2.findContours(myMask,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)

if len(contours)>0:

contours=sorted(contours,key=lambda x:cv2.contourArea(x),reverse=True)

#cv2.drawContours(frame,contours,-1,(255,0,0),3)

contour=contours[0]

x,y,w,h=cv2.boundingRect(contour)

cv2.rectangle(frame,(x,y),(x+w,y+h),(0,0,255),3)

if track==1:

error=(x+w/2)-dispW/2

panAngle=panAngle-error/75

if panAngle>90:

panAngle=90

if panAngle<-90:

panAngle=-90

if abs(error)>35:

pan.set_angle(panAngle)

tiltError=(y+h/2)-dispH/2

tiltAngle=tiltAngle+tiltError/75

if tiltAngle>40:

tiltAngle=40

if tiltAngle<-90:

tiltAngle=-90

if abs(tiltError)>35:

tilt.set_angle(tiltAngle)

cv2.imshow('Camera',frame)

cv2.imshow('Mask',myMaskSmall)

cv2.imshow('My Object',myObjectSmall)

if cv2.waitKey(1)==ord('q'):

break

tEnd=time.time()

loopTime=tEnd-tStart

fps=.9*fps + .1*(1/loopTime)

cv2.destroyAllWindows()

OpenCV, Raspberry Pi

Raspberry Pi LESSON 58: Control System for Pan/Tilt Camera Hat for RPi Camera

April 20, 2023 admin

<span data-mce-type="bookmark" style="display: inline-block; width: 0px; overflow: hidden; line-height: 0;" class="mce_SELRES_start"></span>In this video lesson, we should a simple control algorithm for a pan tilt camera to track an Object of Interest in OpenCV. We train the device to recognize an Object of Interest based on color, and then the camera is adjusted to keep the object in the center of the frame as the device moves. For your convenience, the code developed in the lesson is included belos.

import cv2
from picamera2 import Picamera2
import time
import numpy as np
from servo import Servo
picam2 = Picamera2()

pan=Servo(pin=13)
tilt=Servo(pin=12)

panAngle=0
tiltAngle=0

pan.set_angle(panAngle)
tilt.set_angle(tiltAngle)

dispW=1280
dispH=720
picam2.preview_configuration.main.size = (dispW,dispH)
picam2.preview_configuration.main.format = "RGB888"
picam2.preview_configuration.controls.FrameRate=30
picam2.preview_configuration.align()
picam2.configure("preview")
picam2.start()
fps=0
pos=(30,60)
font=cv2.FONT_HERSHEY_SIMPLEX
height=1.5
weight=3
myColor=(0,0,255)
track=0

def onTrack1(val):
    global hueLow
    hueLow=val
    print('Hue Low',hueLow)
def onTrack2(val):
    global hueHigh
    hueHigh=val
    print('Hue High',hueHigh)
def onTrack3(val):
    global satLow
    satLow=val
    print('Sat Low',satLow)
def onTrack4(val):
    global satHigh
    satHigh=val
    print('Sat High',satHigh)
def onTrack5(val):
    global valLow
    valLow=val
    print('Val Low',valLow)
def onTrack6(val):
    global valHigh
    valHigh=val
    print('Val High',valHigh)
def onTrack7(val):
    global track
    track=val
    print('Track Value',track)

cv2.namedWindow('myTracker')

cv2.createTrackbar('Hue Low','myTracker',10,179,onTrack1)
cv2.createTrackbar('Hue High','myTracker',20,179,onTrack2)
cv2.createTrackbar('Sat Low','myTracker',100,255,onTrack3)
cv2.createTrackbar('Sat High','myTracker',255,255,onTrack4)
cv2.createTrackbar('Val Low','myTracker',100,255,onTrack5)
cv2.createTrackbar('Val High','myTracker',255,255,onTrack6)
cv2.createTrackbar('Train-0 Track-1','myTracker',0,1,onTrack7)


while True:
    tStart=time.time()
    frame= picam2.capture_array()
    frame=cv2.flip(frame,-1)
    frameHSV=cv2.cvtColor(frame,cv2.COLOR_BGR2HSV)
    cv2.putText(frame,str(int(fps))+' FPS',pos,font,height,myColor,weight)
    lowerBound=np.array([hueLow,satLow,valLow])
    upperBound=np.array([hueHigh,satHigh,valHigh])
    myMask=cv2.inRange(frameHSV,lowerBound,upperBound)
    myMaskSmall=cv2.resize(myMask,(int(dispW/2),int(dispH/2)))
    myObject=cv2.bitwise_and(frame,frame, mask=myMask)
    myObjectSmall=cv2.resize(myObject,(int(dispW/2),int(dispH/2)))
    
    contours,junk=cv2.findContours(myMask,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
    if len(contours)>0:
        contours=sorted(contours,key=lambda x:cv2.contourArea(x),reverse=True)
        #cv2.drawContours(frame,contours,-1,(255,0,0),3)
        contour=contours[0]
        x,y,w,h=cv2.boundingRect(contour)
        cv2.rectangle(frame,(x,y),(x+w,y+h),(0,0,255),3)
        if track==1:
            error=(x+w/2)-dispW/2
            if error>35:
                panAngle=panAngle-1
                if panAngle<-90:
                    panAngle=-90
                pan.set_angle(panAngle)
            if error<-35:
                panAngle=panAngle+1
                if panAngle>90:
                    panAngle=90
                pan.set_angle(panAngle)
            tiltError=(y+h/2)-dispH/2
            if tiltError>35:
                tiltAngle=tiltAngle+1
                if tiltAngle>40:
                    tiltAngle=40
                tilt.set_angle(tiltAngle)
            if tiltError<-35:
                tiltAngle=tiltAngle-1
                if tiltAngle<-90:
                    tiltAngle=-90
                tilt.set_angle(tiltAngle)
        
    cv2.imshow('Camera',frame)
    cv2.imshow('Mask',myMaskSmall)
    cv2.imshow('My Object',myObjectSmall)
    if cv2.waitKey(1)==ord('q'):
        break
    tEnd=time.time()
    loopTime=tEnd-tStart
    fps=.9*fps + .1*(1/loopTime)
cv2.destroyAllWindows()

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

import cv2

from picamera2 import Picamera2

import time

import numpy as np

from servo import Servo

picam2 = Picamera2()

pan=Servo(pin=13)

tilt=Servo(pin=12)

panAngle=0

tiltAngle=0

pan.set_angle(panAngle)

tilt.set_angle(tiltAngle)

dispW=1280

dispH=720

picam2.preview_configuration.main.size = (dispW,dispH)

picam2.preview_configuration.main.format = "RGB888"

picam2.preview_configuration.controls.FrameRate=30

picam2.preview_configuration.align()

picam2.configure("preview")

picam2.start()

fps=0

pos=(30,60)

font=cv2.FONT_HERSHEY_SIMPLEX

height=1.5

weight=3

myColor=(0,0,255)

track=0

def onTrack1(val):

global hueLow

hueLow=val

print('Hue Low',hueLow)

def onTrack2(val):

global hueHigh

hueHigh=val

print('Hue High',hueHigh)

def onTrack3(val):

global satLow

satLow=val

print('Sat Low',satLow)

def onTrack4(val):

global satHigh

satHigh=val

print('Sat High',satHigh)

def onTrack5(val):

global valLow

valLow=val

print('Val Low',valLow)

def onTrack6(val):

global valHigh

valHigh=val

print('Val High',valHigh)

def onTrack7(val):

global track

track=val

print('Track Value',track)

cv2.namedWindow('myTracker')

cv2.createTrackbar('Hue Low','myTracker',10,179,onTrack1)

cv2.createTrackbar('Hue High','myTracker',20,179,onTrack2)

cv2.createTrackbar('Sat Low','myTracker',100,255,onTrack3)

cv2.createTrackbar('Sat High','myTracker',255,255,onTrack4)

cv2.createTrackbar('Val Low','myTracker',100,255,onTrack5)

cv2.createTrackbar('Val High','myTracker',255,255,onTrack6)

cv2.createTrackbar('Train-0 Track-1','myTracker',0,1,onTrack7)

while True:

tStart=time.time()

frame= picam2.capture_array()

frame=cv2.flip(frame,-1)

frameHSV=cv2.cvtColor(frame,cv2.COLOR_BGR2HSV)

cv2.putText(frame,str(int(fps))+' FPS',pos,font,height,myColor,weight)

lowerBound=np.array([hueLow,satLow,valLow])

upperBound=np.array([hueHigh,satHigh,valHigh])

myMask=cv2.inRange(frameHSV,lowerBound,upperBound)

myMaskSmall=cv2.resize(myMask,(int(dispW/2),int(dispH/2)))

myObject=cv2.bitwise_and(frame,frame, mask=myMask)

myObjectSmall=cv2.resize(myObject,(int(dispW/2),int(dispH/2)))

contours,junk=cv2.findContours(myMask,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)

if len(contours)>0:

contours=sorted(contours,key=lambda x:cv2.contourArea(x),reverse=True)

#cv2.drawContours(frame,contours,-1,(255,0,0),3)

contour=contours[0]

x,y,w,h=cv2.boundingRect(contour)

cv2.rectangle(frame,(x,y),(x+w,y+h),(0,0,255),3)

if track==1:

error=(x+w/2)-dispW/2

if error>35:

panAngle=panAngle-1

if panAngle<-90:

panAngle=-90

pan.set_angle(panAngle)

if error<-35:

panAngle=panAngle+1

if panAngle>90:

panAngle=90

pan.set_angle(panAngle)

tiltError=(y+h/2)-dispH/2

if tiltError>35:

tiltAngle=tiltAngle+1

if tiltAngle>40:

tiltAngle=40

tilt.set_angle(tiltAngle)

if tiltError<-35:

tiltAngle=tiltAngle-1

if tiltAngle<-90:

tiltAngle=-90

tilt.set_angle(tiltAngle)

cv2.imshow('Camera',frame)

cv2.imshow('Mask',myMaskSmall)

cv2.imshow('My Object',myObjectSmall)

if cv2.waitKey(1)==ord('q'):

break

tEnd=time.time()

loopTime=tEnd-tStart

fps=.9*fps + .1*(1/loopTime)

cv2.destroyAllWindows()

Jetson Xavier NX

Jetson Xavier NX Lesson 9: Tracking Objects with Two Cameras in OpenCV

July 15, 2020 admin

In this lesson we use OpenCV to track an object of interest based on HSV color value. We are running two cameras in parallel, and track the same object in both cameras. The code below is provided for your convenience, and is the code developed in the video lesson. If you want to play along at home, you can get your NVIDIA Jetson Xavier NX HERE.

In this lesson, we are running two raspberry pi cameras. I like the following ones and have verified that they work on the Xavier NX. You can pick your cameras up HERE.

import cv2
import numpy as np
import time
#from adafruit_servokit import ServoKit
print(cv2.__version__)
timeMark=time.time()
dtFIL=0

def nothing(x):
    pass

cv2.namedWindow('TrackBars')
cv2.moveWindow('TrackBars',1320,0)
cv2.createTrackbar('hueLower', 'TrackBars',100,179,nothing)
cv2.createTrackbar('hueUpper', 'TrackBars',116,179,nothing)
cv2.createTrackbar('satLow', 'TrackBars',160,255,nothing)
cv2.createTrackbar('satHigh', 'TrackBars',255,255,nothing)
cv2.createTrackbar('valLow', 'TrackBars',150,255,nothing)
cv2.createTrackbar('valHigh', 'TrackBars',255,255,nothing)
#kit=ServoKit(channels=16)

# tilt=90
# pan=90
# dTilt=10
# dPan=1

# kit.servo[0].angle=pan
# kit.servo[1].angle=tilt
# kit.servo[2].angle=pan
# kit.servo[3].angle=tilt

width=720
height=480
flip=2
font=cv2.FONT_HERSHEY_SIMPLEX
camSet1='nvarguscamerasrc sensor-id=0 ee-mode=1 ee-strength=0 tnr-mode=2 tnr-strength=1 wbmode=3 ! video/x-raw(memory:NVMM), width=3264, height=2464, framerate=21/1,format=NV12 ! nvvidconv flip-method='+str(flip)+' ! video/x-raw, width='+str(width)+', height='+str(height)+', format=BGRx ! videoconvert ! video/x-raw, format=BGR ! videobalance contrast=1.3 brightness=-.2 saturation=1.2 ! appsink drop=True'
camSet2='nvarguscamerasrc sensor-id=1 ee-mode=1 ee-strength=0 tnr-mode=2 tnr-strength=1 wbmode=3 ! video/x-raw(memory:NVMM), width=3264, height=2464, framerate=21/1,format=NV12 ! nvvidconv flip-method='+str(flip)+' ! video/x-raw, width='+str(width)+', height='+str(height)+', format=BGRx ! videoconvert ! video/x-raw, format=BGR ! videobalance contrast=1.3 brightness=-.2 saturation=1.2 ! appsink drop=True'

#camSet='nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM), width=3264, height=2464, framerate=21/1,format=NV12 ! nvvidconv flip-method='+str(flip)+' ! video/x-raw, width='+str(width)+', height='+str(height)+', format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink'
#camSet ='v4l2src device=/dev/video1 ! video/x-raw,width='+str(width)+',height='+str(height)+',framerate=20/1 ! videoconvert ! appsink'
cam1=cv2.VideoCapture(camSet1)
cam2=cv2.VideoCapture(camSet2)
while True:
    _, frame1 = cam1.read()
    _, frame2 = cam2.read()

    hsv1=cv2.cvtColor(frame1,cv2.COLOR_BGR2HSV)
    hsv2=cv2.cvtColor(frame2,cv2.COLOR_BGR2HSV)

    hueLow=cv2.getTrackbarPos('hueLower', 'TrackBars')
    hueUp=cv2.getTrackbarPos('hueUpper', 'TrackBars')

    Ls=cv2.getTrackbarPos('satLow', 'TrackBars')
    Us=cv2.getTrackbarPos('satHigh', 'TrackBars')

    Lv=cv2.getTrackbarPos('valLow', 'TrackBars')
    Uv=cv2.getTrackbarPos('valHigh', 'TrackBars')

    l_b=np.array([hueLow,Ls,Lv])
    u_b=np.array([hueUp,Us,Uv])

    FGmask1=cv2.inRange(hsv1,l_b,u_b)
    FGmask2=cv2.inRange(hsv2,l_b,u_b)

    cv2.imshow('FGmask1',FGmask1)
    cv2.moveWindow('FGmask1',0,0)

    contours1,_ = cv2.findContours(FGmask1,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
    contours1=sorted(contours1,key=lambda x:cv2.contourArea(x),reverse=True)
    for cnt in contours1:
        area=cv2.contourArea(cnt)
        (x,y,w,h)=cv2.boundingRect(cnt)
        if area>=100:
            cv2.rectangle(frame1,(x,y),(x+w,y+h),(0,255,255),3)
            break
    
    contours2,_ = cv2.findContours(FGmask2,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
    contours2=sorted(contours2,key=lambda x:cv2.contourArea(x),reverse=True)
    for cnt in contours2:
        area=cv2.contourArea(cnt)
        (x,y,w,h)=cv2.boundingRect(cnt)
        if area>=100:
            cv2.rectangle(frame2,(x,y),(x+w,y+h),(0,255,255),3)
            break

    frame3=np.hstack((frame1,frame2))
    dt=time.time()-timeMark
    timeMark=time.time()
    dtFIL=.9*dtFIL + .1*dt
    fps=1/dtFIL
    cv2.rectangle(frame3,(0,0),(150,40),(0,0,255),-1)
    cv2.putText(frame3,'fps: '+str(round(fps,1)),(0,30),font,1,(0,255,255),2)

    #cv2.imshow('myCam1',frame1)
    #cv2.imshow('myCam2',frame2)
    cv2.imshow('comboCam',frame3)
    cv2.moveWindow('comboCam',0,450)
    # kit.servo[0].angle=pan
    # kit.servo[2].angle=pan
    # pan=pan+dPan
    # if pan>=179 or pan<=1:
    #     dPan=dPan*(-1)
    #     tilt=tilt+dTilt
    #     kit.servo[1].angle=tilt
    #     kit.servo[3].angle=tilt
    #     if tilt>=169 or tilt<=11:
    #         dTilt=dTilt*(-1)
    if cv2.waitKey(1)==ord('q'):
        break
cam1.release()
cam2.release()
cv2.destroyAllWindows()

100

101

102

103

104

105

106

107

108

109

110

111

112

import cv2

import numpy as np

import time

#from adafruit_servokit import ServoKit

print(cv2.__version__)

timeMark=time.time()

dtFIL=0

def nothing(x):

pass

cv2.namedWindow('TrackBars')

cv2.moveWindow('TrackBars',1320,0)

cv2.createTrackbar('hueLower', 'TrackBars',100,179,nothing)

cv2.createTrackbar('hueUpper', 'TrackBars',116,179,nothing)

cv2.createTrackbar('satLow', 'TrackBars',160,255,nothing)

cv2.createTrackbar('satHigh', 'TrackBars',255,255,nothing)

cv2.createTrackbar('valLow', 'TrackBars',150,255,nothing)

cv2.createTrackbar('valHigh', 'TrackBars',255,255,nothing)

#kit=ServoKit(channels=16)

# tilt=90

# pan=90

# dTilt=10

# dPan=1

# kit.servo[0].angle=pan

# kit.servo[1].angle=tilt

# kit.servo[2].angle=pan

# kit.servo[3].angle=tilt

width=720

height=480

flip=2

font=cv2.FONT_HERSHEY_SIMPLEX

camSet1='nvarguscamerasrc sensor-id=0 ee-mode=1 ee-strength=0 tnr-mode=2 tnr-strength=1 wbmode=3 ! video/x-raw(memory:NVMM), width=3264, height=2464, framerate=21/1,format=NV12 ! nvvidconv flip-method='+str(flip)+' ! video/x-raw, width='+str(width)+', height='+str(height)+', format=BGRx ! videoconvert ! video/x-raw, format=BGR ! videobalance contrast=1.3 brightness=-.2 saturation=1.2 ! appsink drop=True'

camSet2='nvarguscamerasrc sensor-id=1 ee-mode=1 ee-strength=0 tnr-mode=2 tnr-strength=1 wbmode=3 ! video/x-raw(memory:NVMM), width=3264, height=2464, framerate=21/1,format=NV12 ! nvvidconv flip-method='+str(flip)+' ! video/x-raw, width='+str(width)+', height='+str(height)+', format=BGRx ! videoconvert ! video/x-raw, format=BGR ! videobalance contrast=1.3 brightness=-.2 saturation=1.2 ! appsink drop=True'

#camSet='nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM), width=3264, height=2464, framerate=21/1,format=NV12 ! nvvidconv flip-method='+str(flip)+' ! video/x-raw, width='+str(width)+', height='+str(height)+', format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink'

#camSet ='v4l2src device=/dev/video1 ! video/x-raw,width='+str(width)+',height='+str(height)+',framerate=20/1 ! videoconvert ! appsink'

cam1=cv2.VideoCapture(camSet1)

cam2=cv2.VideoCapture(camSet2)

while True:

_, frame1 = cam1.read()

_, frame2 = cam2.read()

hsv1=cv2.cvtColor(frame1,cv2.COLOR_BGR2HSV)

hsv2=cv2.cvtColor(frame2,cv2.COLOR_BGR2HSV)

hueLow=cv2.getTrackbarPos('hueLower', 'TrackBars')

hueUp=cv2.getTrackbarPos('hueUpper', 'TrackBars')

Ls=cv2.getTrackbarPos('satLow', 'TrackBars')

Us=cv2.getTrackbarPos('satHigh', 'TrackBars')

Lv=cv2.getTrackbarPos('valLow', 'TrackBars')

Uv=cv2.getTrackbarPos('valHigh', 'TrackBars')

l_b=np.array([hueLow,Ls,Lv])

u_b=np.array([hueUp,Us,Uv])

FGmask1=cv2.inRange(hsv1,l_b,u_b)

FGmask2=cv2.inRange(hsv2,l_b,u_b)

cv2.imshow('FGmask1',FGmask1)

cv2.moveWindow('FGmask1',0,0)

contours1,_ = cv2.findContours(FGmask1,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)

contours1=sorted(contours1,key=lambda x:cv2.contourArea(x),reverse=True)

for cnt in contours1:

area=cv2.contourArea(cnt)

(x,y,w,h)=cv2.boundingRect(cnt)

if area>=100:

cv2.rectangle(frame1,(x,y),(x+w,y+h),(0,255,255),3)

break

contours2,_ = cv2.findContours(FGmask2,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)

contours2=sorted(contours2,key=lambda x:cv2.contourArea(x),reverse=True)

for cnt in contours2:

area=cv2.contourArea(cnt)

(x,y,w,h)=cv2.boundingRect(cnt)

if area>=100:

cv2.rectangle(frame2,(x,y),(x+w,y+h),(0,255,255),3)

break

frame3=np.hstack((frame1,frame2))

dt=time.time()-timeMark

timeMark=time.time()

dtFIL=.9*dtFIL + .1*dt

fps=1/dtFIL

cv2.rectangle(frame3,(0,0),(150,40),(0,0,255),-1)

cv2.putText(frame3,'fps: '+str(round(fps,1)),(0,30),font,1,(0,255,255),2)

#cv2.imshow('myCam1',frame1)

#cv2.imshow('myCam2',frame2)

cv2.imshow('comboCam',frame3)

cv2.moveWindow('comboCam',0,450)

# kit.servo[0].angle=pan

# kit.servo[2].angle=pan

# pan=pan+dPan

# if pan>=179 or pan<=1:

# dPan=dPan*(-1)

# tilt=tilt+dTilt

# kit.servo[1].angle=tilt

# kit.servo[3].angle=tilt

# if tilt>=169 or tilt<=11:

# dTilt=dTilt*(-1)

if cv2.waitKey(1)==ord('q'):

break

cam1.release()

cam2.release()

cv2.destroyAllWindows()

Jetson Xavier NX

Jetson Xavier NX Lesson 8: Controlling Dual Pan/Tilt Raspberry Pi Cameras

July 8, 2020 admin

<span data-mce-type="bookmark" style="display: inline-block; width: 0px; overflow: hidden; line-height: 0;" class="mce_SELRES_start"></span>
In this lesson we show how to independently control two Raspberry Pi Cameras using servo controlled pan/tilt brackets. This work will serve as the foundation for allowing us to create cameras that scan a room and locate objects of interest.

In this lesson, I am using two pan/tilt camera mounts. You can get the gear I am using on amazon HERE. I suggest purchasing two units.

Then, we also need two Version two raspberry pi cameras. I like the following ones, because they include a neat little acrylic case, and the long cable, which makes it work much better on the pan/tilt bracket. You can get the cameras HERE.

If you do not have a Jetson Xavier NX yet, you can pick up the gear I am using below:

First, you will need the Jetson Xavier NX, which you can get HERE:
You will want a quality, large SD card, I have very good luck with this one HERE:
You will need a camera. I have found that the Jetson Xavier NX works very well with most Logitech Webcams, but these cameras are a little hard to find right now. I suggest the best option if you do not have a logitech webcam is to get the Raspberry Pi Version 2 camera, which works very well. You can pick the camera up HERE.
It is optional, but I have found that it is nice to have an extra, longer cable for the Raspberry Pi camera, which is available HERE. Also, a small case/stand for the camera is nice and you can get the one I use HERE.
The Jetson Nano has a slot for a SSD drive. I really like having the SSD drive attached, and makes it much easier to keep your work backed up. The projects in these lessons will work fine with just the SD card, but if you like, the SSD drive makes life easier (note even with SSD drive, you will still need the SD card above). You can get the SSD drive I am using HERE.
You can use USB keyboard and mouse, but I like to preserve my USB slots for other things, so like using a wireless keyboard and mouse. This is optional, but I have found these work well on the Jetson Xavier NX, and you can get what I am using HERE.
You will need an HDMI cable and monitor, which you probably already have.

import cv2
import numpy as np
import time
from adafruit_servokit import ServoKit
print(cv2.__version__)
timeMark=time.time()
dtFIL=0

kit=ServoKit(channels=16)

tilt=90
pan=90
dTilt=10
dPan=1

kit.servo[0].angle=pan
kit.servo[1].angle=tilt
kit.servo[2].angle=pan
kit.servo[3].angle=tilt

width=720
height=480
flip=2
font=cv2.FONT_HERSHEY_SIMPLEX
camSet1='nvarguscamerasrc sensor-id=0 ee-mode=1 ee-strength=0 tnr-mode=2 tnr-strength=1 wbmode=3 ! video/x-raw(memory:NVMM), width=3264, height=2464, framerate=21/1,format=NV12 ! nvvidconv flip-method='+str(flip)+' ! video/x-raw, width='+str(width)+', height='+str(height)+', format=BGRx ! videoconvert ! video/x-raw, format=BGR ! videobalance contrast=1.3 brightness=-.2 saturation=1.2 ! appsink drop=True'
camSet2='nvarguscamerasrc sensor-id=1 ee-mode=1 ee-strength=0 tnr-mode=2 tnr-strength=1 wbmode=3 ! video/x-raw(memory:NVMM), width=3264, height=2464, framerate=21/1,format=NV12 ! nvvidconv flip-method='+str(flip)+' ! video/x-raw, width='+str(width)+', height='+str(height)+', format=BGRx ! videoconvert ! video/x-raw, format=BGR ! videobalance contrast=1.3 brightness=-.2 saturation=1.2 ! appsink drop=True'

#camSet='nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM), width=3264, height=2464, framerate=21/1,format=NV12 ! nvvidconv flip-method='+str(flip)+' ! video/x-raw, width='+str(width)+', height='+str(height)+', format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink'
#camSet ='v4l2src device=/dev/video1 ! video/x-raw,width='+str(width)+',height='+str(height)+',framerate=20/1 ! videoconvert ! appsink'
cam1=cv2.VideoCapture(camSet1)
cam2=cv2.VideoCapture(camSet2)
while True:
    _, frame1 = cam1.read()
    _, frame2 = cam2.read()
    frame3=np.hstack((frame1,frame2))
    dt=time.time()-timeMark
    timeMark=time.time()
    dtFIL=.9*dtFIL + .1*dt
    fps=1/dtFIL
    cv2.rectangle(frame3,(0,0),(150,40),(0,0,255),-1)
    cv2.putText(frame3,'fps: '+str(round(fps,1)),(0,30),font,1,(0,255,255),2)
    print('fps: ',fps)
    #cv2.imshow('myCam1',frame1)
    #cv2.imshow('myCam2',frame2)
    cv2.imshow('comboCam',frame3)
    cv2.moveWindow('comboCam',0,450)
    kit.servo[0].angle=pan
    kit.servo[2].angle=pan
    pan=pan+dPan
    if pan>=179 or pan<=1:
        dPan=dPan*(-1)
        tilt=tilt+dTilt
        kit.servo[1].angle=tilt
        kit.servo[3].angle=tilt
        if tilt>=169 or tilt<=11:
            dTilt=dTilt*(-1)
    if cv2.waitKey(1)==ord('q'):
        break
cam.release()
cv2.destroyAllWindows()

import cv2

import numpy as np

import time

from adafruit_servokit import ServoKit

print(cv2.__version__)

timeMark=time.time()

dtFIL=0

kit=ServoKit(channels=16)

tilt=90

pan=90

dTilt=10

dPan=1

kit.servo[0].angle=pan

kit.servo[1].angle=tilt

kit.servo[2].angle=pan

kit.servo[3].angle=tilt

width=720

height=480

flip=2

font=cv2.FONT_HERSHEY_SIMPLEX

#camSet ='v4l2src device=/dev/video1 ! video/x-raw,width='+str(width)+',height='+str(height)+',framerate=20/1 ! videoconvert ! appsink'

cam1=cv2.VideoCapture(camSet1)

cam2=cv2.VideoCapture(camSet2)

while True:

_, frame1 = cam1.read()

_, frame2 = cam2.read()

frame3=np.hstack((frame1,frame2))

dt=time.time()-timeMark

timeMark=time.time()

dtFIL=.9*dtFIL + .1*dt

fps=1/dtFIL

cv2.rectangle(frame3,(0,0),(150,40),(0,0,255),-1)

cv2.putText(frame3,'fps: '+str(round(fps,1)),(0,30),font,1,(0,255,255),2)

print('fps: ',fps)

#cv2.imshow('myCam1',frame1)

#cv2.imshow('myCam2',frame2)

cv2.imshow('comboCam',frame3)

cv2.moveWindow('comboCam',0,450)

kit.servo[0].angle=pan

kit.servo[2].angle=pan

pan=pan+dPan

if pan>=179 or pan<=1:

dPan=dPan*(-1)

tilt=tilt+dTilt

kit.servo[1].angle=tilt

kit.servo[3].angle=tilt

if tilt>=169 or tilt<=11:

dTilt=dTilt*(-1)

if cv2.waitKey(1)==ord('q'):

break

cam.release()

cv2.destroyAllWindows()

Technology Tutorials

Tag Archives: Pan/Tilt

AI on the Edge LESSON 30: Tune Object Tracker with Mouse Selected ROI

AI on the Edge LESSON 30: Tune Object Tracker with Mouse Selected ROI

Welcome, Makers!

The Concept: From Hard-Coding to Dynamic Learning

What We’ve Accomplished

A Note on the “Edge”

Get Ready to Build

Raspberry Pi LESSON 59: Improved Pan/Tilt Tracking Control Algorithm

Raspberry Pi LESSON 58: Control System for Pan/Tilt Camera Hat for RPi Camera

Jetson Xavier NX Lesson 9: Tracking Objects with Two Cameras in OpenCV

Jetson Xavier NX Lesson 8: Controlling Dual Pan/Tilt Raspberry Pi Cameras

Making The World a Better Place One High Tech Project at a Time. Enjoy!