Introduction to image processing.
Vision provides a rich information about the immediate environment around the robot. However it requires to process the images to extract pertinent information like the objects in the scene.
The goal here is to detect the specific colored objects that are positioned in the scene along the path of your robot in order to count and place them in the map. A first approache coub be to use color analysis techniques to detect the objects. If objects are textured, more sophisticated techniques could be used. This tutorial gives you some tips to help you with this detection task.
Setup our programming environment
This tutorial focus on OpenCV librairy, addressed in Python development language on a Linux machine.
First we want to be sure that Python3 and some libraries (Numpy, Matplot, Sklearn, Scipy, OpenCV etc.) are installed.
The command whereis permit to localize a command (like python3 interpreter). If python3 is correctly installed, its execution would not be empty:
whereis python3
Python uses its own package managers pip to install libraries. So just to be sure you can:
sudo apt install python3 pip
Then we can use pip to install required modules:
pip3 install numpy==1.26.4 opencv-python opencv-contrib-python scikit-learn scipy matplotlib psutil scikit-image
Now normally you can load the different modules, for instance:
import cv2
import numpy as np
import os
from sklearn.svm import LinearSVC
from scipy.cluster.vq import *
from sklearn.preprocessing import StandardScaler
from sklearn import preprocessing
Image acquisition
Image from simulation
It is possible to access the camera's video streams via the Realsense sensor (if it is available) or simulation.
Gazebo is capable of simulating robot vision (classical and 3D sensor).
Launch a simulation :
'''bash roslaunch larm chalenge-2.launch '''
Observe the topic and mainly the one published the images (camera/rgb/image_raw) with rostopic list, rostopic info and rostopic hz.
Image are published as 'sensor_msgs/Image' message type in camera/rgb/image_raw
If you echo the messages of the image topic (rostopic echo camera/rgb/image_raw), you can observe the following structure of the message
std_msgs/Header header
uint32 height
uint32 width
string encoding
uint8 is_bigendian
uint32 step
uint8[] data
The pixel values are stored in a uint8 array. Several tools are availbale to convert a ROS image to OpenCV image (for instance cv_bridge)
Image from Realsense camera sensor
The following code is able to connect the Realsense camera and to acquire the depth and the color stream. A configuration step is done. Both depth and color images are aligned to make that all pixels of a depth image correpond to all pixels of the corresponding color image.
import pyrealsense2 as rs
import numpy as np
import math
import cv2,time,sys
pipeline = rs.pipeline()
config = rs.config()
colorizer = rs.colorizer()
# Configuration of depth and color image size (840x480), the format of the pixel values and the frame rates (30 fps)
config.enable_stream(rs.stream.depth, 840, 480, rs.format.z16, 30)
config.enable_stream(rs.stream.color, 840, 480, rs.format.bgr8, 30)
pipeline.start(config)
align_to = rs.stream.depth
align = rs.align(align_to)
rayon=10
count=1
refTime= time.process_time()
freq= 60
try:
while True:
# This call waits until a new coherent set of frames is available on a device
frames = pipeline.wait_for_frames()
# Aligning color frame to depth frame
aligned_frames = align.process(frames)
depth_frame = aligned_frames.get_depth_frame()
aligned_color_frame = aligned_frames.get_color_frame()
if not depth_frame or not aligned_color_frame: continue
# Two ways to colorized the depth map
# first : using colorizer of pyrealsense
colorized_depth = colorizer.colorize(depth_frame)
depth_colormap = np.asanyarray(colorized_depth.get_data())
# Get the intrinsic parameters
color_intrin = aligned_color_frame.profile.as_video_stream_profile().intrinsics
color_image = np.asanyarray(aligned_color_frame.get_data())
depth_colormap_dim = depth_colormap.shape
color_colormap_dim = color_image.shape
# Show images
images = np.hstack((color_image, depth_colormap)) # supose that depth_colormap_dim == color_colormap_dim (840x480 for example) otherwize: resized_color_image = cv2.resize(color_image, dsize=(depth_colormap_dim[1], depth_colormap_dim[0]), interpolation=cv2.INTER_AREA)
# Show images
cv2.namedWindow('RealSense', cv2.WINDOW_NORMAL)
# Resize the Window
cv2.resizeWindow('RealSense', 960, 720)
cv2.imshow('RealSense', images)
cv2.waitKey(1)
# Frequency:
if count == 10 :
newTime= time.process_time()
freq= 10/((newTime-refTime))
refTime= newTime
count= 0
count+= 1
except Exception as e:
print(e)
pass
finally:
pipeline.stop()
You have already done the code to send both depth and color image on a ROS topic (camera_driver section). During the challenges you will be able to compute both streams via the ROS image topics or directly after image acquisition (pipeline.wait_for_frames() in the previous code). yYu will have to pay close attention to the time required to ensure the execution of the processing chains of the two video streams in order to guarantee rapid detection.
Basic codes using OpenCV framework
This section introduces some of the functions and techniques illustrated by some codes that you can use to complete the proposed challenges.
First, you will test each proposal on images downloaded from the Internet or from the Vision section of the tutorial. You will then have to integrate or adapt them to process images from the Realsense camera on board the robot. Some codes example propose to acquire and compute the webcam video stream.
Segmentation d'images couleur par seuillage des composantes et Gestion de la souris
Here are a few lines of code to extract a region of interest using the mouse. With these few lines, you can calculate the mean and variance of each component of the image, which is useful for the segmentation stage. In this example, we first acquire an image from the laptop's webcam. We then use the mouse to define a section of the captured image. You can then easily calculate the desired statistical metrics for this section. The mean and variance define a Gaussian model for each component of the clipping.
import cv2
import numpy as np
# connect to a sensor (0: webcam)
cap=cv2.VideoCapture(0)
# capture an image
ret, frame=cap.read()
# Select ROI
r = cv2.selectROI(frame)
# Crop image
imCrop = frame[int(r[1]):int(r[1]+r[3]), int(r[0]):int(r[0]+r[2])]
average_h = np.mean(imCrop[:,:,0])
average_s = np.mean(imCrop[:,:,1])
average_v = np.mean(imCrop[:,:,2])
print(average_h,average_s,average_v)
# Display cropped image
cv2.imshow("Image", imCrop)
cv2.waitKey(0)
In this example we will create a mask of pixels whose HSV components are between the variables lo and hi. In this example, by clicking with the left or right mouse button, you decrease or increase the hue h of the two variables lo and hi. You will also see that in our example this two thresholds differ not only in their hue but also in their saturation. You can test this script on an image of your face.
These few lines of code also illustrate how to manage actions on the mouse. They manage mouse events such as mouse move (cv2.EVENT_MOUSEMOVE), double middle click (EVENT_MBUTTONDBLCLK), right click (EVENT_RBUTTONDOWN) and left click (EVENT_LBUTTONDOWN).
import cv2
import numpy as np
def souris(event, x, y, flags, param):
global lo, hi, color, hsv_px
if event == cv2.EVENT_MOUSEMOVE:
# Conversion des trois couleurs RGB sous la souris en HSV
px = frame[y,x]
px_array = np.uint8([[px]])
hsv_px = cv2.cvtColor(px_array,cv2.COLOR_BGR2HSV)
if event==cv2.EVENT_MBUTTONDBLCLK:
color=image[y, x][0]
if event==cv2.EVENT_LBUTTONDOWN:
if color>5:
color-=1
if event==cv2.EVENT_RBUTTONDOWN:
if color<250:
color+=1
lo[0]=color-10
hi[0]=color+10
color=100
lo=np.array([color-5, 100, 50])
hi=np.array([color+5, 255,255])
color_info=(0, 0, 255)
cap=cv2.VideoCapture(0)
cv2.namedWindow('Camera')
cv2.setMouseCallback('Camera', souris)
hsv_px = [0,0,0]
# Creating morphological kernel
kernel = np.ones((3, 3), np.uint8)
while True:
ret, frame=cap.read()
image=cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
mask=cv2.inRange(image, lo, hi)
mask=cv2.erode(mask, kernel, iterations=1)
mask=cv2.dilate(mask, kernel, iterations=1)
image2=cv2.bitwise_and(frame, frame, mask= mask)
cv2.putText(frame, "Couleur: {:d}".format(color), (10, 30), cv2.FONT_HERSHEY_DUPLEX, 1, color_info, 1, cv2.LINE_AA)
# Affichage des composantes HSV sous la souris sur l'image
pixel_hsv = " ".join(str(values) for values in hsv_px)
font = cv2.FONT_HERSHEY_SIMPLEX
cv2.putText(frame, "px HSV: "+pixel_hsv, (10, 260),
font, 1, (255, 255, 255), 1, cv2.LINE_AA)
cv2.imshow('Camera', frame)
cv2.imshow('image2', image2)
cv2.imshow('Mask', mask)
if cv2.waitKey(1)&0xFF==ord('q'):
break
cap.release()
cv2.destroyAllWindows()
In general, it is very interesting to change the colour space to better target the space in which the object of interest is distinguishable: image=cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
After creating the mask with mask=cv2.inRange(image, lo, hi) it is sometimes relevant to denoise the resulting image by blurring or by some morphological operations. This allows the shapes to be closed and filled:
# Flouttage de l'image
image=cv2.blur(image, (7, 7))
# Erosion d'un mask
mask=cv2.erode(mask, None, iterations=4)
# dilatation d'un mask
mask=cv2.dilate(mask, None, iterations=4)
In the segmentation code of a colour image provided earlier, you will play with the kernel size (3x3 in our example), you will add a blurring step for each channel by playing with the neighbourhood size (7x7 in our example). Finally, play with the erosion and dilation steps by varying the number of times each morphological operator is applied (4 times in our example).
The segmentation code of a colour image provided previously allows you to define a binary mask of pixels whose HSV components are in the interval [lo,hi]. It is then possible to detect the connected elements in the mask to extract certain information, in this case the minEnclosingCircle. Other functions may be useful. You can find them here: https://docs.opencv.org/3.4/dd/d49/tutorial_py_contour_features.html.
Assuming that an object of interest is represented by a set of connected pixels whose colour is in the interval [lo,hi], it is then possible to define constraints on a set of features that allow the classification of the objects thus detected. You will add the following lines to the previous segmentation code.
elements=cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)[-2]
if len(elements) > 0:
c=max(elements, key=cv2.contourArea)
((x, y), rayon)=cv2.minEnclosingCircle(c)
if rayon>30:
cv2.circle(image2, (int(x), int(y)), int(rayon), color_info, 2)
cv2.circle(frame, (int(x), int(y)), 5, color_info, 10)
cv2.line(frame, (int(x), int(y)), (int(x)+150, int(y)), color_info, 2)
cv2.putText(frame, "Objet !!!", (int(x)+10, int(y) -10), cv2.FONT_HERSHEY_DUPLEX, 1, color_info, 1, cv2.LINE_AA)
Extraction de régions dans une image binarisée
Here are some lines in Python to extract regions of connected pixels in a binarised image label().
From these regions some properties are extracted regionprops().
This code works strategically in the same way as the previous segmentation script, but using the skimage library.
import cv2
import numpy as np
import matplotlib.pyplot as plt
from skimage.measure import label, regionprops
import math
image = cv2.imread('./imageasegmenter.jpg')
# passage en niveau de gris
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
###### extration des régions avec la lib skimage
# Binarisation de l'image
ret, thresh = cv2.threshold(gray, 127, 255, 1)
cv2.imshow("image seuillée",thresh)
cv2.waitKey(0)
# extraction des régions et des propriétés des régions
label_img = label(thresh)
regions = regionprops(label_img)
print(regions)
cv2.waitKey(0)
# affichage des régions et des boites englobantes
fig, ax = plt.subplots()
ax.imshow(thresh, cmap=plt.cm.gray)
for props in regions:
y0, x0 = props.centroid
orientation = props.orientation
x1 = x0 + math.cos(orientation) * 0.5 * props.minor_axis_length
y1 = y0 - math.sin(orientation) * 0.5 * props.minor_axis_length
x2 = x0 - math.sin(orientation) * 0.5 * props.major_axis_length
y2 = y0 - math.cos(orientation) * 0.5 * props.major_axis_length
ax.plot((x0, x1), (y0, y1), '-r', linewidth=2.5)
ax.plot((x0, x2), (y0, y2), '-r', linewidth=2.5)
ax.plot(x0, y0, '.g', markersize=15)
minr, minc, maxr, maxc = props.bbox
bx = (minc, maxc, maxc, minc, minc)
by = (minr, minr, maxr, maxr, minr)
ax.plot(bx, by, '-b', linewidth=2.5)
ax.axis((0, 600, 600, 0))
plt.show()
cv2.waitKey(0)
Détection d'objets par template matching
Il est possible de détecter un ou plusieurs objets dans une image en appliquant une procédure
de matching d'un template de chaque objet à détecter. Un template est une image du ou des objets en
question. La fonction à utiliser est cv.matchTemplate(img_gray,template,parametre). Plusieurs
parametre de matching sont possibles correspondant chacun à une métrique de corrélation.
Voici les lignes de codes que vous testerez. Vous testerez les parametres suivants afin de définir celui
qui fournit les meilleurs résultats. Par ailleurs, vous adapterez le code afin de prendre un charge le flux
des images de la Realsense et une image template de l'objet que vous voulez détecté.
It is possible to detect one or more objects in an image by applying a matching procedure to a template of each object to be detected. A template is one or several images of the object to be detected. The function to use is cv.matchTemplate(img_gray,template,parameter). Several matching parameters are possible, each corresponding to a correlation metric (cf. https://docs.opencv.org/4.x/d4/dc6/tutorial_py_template_matching.html).
Here are the lines of code you will test. You will test the following parameters to determine the one that gives the best results. You will also adapt the code to take a load of the Realsense image stream and a template image of the object you want to detect.
import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt
# charger l'image dans laquelle on cherche l'objet
img_rgb = cv.imread('car.png')
img_gray = cv.cvtColor(img_rgb, cv.COLOR_BGR2GRAY)
# charger le template de l'objet à rechercher
template = cv.imread('template.png',0)
# Récupération des dimensions de l'image
w, h = template.shape[::-1]
# Application du template atching
res = cv.matchTemplate(img_gray,template,cv.TM_CCOEFF_NORMED)
# Sélection des meilleurs matched objects
threshold = 0.8
loc = np.where( res >= threshold)
# Affichage de la boite englobante de chaque objet détecté
for pt in zip(*loc[::-1]):
cv.rectangle(img_rgb, pt, (pt[0] + w, pt[1] + h), (0,0,255), 2)
Segmentation des images par la méthodes des k-moyennes (kmeans)
Kmeans is a clustering algorithm whose objective is to divide a set of data points into several clusters. Each of point is assigned to a cluster with the nearest mean. The mean of each group is called the "centroid" or "centre". In total, k-means gives k distinct clusters of the original n data points. Data points within a particular cluster are considered to be "more similar" to each other than data points belonging to other groups. This algorithm can be applied to geometric, colourimetric and other original points.
We will use this method to provide a colour segmentation of an image, i.e. it amounts to finding the domain colours in the image.
from sklearn.cluster import KMeans
import matplotlib.pyplot as plt
import cv2
import numpy as np
#Ensuite charger une image et la convertir de BGR à RGB si nécessaire et l’afficher :
image = cv2.imread('lena.jpg')
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
plt.figure()
plt.axis("off")
plt.imshow(image)
In order to compute the image as a data point, it must be converted from a raster form to a vector form (rgb color list) before applying the clustering function:
n_clusters=5
image = image.reshape((image.shape[0] * image.shape[1], 3))
clt = KMeans(n_clusters = n_clusters )
clt.fit(image)
To display the most dominant colours in the image, two functions must be defined: centroid_histogram() to retrieve the number of different clusters and create a histogram based on the number of pixels assigned to each cluster; and plot_colors() to initialise the bar graph representing the relative frequency of each colour.
def centroid_histogram(clt):
numLabels = np.arange(0, len(np.unique(clt.labels_)) + 1)>>>
(hist, _) = np.histogram(clt.labels_, bins=numLabels)
# normalize the histogram, such that it sums to one
hist = hist.astype("float")
hist /= hist.sum()
return hist
def plot_colors(hist, centroids):
bar = np.zeros((50, 300, 3), dtype="uint8")
startX = 0
# loop over the percentage of each cluster and the color of
# each cluster
for (percent, color) in zip(hist, centroids):
# plot the relative percentage of each cluster
endX = startX + (percent * 300)
cv2.rectangle(bar, (int(startX), 0), (int(endX), 50),
color.astype("uint8").tolist(), -1)
startX = endX
return bar
Now you just need to build a cluster histogram and then create a figure displaying the number of pixels labeled for each color.
hist = centroid_histogram(clt)
bar = plot_colors(hist, clt.cluster_centers_)
plt.figure()
plt.axis("off")
plt.imshow(bar)
plt.show()
Classification d'images par la mathode des K plus proches voisins (k-NN ou KNN)
This exercise will allow you to learn a model from images in the CIFAR-10 database, downloadable here: https://www.cs.toronto.edu/~kriz/cifar.html.
Unzip the files into a folder that you will use in the following script.
As you can see, the CIFAR images are contained in the ./data folder.
import numpy as np
import cv2
basedir_data = "./data/"
rel_path = basedir_data + "cifar-10-batches-py/"
#Désérialiser les fichiers image afin de permettre l’accès aux données et aux labels:
def unpickle(file):
import pickle
with open(file, 'rb') as fo:
dict = pickle.load(fo,encoding='bytes')
return dict
X = unpickle(rel_path + 'data_batch_1')
img_data = X[b'data']
img_label_orig = img_label = X[b'labels']
img_label = np.array(img_label).reshape(-1, 1)
You can use teh following to check the résults :
print(img_data)
print('shape', img_data.shape)
print(img_label)
print('shape', img_label.shape)
You should find a 10000x3072 numpy array of uint8s (the 3072 comes from the 3 x 1024). Each row of the array stores a 32x32 RGB colour image. The images are stored in row-major order, so the first 32 entries in the array correspond to the red channel values of the first row of the image. The labels are stored in a 10000 x 1 array. To check the labels:
To load the test data, use the same procedure as before because the shape of the test data is the same as the shape of the training data:
test_X = unpickle(rel_path + 'test_batch');
test_data = test_X[b'data']
test_label = test_X[b'labels']
test_label = np.array(test_label).reshape(-1, 1)
print(test_data)
print('shape', test_data.shape)
print(test_label)
print('shape', test_label.shape)
Please note that the RGB components of the images are arranged in the form of a 1-dimensional vector. To display each image, it is therefore necessary to put it back in the form of a 2D RGB image. To do this, we operate as follows, considering that the images are of 32x32 resolution:
one_img=sample_img_data[0,:]
r = one_img[:1024].reshape(32, 32)
g = one_img[1024:2048].reshape(32, 32)
b = one_img[2048:].reshape(32, 32)
rgb = np.dstack([r, g, b])
cv2.imshow('Image CIFAR',rgb)
cv2.waitKey(0)
cv2.destroyAllWindows()
Now, you will apply the k-NN algorithm on all the images of the training base img_data and their labels img_label_orig.
You will use the fubnction KNeighborsClassifier of the sklearn python modules.
from sklearn.neighbors import KNeighborsClassifier
#def pred_label_fn(i, original):
# return original + '::' + meta[YPred[i]].decode('utf-8')
nbrs = KNeighborsClassifier(n_neighbors=3, algorithm='brute').fit(img_data, img_label_orig)
# test sur les 10 premières images
data_point_no = 10
sample_test_data = test_data[:data_point_no, :]
YPred = nbrs.predict(sample_test_data)
for i in range(0, len(YPred)):
#show_im(sample_test_data, test_label, meta, i, label_fn=pred_label_fn)
r = sample_test_data[i][:1024].reshape(32, 32)
g = sample_test_data[i][1024:2048].reshape(32, 32)
b = sample_test_data[i][2048:].reshape(32, 32)
print(YPred[i])
cv2.imshow('image test',np.dstack([r, g, b]))
neigh_dist,neigh_ind = nbrs.kneighbors([sample_test_data[i]])
print(neigh_ind)
for j in range(0, len(neigh_ind[0])):
one_img=img_data[neigh_ind[0][j],:]
r = one_img[:1024].reshape(32, 32)
g = one_img[1024:2048].reshape(32, 32)
b = one_img[2048:].reshape(32, 32)
rgb = np.dstack([r, g, b])
cv2.imshow('K plus proche image',np.dstack([r, g, b]))
cv2.waitKey(0)
Gestion de la depth map et estimation de la distance
This code measures the distance of the parts of the scene that are projected at each pixel of the camera by combining the depth image and the color image. It uses the following new part of code which was then integrated into the code you have already tested before.
# Use pixel value of depth-aligned color image to get 3D axes
x, y = int(color_colormap_dim[1]/2), int(color_colormap_dim[0]/2)
depth = depth_frame.get_distance(x, y)
dx ,dy, dz = rs.rs2_deproject_pixel_to_point(color_intrin, [x,y], depth)
distance = math.sqrt(((dx)**2) + ((dy)**2) + ((dz)**2))
The complete code integrating the acquisition and visualization part is as follows. The position is marked by a circle in both streams with the measured distance:
import pyrealsense2 as rs
import numpy as np
import math
import cv2,time,sys
pipeline = rs.pipeline()
config = rs.config()
colorizer = rs.colorizer()
# fps plus bas (30)
config.enable_stream(rs.stream.depth, 840, 480, rs.format.z16, 30)
config.enable_stream(rs.stream.color, 840, 480, rs.format.bgr8, 30)
pipeline.start(config)
align_to = rs.stream.depth
align = rs.align(align_to)
color_info=(0, 0, 255)
rayon=10
count=1
refTime= time.process_time()
freq= 60
try:
while True:
# This call waits until a new coherent set of frames is available on a device
frames = pipeline.wait_for_frames()
#Aligning color frame to depth frame
aligned_frames = align.process(frames)
depth_frame = aligned_frames.get_depth_frame()
aligned_color_frame = aligned_frames.get_color_frame()
if not depth_frame or not aligned_color_frame: continue
# Two ways to colorized the depth map
# first : using colorizer of pyrealsense
colorized_depth = colorizer.colorize(depth_frame)
depth_colormap = np.asanyarray(colorized_depth.get_data())
# second : using opencv by applying colormap on depth image (image must be converted to 8-bit per pixel first)
#depth_image = np.asanyarray(depth_frame.get_data())
#depth_colormap = cv2.applyColorMap(cv2.convertScaleAbs(depth_image, alpha=0.03), cv2.COLORMAP_JET)
# Get the intrinsic parameters
color_intrin = aligned_color_frame.profile.as_video_stream_profile().intrinsics
color_image = np.asanyarray(aligned_color_frame.get_data())
depth_colormap_dim = depth_colormap.shape
color_colormap_dim = color_image.shape
#Use pixel value of depth-aligned color image to get 3D axes
x, y = int(color_colormap_dim[1]/2), int(color_colormap_dim[0]/2)
depth = depth_frame.get_distance(x, y)
dx ,dy, dz = rs.rs2_deproject_pixel_to_point(color_intrin, [x,y], depth)
distance = math.sqrt(((dx)**2) + ((dy)**2) + ((dz)**2))
#print("Distance from camera to pixel:", distance)
#print("Z-depth from camera surface to pixel surface:", depth)
# Show images
images = np.hstack((color_image, depth_colormap)) # supose that depth_colormap_dim == color_colormap_dim (640x480) otherwize: resized_color_image = cv2.resize(color_image, dsize=(depth_colormap_dim[1], depth_colormap_dim[0]), interpolation=cv2.INTER_AREA)
cv2.circle(images, (int(x), int(y)), int(rayon), color_info, 2)
cv2.circle(images, (int(x+color_colormap_dim[1]), int(y)), int(rayon), color_info, 2)
# Affichage distance au pixel (x,y)
cv2.putText(images, "D="+str(round(distance,2)), (int(x)+10, int(y) -10), cv2.FONT_HERSHEY_DUPLEX, 1, color_info, 1, cv2.LINE_AA)
cv2.putText(images, "D="+str(round(distance,2)), (int(x+color_colormap_dim[1])+10, int(y) -10), cv2.FONT_HERSHEY_DUPLEX, 1, color_info, 1, cv2.LINE_AA)
# Show images
cv2.namedWindow('RealSense', cv2.WINDOW_NORMAL)
# Resize the Window
cv2.resizeWindow('RealSense', 960, 720)
cv2.imshow('RealSense', images)
cv2.waitKey(1)
# Frequency:
if count == 10 :
newTime= time.process_time()
freq= 10/((newTime-refTime))
refTime= newTime
count= 0
count+= 1
except Exception as e:
print(e)
pass
finally:
pipeline.stop()