Osama Asif Image
By
Osama Asif
September 2020

Optimized Approaches for Object Detections on Raspberry Pi

circuit board

Optimized Approaches for Object Detections on Raspberry Pi

You might have heard of Raspberry Pi, it’s a perfect mini computing machine that comes with decent hardware, a portable camera and can be used to for image processing and object detections in your IoT projects. You can even use your Raspberry Pi to train detections systems with your own datasets. In this post, we’ll look at different frameworks for setting up object detections on images and videos.

Raspberry Pi 3B which I’ll be using for this post comes with 1.2 GHz quad-core processor and 1 Gigabyte of RAM. If you have worked with detection systems like Darknet before you’ll be familiar that running detections on a GPU gives us a significant speed boost. However, Raspberry Pi does not contain a powerful enough GPU that can run detection algorithms in a short time. To overcome this constraint we’ll look at two methods which can take advantage of Pi’s multicore processor and we’ll be able to get pretty instant results.

Before we begin to discuss two optimized methods make sure you have the latest version of Raspbian OS(Lite or Full Desktop version) and the packages in your Raspbian installation are updated. If you aren’t sure you can run the following commands in the terminal to get the latest packages.


sudo apt-get update
sudo apt-get dist-upgrade

First Approach: Darknet-NNPACK
Darknet uses YOLO, a super fast classifier which works better than other detections networks. For Raspberry Pi, we’ll be using an optimized version of the original Darknet which is Darknet-NNPACK. There are multiple NNPACK optimized darknet repos on GitHub. We’ll be using the most tested and stable one.

Build Procedure

Step 1 – Install or Update Python-pip

sudo apt-get install python-pip


Step 2 – Install PeachyPy and Confu

sudo pip install --upgrade git+https://github.com/Maratyszcza/PeachPy
sudo pip install --upgrade git+https://github.com/Maratyszcza/confu


Step 3 – Install and configure Ninja

git clone https://github.com/ninja-build/ninja.git
cd ninja
git checkout release
./configure.py --bootstrap
export NINJA_PATH=$PWD
Note: Your Pi might hang in the configure ninja step.
If that happens you can stop the configuring process and start over.
If the problem still persists I recommend increasing the swap space
of your Raspberry Pi during the build process.


Step 4 – Build NNPACK for Darknet

cd
git clone https://github.com/digitalbrain79/NNPACK-darknet.git
cd NNPACK-darknet
confu setup
python ./configure.py --backend auto
$NINJA_PATH/ninja
sudo cp -a lib/* /usr/lib/
sudo cp include/nnpack.h /usr/include/
sudo cp deps/pthreadpool/include/pthreadpool.h /usr/include/

Step 5 – Build the Darknet

cd
git clone https://github.com/digitalbrain79/darknet-nnpack.git
cd darknet-nnpack
Make

Final Step – Download Weights and Test

To test your installation you first need to download YOLO weights file. YoloV2 and YoloV3-tiny work well on the Raspberry Pi. Run the following commands in terminal
wget https://pjreddie.com/media/files/yolov3-tiny.weights
./darknet detect cfg/yolov3-tiny.cfg yolov3-tiny.weights data/dog.jpg
If everything went well according to the build instructions above and the process didn’t run into any errors obviously! You’ll see detection results from the dog.jpg image in your terminal. As far as the detection time is concerned my device achieved a detection time of 2 seconds per image with YoloV3 tiny.

Second Approach: Optimized OpenCV


OpenCV is an open-source computer vision library written in C++ and it has bindings available in other languages for example Python. In the second approach, we’ll be using optimizing OpenCV for Raspberry Pi. Raspberry Pi uses an ARM processor which provides NEON and VFP as a part of it’s processing core. We’ll explicitly build OpenCV with flags for NEON and VFP to get the most of Pi’s processor.

Build Procedure

Step 1 – Download Dependencies for Optimized OpenCV

sudo apt-get install build-essential cmake pkg-config
sudo apt-get install libjpeg-dev libtiff5-dev libjasper-dev libpng12-dev
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
sudo apt-get install libxvidcore-dev libx264-dev
sudo apt-get install libgtk2.0-dev libgtk-3-dev
sudo apt-get install libcanberra-gtk*
sudo apt-get install libatlas-base-dev gfortran
sudo apt-get install python2.7-dev python3-dev
sudo apt-get install python3-pip


Step 2 – Download OpenCV source

cd
wget -O opencv401.zip https://github.com/opencv/opencv/archive/4.0.1.zip
unzip opencv401.zip
wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/4.0.1.zip
unzip opencv_contrib.zip

Step 3 – Download NumPy

pip3 install numpy or pip install numpy(for python 2.7)


Step 4 – Build OpenCV

cd ~/opencv401/
mkdir build
cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \

-D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib-4.0.1/modules \

-D ENABLE_NEON=ON \

-D ENABLE_VFPV3=ON \

-D BUILD_TESTS=OFF \

-D INSTALL_PYTHON_EXAMPLES=OFF \

-D BUILD_EXAMPLES=OFF ..


Step 5 – Compile OpenCV

Before this step make sure that you have at least 7-8 GB free space available on your SD card. You can also use an external HDD or USB pen drive, search for instructions on how to mount an external drive in Raspberry Pi.
Compiling OpenCV with all cores can take about 3 hours in total so you have to be patient.
If you run into virtual memory exhaust errors try to increase the swap space of Pi up to 1024 MB. Though, remember to bring it back to 100MB after completing the build.
Multicore compiling might not work so if the compiling process fails, again and again, you can try compiling it with one core. Compiling this way will take about 8 to 10 hours.
Keeping all the above instructions in mind you can start building opencv with  “make -j4” or “make” this is for single core build
Step 6 – Installing the compiled OpenCV
sudo make install
sudo ldconfig
After a successful build, you can test the installation with a simple python script, write import cv2 on the top of the script to verify installation. In my tests OpenCV performed better than Darknet-NNPACK, detections took 1.5 seconds per image.

Conclusion

Phew! That was a long post. Anyway, there you have it two optimized ways of running object detections in Raspberry Pi. Both methods have their pros and cons. Darknet-NNPACK is easier to set up but it’s kind of rigid when you want to call it’s detect method in your python scripts, you might have to write a wrapper yourself to call darknet commands from a script. OpenCV on the other side tests your patience while setting up but it’s a lot easier to use with bindings such as python.

Enou is a design centered agency based in Lahore. Our mission is “To reinforce businesses and start-ups by creating value in their work, through our digital services” and the driving force that keeps us motivated is our desire for perfection. We truly believe in the transformative power of illustration and design and their ability to simplify communications, elevate experiences, engage and inspire people because in the end, what they see is what they believe!

(This blog has been copied from old website enou.co)

Osama Asif Image
By
Osama Asif

Osama Asif is Engineering Lead at ENOU. He talks about software development and often shares his practices and views about emerging technologies.

Continue reading...

Let's build something great together

Would you like to start a new project or work on an idea with us? We’d be happy to collaborate.

Tell us everything