Package: berrynet
Deep Learning Gateway on Raspberry Pi And Other Edge DevicesSupporting BerryNetBecome a backer or sponsor on Open Collective.One-time donation via PayPal or crypto-currencies.IntroductionThis project turns edge devices such as Raspberry Pi into an intelligent gateway with deep learning running on it. No internet connection is required, everything is done locally on the edge device itself. Further, multiple edge devices can create a distributed AIoT network.At DT42, we believe that bringing deep learning to edge devices is the trend towards the future. It not only saves costs of data transmission and storage but also makes devices able to respond according to the events shown in the images or videos without connecting to the cloud.Figure 1: BerryNet architectureFigure 1 shows the software architecture of the project, we use Node.js/Python, MQTT and an AI engine to analyze images or video frames with deep learning. So far, there are two default types of AI engines, the classification engine (with Inception v3[1]model) and the object detection engine (with TinyYOLO[2]model or MobileNet SSD[3]model). Figure 2 shows the differences between classification and object detection.Figure 2: Classification vs detectionOne of the application of this intelligent gateway is to use the camera to monitor the place you care about. For example, Figure 3 shows the analyzed results from the camera hosted in the DT42 office. The frames were captured by the IP camera and they were submitted into the AI engine. The output from the AI engine will be shown in the dashboard. We are working on the Email and IM notification so you can get a notification when there is a dog coming into the meeting area with the next release.Figure 3: Object detection result exampleTo bring easy and flexible edge AI experience to user, we keep expending support of the AI engines and the reference HWs.Figure 4: Reference hardwaresInstallationYou can install BerryNet by using pre-built image or from source. Please refer to theWiki pagefor the details.We are pushing BerryNet into Debian repository, so you will be able to install by only typing one command in the future.Here is the quick steps to install from source:$ git clone https://github.com/DT42/BerryNet.git
$ cd BerryNet
$ ./configureStart and Stop BerryNetBerryNet performs an AIoT application by connecting independent components together. Component types include but not limited to AI engine, I/O processor, data processor (algorithm), or data collector.We recommend to manage BerryNet componetns bysupervisor, but you can also run BerryNet components manually. You can manage BerryNet viasupervisorctl:# Check status of BerryNet components
$ sudo supervisorctl status all

# Stop Camera client
$ sudo supervisorctl stop camera

# Restart all components
$ sudo supervisorctl restart all

# Show last stderr logs of camera client
$ sudo supervisorctl tail camera stderrFor more possibilities of supervisorctl, please refer to theofficial tutorial.The default application has three components:Camera client to provide input imagesObject detection engine to find type and position of the detected objects in an imageDashboard to display the detection resultsYou will learn how to configure or change the components in theConfigurationsection.Dashboard: FreeboardOpen Freeboard on RPi (with touch screen)Freeboard is a web-based dashboard. Here are the steps to show the detection result iamge and text on Freeboard:1: Enterhttp://127.0.0.1:8080in browser's URL bar, and press enter2:Downloadthe Freeboard configuration for default application,dashboard-tflitedetector.json2: ClickLOAD FREEBOARD, and select the newly downloadeddashboard-tflitedetector.json3: Wait for seconds, you should see the inference result image and text on FreeboardOpen Freeboard on another computerAssuming that you have two devices:Device A with IP192.168.1.42, BerryNet default application runs on itDevice B with IP192.168.1.43, you want to open Freeboard and see the detection result on itHere are the steps:1: Enterhttp://192.168.1.42:8080in browser's URL bar, and press enter2:Downloadthe Freeboard configuration for default application,dashboard-tflitedetector.json3: Replace all thelocalhostto192.168.1.42indashboard-tflitedetector.json2: ClickLOAD FREEBOARD, and select the newly downloadeddashboard-tflitedetector.json3: Wait for seconds, you should see the inference result image and text on FreeboardFor more details about dashboard configuration (e.g. how to add widgets), please refer toFreeboard project.Enable Data CollectorYou might want to store the snapshot and inference results for data analysis.To run BerryNet data collector manually, you can run the command below:$ bn_data_collector --topic-config <topic-config-filepath> --data-dirpath <result-dirpath>The topic config indicates what MQTT topic the data collector will listen, and what handler will be triggered. Here is a topic config exmaple:{
    "berrynet/engine/tflitedetector/result": "self.update"
}The inference result image and text will be saved into the indicated result directory.ConfigurationThe default supervisor config is at/etc/supervisor/conf.d/berrynet-tflite.conf. To write your own supervisor config, you can refer toherefor more example supervisor configs of BerryNetCamera ClientBerryNet camera client can run in two modes: stream or file. In stream mode, local camera (e.g. USB camera and RPi camera) and IP camera can be supported, and input frame rate (FPS) can be changed on demand (default is 1). In file mode, user can indicate filepath as input source.To run camera client in stream mode:$ bn_camera --fps 5To run camera client in file mode:$ bn_camera --mode file --filepath <image-filepath>Use Your Data To TrainThe original instruction of retraining YOLOv2 model seegithub repository of darknetIn the current of BerryNet, TinyYolo is used instead of YOLOv2.
The major differences are:Create file yolo-obj.cfg with the same content as intiny-yolo.cfgDownload pre-trained weights of darknet reference model,darknet.weights.12, for the convolutional layers (6.1MB)https://drive.google.com/drive/folders/0B-oZJEwmkAObMzAtc2QzZDhyVGM?usp=sharingThe rest parts are the same as retraining YOLO.If you useLabelMeto annotate data,utils/xmlTotxt.pycan help convert the xml format to the text format that darknet uses.DiscussionPlease refer to theSlack,Telegram GrouporGoogle Groupfor questions, suggestions, or any idea discussion.