Query: Pruning robot Computer vision

title: A Conceptual Model of a Stereo Vision System to Aid a Teleoperated Robot in Pruning Vegetation Close to Overhead Urban Power Lines
abstract: Detection of overhead urban lines is a primal information to ensure that a telerobotic pruning system will perform its task in a safe manner. In this context, this paper proposes a conceptual model of a computer vision system capable of identifying overhead urban lines and its three dimensional positioning by reviewing sensing technologies, applications of stereo vision with telerobotics and proposing a conceptual model to perform such task.

title: E2M: an energy efficient middleware for computer vision applications on autonomous mobile robots
abstract: Autonomous mobile robots (AMRs) have been widely utilized in industry to execute various on board computer vision applications including autonomous guidance, security patrol, object detection, and face recognition. Most of the applications executed by an AMR involve the analysis of camera images through trained machine learning models. Many research studies on machine learning focus either on performance without considering energy efficiency or on techniques such as pruning and compression to make the model more energy efficient. However, most previous work do not study the root causes of energy inefficiency for the execution of those applications on AMRs. The computing stack on an AMR accounts for 33% of the total energy consumption and can thus highly impact the battery life of the robot. Because recharging an AMR may disrupt the application execution, it is important to efficiently utilize the available energy for maximized battery life. In this paper, we first analyze the breakdown of power dissipation for the execution of computer vision applications on AMRs and discover three main root causes of energy inefficiency: uncoordinated access to sensor data, performance oriented model inference execution, and uncoordinated execution of concurrent jobs. In order to fix these three inefficiencies, we propose E2M, an energy efficient middleware software stack for autonomous mobile robots. First, E2M regulates the access of different processes to sensor data, e.g. camera frames, so that the amount of data actually captured by concurrently executing jobs can be minimized. Second, based on a predefined per process performance metric (e.g. safety, accuracy) and desired target, E2M manipulates the process execution period to find the best energy performance trade off. Third, E2M coordinates the execution of the concurrent processes to maximize the total contiguous sleep time of the computing hardware for maximized energy savings. We have implemented a prototype of E2M on HydraOne, a real world AMR. Our experimental results show that, compared to several baselines, E2M leads to 24% energy savings for the computing platform, which translates into an extra 11.5% of battery time and 14 extra minutes of robot runtime, with a performance degradation lower than 7.9% for safety and 1.84% for accuracy.

title: A Robot System for Pruning Grape Vines
abstract: This paper describes a robot system for the automatic pruning of grape vines. A mobile platform straddles the row of vines, and it images them with trinocular stereo cameras as it moves. A computer vision system builds a three dimensional (3D) model of the vines, an artificial intelligence (AI) system decides which canes to prune, and a six degree of freedom robot arm makes the required cuts. The system is demonstrated cutting vines in the vineyard. The main contributions of this paper are the computer vision system that builds 3D vine models, and the test of the complete integrated system. The vine models capture the structure of the plants so that the AI system can decide where to prune, and they are accurate enough that the robot arm can reach the required cuts. Vine models are reconstructed by matching features between images, triangulating feature matches to give a 3D model, then optimizing the model and the robot's trajectory jointly (incremental bundle adjustment) Trajectories are estimated online at 0.25 m/s, and they have errors below 1% when modeling a 96 m row of 59 vines. Pruning each vine requires the robot arm to cut an average of 8.4 canes. A collision free trajectory for the arm is planned in intervals of 1.5 s/vine with a rapidly exploring random tree motion planner. The total time to prune one vine is 2 min in field trials, which is similar to human pruners, and it could be greatly reduced with a faster arm. Trials also show that the long chain of interdependent components limits reliability. A commercially feasible pruning robot should stop and prune each vine in turn.

title: Evaluating pruned object detection networks for real time robot vision
abstract: Convolutional Neural Networks are the state of the art for computer vision problems such as classification and detection. Networks like YOLO and SSD have demonstrated excellent results on benchmark datasets such as the PASCAL VOC and COCO datasets. However these networks only run at real time with the support of powerful GPUs and are infeasible for use in low power embedded real time robotic applications. Pruning has been shown to be an efficient technique for reducing the runtime computational cost of a neural network while maintaining performance in image classification tasks. In this work we evaluate the efficacy of pruning on the problem of object detection using a modified tiny YOLO network. The network was trained on a custom object detection task and three pruning techniques were evaluated, including our contribution which specifically targets reducing the FLOPS in the network. The results show that pruning with our method followed by extended fine tuning achieved a 4.5x reduction in FLOPS and a 7x reduction in parameters with no drop in accuracy.

title: Design and Calibration of Multi camera Systems for 3D Computer Vision: Lessons Learnt from Two Case Studies
abstract: This paper examines how the design of imaging hardware for multi view 3D reconstruction affects the performance and complexity of the computer vision system as a whole. We examine two such systems: a grape vine pruning robot a 4.5 year/20 man year project, and a breast cancer screening device a 10 year/25 man year project. In both cases, mistakes in the initial imaging hardware design greatly increased the overall development time and cost by making the computer vision unnecessarily challenging, and by requiring the hardware to be redesigned and rebuilt. In this paper we analyse the mistakes made, and the successes experienced on subsequent hardware iterations. We summarise the lessons learned about platform design, camera setup, lighting, and calibration, so that this knowledge can help subsequent projects to succeed.

title: Pruning robot system for grape vines
abstract: A pruning robot system for grape vines comprises an automatic navigation module, a pruning module and a binocular stereo vision module. The automatic navigation module comprises a tracked truck, a pan and tilt camera, a left wheel driver, a right wheel driver, and an industrial personal computer. The pruning module comprises an agricultural mechanical arm and terminal pruning shears, wherein the agricultural mechanical arm is used for mounting the terminal pruning shears and guiding the same to shear branches. The terminal pruning shears are used for pruning branches. The industrial personal computer is used for analyzing and processing image information of the grape vines, extracting coordinate information of pruning points, planning trajectory of the mechanical arm and giving out movement directives to joints. The binocular stereo vision module comprises two industrial cameras, an image acquisition card and the industrial personal computer, wherein the two industrial cameras are used for acquiring image information of the grape vines, and the information is transmitted to the industrial personal computer for image identification and processing through the image acquisition card. By the aid of the pruning robot system, labor intensity can be lowered, and work efficiency can be improved.

title: Differentiable Mask for Pruning Convolutional and Recurrent Networks
abstract: Pruning is one of the most effective model reduction techniques. Deep networks require massive computation and such models need to be compressed to bring them on edge devices. Most existing pruning techniques are focused on vision based models like convolutional networks, while text based models are still evolving. The emergence of multi modal multi task learning calls for a general method that works on vision and text architectures simultaneously. We introduce a differentiable mask, that induces sparsity on various granularity to fill this gap. We apply our method successfully to prune weights, filters, subnetwork of a convolutional architecture, as well as nodes of a recurrent network.

title: An automatic measuring method for the trunk curvature of standing tree based on computer vision.
abstract: For the automatic pruning of the standing trees by pruning robot,the automatic measuring of trunk curvature of standing tree is a key step.According to the characteristics of digital image of standing tree,this paper puts forward a computer vision based automatic measuring method for trunk curvature of standing tree.The method includes four steps: the tree trunk image is detected by threshold segmentation;the edge of tree trunk is detected by Laplacian operator based on the tree trunk image filtered by median filtering method;the edge image of tree trunk is thinned based on the mathematical morphology;the last step is to fit the curve and compute the tree trunk curvature automatically.The experimental results indicate that the method is available for getting accurate tree trunk curvature.

title: Automating Node Pruning for LiDAR Based Topometric Maps in the Context of Teach and Repeat
abstract: Maps are a vital component in autonomous mobile systems. However, they can be computationally expensive to build, particularly in the case of globally consistent ones. To circumvent this issue, topometric approaches have been proposed, especially in the context of the Teach and Repeat paradigm. These topometric maps are graphs, where nodes are local maps and edges are the known transform between these nodes. When building these topometric maps, one important question is how far apart should these local maps be collected in the environment. On one hand, a densely sampled topometric map will be more robust, but at the cost of an increased database size. On the other hand, fewer local maps mean a more efficient topometric map. Traditional approaches record these local maps at a fixed, regular intervals. In this paper, we propose an offline algorithm that automatically prunes nodes in a way that offers some empirical guarantees on localization errors. In particular, our approach is based on first collecting a densely sampled topometric map, then finding a minimal subset of nodes using a cost based approach via Dijkstra's Algorithm. Offline experiments on three datasets confirmed the ability of our approach to minimize the size of topometric maps, and showed that the size of the map will vary given a certain demanded localization error tolerance and environment complexity.

title: Resource Aware Optimization of DNNs for Embedded Applications
abstract: Despite their outstanding success in solving complex computer vision problems, Deep Neural Networks (DNNs) still require high performance hardware for real time inference. Therefore they are not applicable to low cost embedded hardware, where memory resources, computational performance and power consumption are restricted. Furthermore, current approaches of fitting neural networks to embedded hardware are time consuming, inducing slow development cycles. To address these drawbacks and satisfy the demands of embedded hardware, this paper proposes a computationally efficient magnitude based pruning scheme, based on a half interval search, combined with effective weight sharing, fixed point quantization, and lossless compression. The proposed solution can be utilized to generate an optimized model, either with respect to memory demand or execution time. For instance, the memory demand of LeNet is compressed about 385x. VGG16 is pruned by about 14.5x, whilst its computational costs are reduced by about 1.6x for a CPU based application and 4.8x for an FPGA one.