title: Time-of-flight perception pipeline for selective green asparagus harvesting: Theory and application

authors: Peebles, Matthew Christopher Scott

subjects: Asparagus

abstract: Generally declining labour markets, coupled with a significant increase in the projected global population have raised concerns over a potential food scarcity crisis. In response to this, the agricultural industry is currently undergoing a technological revolution, especially with respect to robotic harvesting.
Green asparagus, due to its unique physiology and labour intensive harvesting requirements, has been a long-time crop of focus in this field. Despite the relatively large body of work surrounding the development of selective robotic asparagus harvesting solutions, no such machine has yet ascended to the level of being commercially competitive. A critical component of a prospective selective robotic asparagus harvester, that is missing from contemporary machines, is a robust vision system, capable of detecting, and localising, harvest-eligible spears in real-time from a commercial asparagus field.
This thesis presents a novel perception pipeline for time-of-flight images that can achieve this task, and describes the implementation of the proposed perception pipeline into two functional robotic harvesters, AHR-1 and AHR2, capable of harvesting green asparagus spears in a real-world commercial setting.
The proposed perception pipeline achieves spear detection by firstly, segmenting the soil plane from an input pointcloud of an asparagus row, generated with a time-of-flight camera. Two plane detection methods, namely RANSAC and a modified version Hyun’s method (MHM) were investigated for this purpose. A detailed evaluation of the performance of each method on cluttered scenes revealed both RANSAC and MHM to be suitable for soil plane segmentation of asparagus beds, with both methods demonstrating similar RMSE error across a variety of scenes. The stability of model predictions made by RANSAC was generally found to be lower than that achieved by MHM, particularly for high-clutter scenes. This was determined to be due to the non-deterministic nature of RANSAC coupled with high degrees of soil plane occlusion in high clutter scenes.
Following soil plane removal, non-asparagus points pertaining to rocks, weeds, and other field debris are then filtered from the scene. This is achieved by coarse filtering input points based on the output of a FRCNN model. The various FRCNN models utilised in this work were trained with a novel dataset of labelled images, collected from various asparagus farms throughout New Zealand and California, USA. Evaluation of these models revealed a typical maximum F1 score of 0.73, providing reasonable frame-by-frame identification of asparagus features.
The remaining point clusters, representing each asparagus spear in the scene, are then filtered to remove flying pixels; a typical artefact of time-of-flight imaging. A novel geometric filter, named the closest point filter (CP filter), was developed for this task. Based on laboratory testing, it was found that this filter achieved a 68% reduction in the mean standard deviation of intra-cluster distance with respect to ground-truth positions, resulting in a significant improvement in the accuracy of base point predictions.
The proposed perception pipeline is implemented inside a ROS framework, and deployed on two robotic harvesting platforms. The first platform, AHR-1, was developed as a proof-of-concept system. This system provided a wealth of knowledge which was utilised to develop a more complete prototype asparagus harvester, AHR-2. The design of AHR-1 and AHR-2 was largely informed by the shortcomings of existing asparagus harvesting robots from the literature, particularly with respect to their harvesting, and detection strategies.
The literature surrounding robotic asparagus harvesters does not provide an adequate method for objectively evaluating the performance of such systems. Consequentially, performance metrics pertaining to existing robotic harvesters are relatively opaque, particularly with respect to the selective nature of the harvesting task. These inadequacies necessitated the development of a novel evaluation method for the evaluation of a selective asparagus harvester, which was utilised to measure the performance of both AHR-1 and AHR-2.
Several field trials of AHR-1 and AHR-2 were conducted on farms throughout New Zealand and California, USA. The resulting analysis revealed that AHR-2 achieved state-of-the-art performance, harvesting 45.9% of all harvest eligible spears with a precision of 87.2% at its nominal ground speed of 0.3m/s. The vision system detected 97% of all harvest-eligible spears with a precision of 74.5% at this speed. When operating at a ground speed of 0.7m/s AHR 2’s harvesting rate fell to 22% with 95% precision. The corresponding drop in the vision systems detection rate was relatively small, dropping to 92.5% with 87.2% precision at a ground speed of 0.7m/s. From this it was concluded that the vision system outperformed the available hardware. The perception pipeline proposed by this thesis achieved state-of-the-art performance at a variety of ground speeds. The pipeline demonstrated a robustness to the unstructured nature of commercial asparagus fields, providing spear locations which successfully facilitated the robotic harvest of green asparagus spears.

text: Time-of-flight perception pipeline for selective green asparagus harvesting: Theory and application Generally declining labour markets, coupled with a significant increase in the projected global population have raised concerns over a potential food scarcity crisis. In response to this, the agricultural industry is currently undergoing a technological revolution, especially with respect to robotic harvesting.
Green asparagus, due to its unique physiology and labour intensive harvesting requirements, has been a long-time crop of focus in this field. Despite the relatively large body of work surrounding the development of selective robotic asparagus harvesting solutions, no such machine has yet ascended to the level of being commercially competitive. A critical component of a prospective selective robotic asparagus harvester, that is missing from contemporary machines, is a robust vision system, capable of detecting, and localising, harvest-eligible spears in real-time from a commercial asparagus field.
This thesis presents a novel perception pipeline for time-of-flight images that can achieve this task, and describes the implementation of the proposed perception pipeline into two functional robotic harvesters, AHR-1 and AHR2, capable of harvesting green asparagus spears in a real-world commercial setting.
The proposed perception pipeline achieves spear detection by firstly, segmenting the soil plane from an input pointcloud of an asparagus row, generated with a time-of-flight camera. Two plane detection methods, namely RANSAC and a modified version Hyun’s method (MHM) were investigated for this purpose. A detailed evaluation of the performance of each method on cluttered scenes revealed both RANSAC and MHM to be suitable for soil plane segmentation of asparagus beds, with both methods demonstrating similar RMSE error across a variety of scenes. The stability of model predictions made by RANSAC was generally found to be lower than that achieved by MHM, particularly for high-clutter scenes. This was determined to be due to the non-deterministic nature of RANSAC coupled with high degrees of soil plane occlusion in high clutter scenes.
Following soil plane removal, non-asparagus points pertaining to rocks, weeds, and other field debris are then filtered from the scene. This is achieved by coarse filtering input points based on the output of a FRCNN model. The various FRCNN models utilised in this work were trained with a novel dataset of labelled images, collected from various asparagus farms throughout New Zealand and California, USA. Evaluation of these models revealed a typical maximum F1 score of 0.73, providing reasonable frame-by-frame identification of asparagus features.
The remaining point clusters, representing each asparagus spear in the scene, are then filtered to remove flying pixels; a typical artefact of time-of-flight imaging. A novel geometric filter, named the closest point filter (CP filter), was developed for this task. Based on laboratory testing, it was found that this filter achieved a 68% reduction in the mean standard deviation of intra-cluster distance with respect to ground-truth positions, resulting in a significant improvement in the accuracy of base point predictions.
The proposed perception pipeline is implemented inside a ROS framework, and deployed on two robotic harvesting platforms. The first platform, AHR-1, was developed as a proof-of-concept system. This system provided a wealth of knowledge which was utilised to develop a more complete prototype asparagus harvester, AHR-2. The design of AHR-1 and AHR-2 was largely informed by the shortcomings of existing asparagus harvesting robots from the literature, particularly with respect to their harvesting, and detection strategies.
The literature surrounding robotic asparagus harvesters does not provide an adequate method for objectively evaluating the performance of such systems. Consequentially, performance metrics pertaining to existing robotic harvesters are relatively opaque, particularly with respect to the selective nature of the harvesting task. These inadequacies necessitated the development of a novel evaluation method for the evaluation of a selective asparagus harvester, which was utilised to measure the performance of both AHR-1 and AHR-2.
Several field trials of AHR-1 and AHR-2 were conducted on farms throughout New Zealand and California, USA. The resulting analysis revealed that AHR-2 achieved state-of-the-art performance, harvesting 45.9% of all harvest eligible spears with a precision of 87.2% at its nominal ground speed of 0.3m/s. The vision system detected 97% of all harvest-eligible spears with a precision of 74.5% at this speed. When operating at a ground speed of 0.7m/s AHR 2’s harvesting rate fell to 22% with 95% precision. The corresponding drop in the vision systems detection rate was relatively small, dropping to 92.5% with 87.2% precision at a ground speed of 0.7m/s. From this it was concluded that the vision system outperformed the available hardware. The perception pipeline proposed by this thesis achieved state-of-the-art performance at a variety of ground speeds. The pipeline demonstrated a robustness to the unstructured nature of commercial asparagus fields, providing spear locations which successfully facilitated the robotic harvest of green asparagus spears.

year: 2021

title: “We’re trying to heal, you know?” A mixed methods analysis of the spatial equity of General Practitioner services in the Waikato District Health Board region

authors: Whitehead, Jesse

subjects: spatial equity

abstract: Inequitable access to health services can cause and exacerbate inequities in health outcomes and should therefore be monitored regularly to ensure that service distributions match population needs. Health service accessibility includes several factors and can be monitored using both quantitative and qualitative methods. We present an exploratory analysis of the spatial equity of general practice services in the Waikato District Health Board region using a mixed methods approach. Geographic Information Systems are used to assess the spatial accessibility of GP services, and in-depth qualitative interviews provide a better understanding of not only where inequities exist, but why they occur.

text: “We’re trying to heal, you know?” A mixed methods analysis of the spatial equity of General Practitioner services in the Waikato District Health Board region Inequitable access to health services can cause and exacerbate inequities in health outcomes and should therefore be monitored regularly to ensure that service distributions match population needs. Health service accessibility includes several factors and can be monitored using both quantitative and qualitative methods. We present an exploratory analysis of the spatial equity of general practice services in the Waikato District Health Board region using a mixed methods approach. Geographic Information Systems are used to assess the spatial accessibility of GP services, and in-depth qualitative interviews provide a better understanding of not only where inequities exist, but why they occur.

year: 2020

title: Akuhata Koropango Lipsey

authors: Hart, Philip

abstract: The eldest son of George and Ema Lipsey, Augie, as he was known, was most notable, and popular, as a sportsman. A good rugby player in his youth, he later became a keen golfer. As well, he owned and raced horses, an interest that would result in his financial downfall. Although a farmer off and on, he had no farming skills and had to take on other work, in his last years working in a billiard saloon.  
After marrying an English girl and fathering several children, he proved to be a poor provider for his family. His wife, well liked for her involvement in various worthy causes, especially those organized by the Anglican Church, had to teach dancing and to milk cows to raise money, for Lipsey squandered his inheritance, notably on the Auckland racecourse. His method of meeting his debts and living expenses was to sell land, and although over many years officials tried to prevent this (to protect his family), they gradually let him have his way. Bankrupted twice, on his death his family was left in poverty.

text: Akuhata Koropango Lipsey The eldest son of George and Ema Lipsey, Augie, as he was known, was most notable, and popular, as a sportsman. A good rugby player in his youth, he later became a keen golfer. As well, he owned and raced horses, an interest that would result in his financial downfall. Although a farmer off and on, he had no farming skills and had to take on other work, in his last years working in a billiard saloon.  
After marrying an English girl and fathering several children, he proved to be a poor provider for his family. His wife, well liked for her involvement in various worthy causes, especially those organized by the Anglican Church, had to teach dancing and to milk cows to raise money, for Lipsey squandered his inheritance, notably on the Auckland racecourse. His method of meeting his debts and living expenses was to sell land, and although over many years officials tried to prevent this (to protect his family), they gradually let him have his way. Bankrupted twice, on his death his family was left in poverty.

year: 2016