title: Vision based system for detecting and counting mobility aids in surveillance videos

authors: Mukhtar, Amir

subjects: Mobility Aids

abstract: Automatic surveillance video analysis is popular among computer vision researchers due to its wide range of applications that require automated systems. Automated systems are to replace manual analysis of videos which is tiresome, expensive, and time-consuming. Image and video processing techniques are often used in the design of automatic detection and monitoring systems. Compared with normal indoor videos, outdoor surveillance videos are often difficult to process due to the uncontrolled environment, camera angle, and varying lighting and weather conditions. 
This research aims to contribute to the computer vision field by proposing an object detection and tracking algorithm that can handle multi-object and multi-class scenarios. The problem is solved by developing an application to count disabled pedestrians in surveillance videos by automatically detecting and tracking mobility aids and pedestrians. The application demonstrates that the proposed ideas achieve the desired outcomes. There are extensive studies on pedestrian detection and gait analysis in the computer vision field, but limited work is carried out on identifying disabled pedestrians or mobility aids. Detection of mobility aids in videos is challenging since the disabled person often occludes mobility aids and visibility of mobility aid depends on the direction of the walk with respect to the camera. For example, a walking stick is visible most times in front-on view while it is occluded when it happens to be on the walker's rear side. Furthermore, people use various mobility aids and their make and type changes with time as technology advances. The system should detect the majority of mobility aids to report reliable counting data. The literature review revealed that no system exists for detecting disabled pedestrians or mobility aids in surveillance videos. A lack of annotated image data containing mobility aids is also an obstacle to developing a machine-learning-based solution to detect mobility aids.
In the first part of this thesis, we explored moving pedestrians' video data to extract the gait signals using manual and automated procedures. Manual extraction involved marking the pedestrians' head and leg locations and analysing those signals in the time domain. Analysis of stride length and velocity features indicate an abnormality if a walker is physically disabled.  The automated system is built by combining the \acrshort{yolo} object detector, GMM based foreground modelling and star skeletonisation in a pipeline to extract the gait signal. The automated system failed to recognise a disabled person from its gait due to poor localisation by \acrshort{yolo}, incorrect segmentation and silhouette extraction due to moving backgrounds and shadows. The automated gait analysis approach failed due to various factors including environmental constraints, viewing angle, occlusions, shadows, imperfections in foreground modelling, object segmentation and silhouette extraction. 
In the later part of this thesis, we developed a CNN based approach to detect mobility aids and pedestrians. The task of identifying and counting disabled pedestrians in surveillance videos is divided into three sub-tasks: mobility aid and person detection, tracking and data association of detected objects, and counting healthy and disabled pedestrians. A modern object detector called YOLO, an improved data association algorithm (SORT), and a new pairing approach are applied to complete the three sub-tasks. Improvement of the SORT algorithm and introducing a pairing approach are notable contributions to the computer vision field. The SORT algorithm is strictly one class and without an object counting feature. SORT is enhanced to be multi-class and able to track accelerating or temporarily occluded objects. The pairing strategy associates a mobility aid with the nearest pedestrian and monitors them over time to see if the pair is reliable. A reliable pair represents a disabled pedestrian and counting reliable pairs calculates the number of disabled people in the video. The thesis also introduces an image database that was gathered as part of this study. The dataset comprises 5819 images belonging to eight different object classes, including five mobility aids, pedestrians, cars, and bicycles. The dataset was needed to train a CNN that can detect mobility aids in videos. 
The proposed mobility aid counting system is evaluated on a range of surveillance videos collected from outdoors with real-world scenarios. The results prove that the proposed solution offers a satisfactory performance in picking mobility aids from outdoor surveillance videos. The counting accuracy of 94% on test videos meets the design goals set by the advocacy group that need this application. Most test videos had objects from multiple classes in them. The system detected five mobility aids (wheelchair, crutch, walking stick, walking frame and mobility scooter), pedestrians and two distractors (car and bicycle). The training system on distractors' classes was to ensure the system can distinguish objects that are similar to mobility aids from mobility aids. In some cases, the convolutional neural network reports a mobility aid with an incorrect type. For example, the shape of crutch and stick are very much alike, and therefore, the system confuses one with the other. However, it does not affect the final counts as the aim was to get the overall counts of mobility aids (of any type) and determining the exact type of mobility aid is optional.

text: Vision based system for detecting and counting mobility aids in surveillance videos Automatic surveillance video analysis is popular among computer vision researchers due to its wide range of applications that require automated systems. Automated systems are to replace manual analysis of videos which is tiresome, expensive, and time-consuming. Image and video processing techniques are often used in the design of automatic detection and monitoring systems. Compared with normal indoor videos, outdoor surveillance videos are often difficult to process due to the uncontrolled environment, camera angle, and varying lighting and weather conditions. 
This research aims to contribute to the computer vision field by proposing an object detection and tracking algorithm that can handle multi-object and multi-class scenarios. The problem is solved by developing an application to count disabled pedestrians in surveillance videos by automatically detecting and tracking mobility aids and pedestrians. The application demonstrates that the proposed ideas achieve the desired outcomes. There are extensive studies on pedestrian detection and gait analysis in the computer vision field, but limited work is carried out on identifying disabled pedestrians or mobility aids. Detection of mobility aids in videos is challenging since the disabled person often occludes mobility aids and visibility of mobility aid depends on the direction of the walk with respect to the camera. For example, a walking stick is visible most times in front-on view while it is occluded when it happens to be on the walker's rear side. Furthermore, people use various mobility aids and their make and type changes with time as technology advances. The system should detect the majority of mobility aids to report reliable counting data. The literature review revealed that no system exists for detecting disabled pedestrians or mobility aids in surveillance videos. A lack of annotated image data containing mobility aids is also an obstacle to developing a machine-learning-based solution to detect mobility aids.
In the first part of this thesis, we explored moving pedestrians' video data to extract the gait signals using manual and automated procedures. Manual extraction involved marking the pedestrians' head and leg locations and analysing those signals in the time domain. Analysis of stride length and velocity features indicate an abnormality if a walker is physically disabled.  The automated system is built by combining the \acrshort{yolo} object detector, GMM based foreground modelling and star skeletonisation in a pipeline to extract the gait signal. The automated system failed to recognise a disabled person from its gait due to poor localisation by \acrshort{yolo}, incorrect segmentation and silhouette extraction due to moving backgrounds and shadows. The automated gait analysis approach failed due to various factors including environmental constraints, viewing angle, occlusions, shadows, imperfections in foreground modelling, object segmentation and silhouette extraction. 
In the later part of this thesis, we developed a CNN based approach to detect mobility aids and pedestrians. The task of identifying and counting disabled pedestrians in surveillance videos is divided into three sub-tasks: mobility aid and person detection, tracking and data association of detected objects, and counting healthy and disabled pedestrians. A modern object detector called YOLO, an improved data association algorithm (SORT), and a new pairing approach are applied to complete the three sub-tasks. Improvement of the SORT algorithm and introducing a pairing approach are notable contributions to the computer vision field. The SORT algorithm is strictly one class and without an object counting feature. SORT is enhanced to be multi-class and able to track accelerating or temporarily occluded objects. The pairing strategy associates a mobility aid with the nearest pedestrian and monitors them over time to see if the pair is reliable. A reliable pair represents a disabled pedestrian and counting reliable pairs calculates the number of disabled people in the video. The thesis also introduces an image database that was gathered as part of this study. The dataset comprises 5819 images belonging to eight different object classes, including five mobility aids, pedestrians, cars, and bicycles. The dataset was needed to train a CNN that can detect mobility aids in videos. 
The proposed mobility aid counting system is evaluated on a range of surveillance videos collected from outdoors with real-world scenarios. The results prove that the proposed solution offers a satisfactory performance in picking mobility aids from outdoor surveillance videos. The counting accuracy of 94% on test videos meets the design goals set by the advocacy group that need this application. Most test videos had objects from multiple classes in them. The system detected five mobility aids (wheelchair, crutch, walking stick, walking frame and mobility scooter), pedestrians and two distractors (car and bicycle). The training system on distractors' classes was to ensure the system can distinguish objects that are similar to mobility aids from mobility aids. In some cases, the convolutional neural network reports a mobility aid with an incorrect type. For example, the shape of crutch and stick are very much alike, and therefore, the system confuses one with the other. However, it does not affect the final counts as the aim was to get the overall counts of mobility aids (of any type) and determining the exact type of mobility aid is optional.

year: 2022

title: Modification of kraft wood-pulp fibre with silica for surface functionalisation

authors: Love, Karen T.

subjects: wood

abstract: A new science strategy for natural fibre modification was devised in which glass surface properties would be imparted to wood-derived fibre. The enhancements known from addition of silane reagents to glass fibre–polymer composites could therefore be realised for modified cellulose fibre–polymer composites. A process is described whereby the internal void spaces and micropores of never-dried Kraft pulp fibre walls were impregnated with silica. This was achieved by initial dehydration of never-dried fibre through azeotropic distillation to achieve substitution of fibre water with the silicon chemical solution over a range of concentrations. Kraft fibres were stiffened and made resistant to collapse from the effect of the azeotrope drying. Specific chemical reaction of azeotrope-dried fibre with the reagent ClSi(OEt)3 followed by base-catalysed hydrolysis of the ester groups formed a fibre-bound silica composite. The physico-chemical substitution of water from micropores and internal voids of never-dried fibre with property-modifying chemicals offers possibilities in the development of new fibre characteristics, including fibres which may be hardened, plasticised, and/or stabilised against moisture, biodegradation or fire. The embedded silica may also be used as sites of attachment for coupling agents to modify the hydrophilic character of the fibre or to functionalise the fibre surface.

text: Modification of kraft wood-pulp fibre with silica for surface functionalisation A new science strategy for natural fibre modification was devised in which glass surface properties would be imparted to wood-derived fibre. The enhancements known from addition of silane reagents to glass fibre–polymer composites could therefore be realised for modified cellulose fibre–polymer composites. A process is described whereby the internal void spaces and micropores of never-dried Kraft pulp fibre walls were impregnated with silica. This was achieved by initial dehydration of never-dried fibre through azeotropic distillation to achieve substitution of fibre water with the silicon chemical solution over a range of concentrations. Kraft fibres were stiffened and made resistant to collapse from the effect of the azeotrope drying. Specific chemical reaction of azeotrope-dried fibre with the reagent ClSi(OEt)3 followed by base-catalysed hydrolysis of the ester groups formed a fibre-bound silica composite. The physico-chemical substitution of water from micropores and internal voids of never-dried fibre with property-modifying chemicals offers possibilities in the development of new fibre characteristics, including fibres which may be hardened, plasticised, and/or stabilised against moisture, biodegradation or fire. The embedded silica may also be used as sites of attachment for coupling agents to modify the hydrophilic character of the fibre or to functionalise the fibre surface.

year: 2008

title: Association of Single Nucleotide Polymorphisms in Surfactant Protein A and D with Otitis Media.

authors: Barnett, Catherine Margaret Eleanor

subjects: otitis media

abstract: Otitis Media is one of the most common childhood diseases. Recurrent acute otitis media RAOM is characterized by repeated episodes of inflammation of the middle ear in conjunction with middle ear fluid, and often with an inflamed or bulging eardrum. Defective clearance by the Eustachian tube results in mucus build-up and is characteristic of otitis media with effusion (OME). Streptococcus pneumoniae, Haemophilus influenzae, Moraxella catarrhalis, respiratory syncytial virus, and rhinovirus are the most common contributors to otitis media pathogenesis. In New Zealand, OME has been implicated with conductive hearing loss in childhood and has been shown to significantly impact on speech and language development. New Zealand Māori and Polynesian children have displayed significantly higher hearing test failure rates than European-Caucasian children.

The collectins, Surfactant Protein (SP)-A and -D are encoded by three genes (SP-A1, SP-A2, and SP-D) and are host defense proteins present in the middle ear and Eustachian tube. Single nucleotide polymorphisms (SNPs) in SP-A1 and SP-A2 have been associated with increased or decreased susceptibility to otitis media, meningococcal disease, and range of respiratory diseases. Using allele-specific primers and real-time PCR with SYBR  Green I melting curve analysis, four groups of individuals were genotyped for eleven SP-A1, SP-A2, and SP-D SNPs: European-Caucasian individuals with RAOM/OME; New Zealand Māori/Polynesian individuals with RAOM/OME; individuals with meningococcal disease; and a control group. The computer program, Haploview, was employed to perform χ2 analyses and identify statistically significant associations of alleles/haplotypes with RAOM/OME or meningococcal disease.

In the European-Caucasian population, two SP-A1 alleles, one SP-A2 allele, and four haplotypes (CGAGC, 1A3, 1A9, and 1A10) were found to be associated with increased risk of RAOM/OME (P lt; 0.05). Conversely, haplotypes 6A2 and 1A2 were found to be protective against susceptibility to RAOM/OME (P lt; 0.05). In New Zealand Māori and Polynesian individuals, two SP-A1 alleles, three SP-A2 alleles, one SP-D allele, and four haplotypes (6A8, 6A10, 1A3, and 1A10) were found to be associated with increased risk of RAOM/OME (P lt; 0.05). An additional four haplotypes (6A2, 1A0, 1A2, and TA) were determined to be protective against susceptibility to RAOM/OME (P lt; 0.05). However, protective SPA1/SPA2/SPD haplotype 6A2-1A0-TA was significantly under-represented in the New Zealand Māori and Polynesian population (P lt; 0.05). A single allele and haplotype were associated with increased risk of meningococcal disease (P lt; 0.05).

The findings of this study confirm that specific genetic variants of SP-A and SP-D are associated with either increased or decreased risk of developing RAOM and/or OME. Furthermore, it was demonstrated that New Zealand Māori and Polynesian individuals appear to exhibit more haplotypes susceptible to RAOM/OME. This may provide a partial explanation for the higher RAOM/OME-related failure rates of hearing tests in New Zealand Māori and Polynesian children. However, there are numerous socio-economic and environmental factors that also contribute to otitis media pathogenesis which were not considered in this study. The effects of the SP-A1, SP-A2, and SP-D alleles and haplotypes on the bacterial/viral binding efficiencies of SP-A and SP-D need to be investigated by further research, using a large population, to confirm the association with susceptibility or resistance with RAOM/OME.

text: Association of Single Nucleotide Polymorphisms in Surfactant Protein A and D with Otitis Media. Otitis Media is one of the most common childhood diseases. Recurrent acute otitis media RAOM is characterized by repeated episodes of inflammation of the middle ear in conjunction with middle ear fluid, and often with an inflamed or bulging eardrum. Defective clearance by the Eustachian tube results in mucus build-up and is characteristic of otitis media with effusion (OME). Streptococcus pneumoniae, Haemophilus influenzae, Moraxella catarrhalis, respiratory syncytial virus, and rhinovirus are the most common contributors to otitis media pathogenesis. In New Zealand, OME has been implicated with conductive hearing loss in childhood and has been shown to significantly impact on speech and language development. New Zealand Māori and Polynesian children have displayed significantly higher hearing test failure rates than European-Caucasian children.

The collectins, Surfactant Protein (SP)-A and -D are encoded by three genes (SP-A1, SP-A2, and SP-D) and are host defense proteins present in the middle ear and Eustachian tube. Single nucleotide polymorphisms (SNPs) in SP-A1 and SP-A2 have been associated with increased or decreased susceptibility to otitis media, meningococcal disease, and range of respiratory diseases. Using allele-specific primers and real-time PCR with SYBR  Green I melting curve analysis, four groups of individuals were genotyped for eleven SP-A1, SP-A2, and SP-D SNPs: European-Caucasian individuals with RAOM/OME; New Zealand Māori/Polynesian individuals with RAOM/OME; individuals with meningococcal disease; and a control group. The computer program, Haploview, was employed to perform χ2 analyses and identify statistically significant associations of alleles/haplotypes with RAOM/OME or meningococcal disease.

In the European-Caucasian population, two SP-A1 alleles, one SP-A2 allele, and four haplotypes (CGAGC, 1A3, 1A9, and 1A10) were found to be associated with increased risk of RAOM/OME (P lt; 0.05). Conversely, haplotypes 6A2 and 1A2 were found to be protective against susceptibility to RAOM/OME (P lt; 0.05). In New Zealand Māori and Polynesian individuals, two SP-A1 alleles, three SP-A2 alleles, one SP-D allele, and four haplotypes (6A8, 6A10, 1A3, and 1A10) were found to be associated with increased risk of RAOM/OME (P lt; 0.05). An additional four haplotypes (6A2, 1A0, 1A2, and TA) were determined to be protective against susceptibility to RAOM/OME (P lt; 0.05). However, protective SPA1/SPA2/SPD haplotype 6A2-1A0-TA was significantly under-represented in the New Zealand Māori and Polynesian population (P lt; 0.05). A single allele and haplotype were associated with increased risk of meningococcal disease (P lt; 0.05).

The findings of this study confirm that specific genetic variants of SP-A and SP-D are associated with either increased or decreased risk of developing RAOM and/or OME. Furthermore, it was demonstrated that New Zealand Māori and Polynesian individuals appear to exhibit more haplotypes susceptible to RAOM/OME. This may provide a partial explanation for the higher RAOM/OME-related failure rates of hearing tests in New Zealand Māori and Polynesian children. However, there are numerous socio-economic and environmental factors that also contribute to otitis media pathogenesis which were not considered in this study. The effects of the SP-A1, SP-A2, and SP-D alleles and haplotypes on the bacterial/viral binding efficiencies of SP-A and SP-D need to be investigated by further research, using a large population, to confirm the association with susceptibility or resistance with RAOM/OME.

year: 2007