title: Improving the Evaluation of Network Anomaly Detection Using a Data
Fusion Approach

authors: Löf, Andreas

subjects: network anomaly detection

abstract: Currently, the evaluation of network anomaly detection methods is often not repeatable. It is difficult to ascertain if different implementations of the same methods have the same performance or the relative performance of different methods. This is in part due to a lack of open implementations, the absence of recent datasets and no common format to express results.

A common approach to evaluating a method is to use the Defense Advanced Research Projects Agency (DARPA) 1999 datasets, or a derivative of them, in combination with a different dataset or network capture. The DARPA datasets are relatively old and bear little resemblance to modern day traffic and the other datasets are unlabelled and typically publicly unavailable making it difficult to ascertain the validity of the research evaluated in such a way.

This thesis primarily contributes a new evaluation methodology that uses a data fusion based approach that allows for reproducible evaluations with modern datasets.

The new methodology incorporates three other contributions: A new way to capture network traces that are fully anonymised yet retains more information than any current network traces and a new trace annotation format and a method for verifying the correctness of the annotations.

The DARPA 1999 dataset was used to demonstrate the validity of the approach and an evaluation was performed on a new dataset that has been captured using the methods introduced. In the evaluation we find that methodology is a viable approach forward, but that it comes with a different set of drawbacks than the current state of the art.

text: Improving the Evaluation of Network Anomaly Detection Using a Data
Fusion Approach Currently, the evaluation of network anomaly detection methods is often not repeatable. It is difficult to ascertain if different implementations of the same methods have the same performance or the relative performance of different methods. This is in part due to a lack of open implementations, the absence of recent datasets and no common format to express results.

A common approach to evaluating a method is to use the Defense Advanced Research Projects Agency (DARPA) 1999 datasets, or a derivative of them, in combination with a different dataset or network capture. The DARPA datasets are relatively old and bear little resemblance to modern day traffic and the other datasets are unlabelled and typically publicly unavailable making it difficult to ascertain the validity of the research evaluated in such a way.

This thesis primarily contributes a new evaluation methodology that uses a data fusion based approach that allows for reproducible evaluations with modern datasets.

The new methodology incorporates three other contributions: A new way to capture network traces that are fully anonymised yet retains more information than any current network traces and a new trace annotation format and a method for verifying the correctness of the annotations.

The DARPA 1999 dataset was used to demonstrate the validity of the approach and an evaluation was performed on a new dataset that has been captured using the methods introduced. In the evaluation we find that methodology is a viable approach forward, but that it comes with a different set of drawbacks than the current state of the art.

year: 2013

title: Effect of footwear and overhead goal on the Landing Error Scoring System

authors: Boswell-Smith, Caleb

subjects: Anterior Cruciate Ligament

abstract: A large proportion of the population undertakes sport and exercise either for recreation or competition. However, with increased participation in sport and exercise, the occurrence of sport-related injuries also increases. Anterior Cruciate Ligament (ACL) injuries are injuries to the knee and one of the most common injuries in a sporting environment that occur from noncontact mechanisms. The Landing Error Scoring System (LESS) is a common screening tool used to identify individuals who are more at risk of sustaining an ACL injury based on double-leg jump-landing (DLJL) kinematics. Despite the LESS demonstrating good-toexcellent reliability and predictive value, the LESS is sensitive to various factors (e.g., protocol alterations) and has been criticised for lacking sport specificity. Despite studies conducting the LESS both with and without footwear and using an overhead goal during jumping tasks to increase sport specificity, to date, there is no research comparing LESS outcomes based on the presence/absence of footwear or presence/absence of an overhead goal.
Chapter One summarises literature surrounding sports injuries, ACL injuries, injury aetiology, injury prevention models, and injury risk screening tools used in sports with a focus on the LESS. Footwear and cueing in sports are also addressed. Chapter Two is experimental and compares overall LESS scores, group- and individual-level injury risk categorisation, specific LESS errors, and jump heights between footwear and no footwear conditions. Chapter Three is experimental and compares these same outcome measures between overhead goal and no goal conditions. Chapter Four summarises the key findings from the two experimental chapters, and addresses the limitations, strengths, and future research directions arising from this Thesis.
In Chapter Two, 80 participants (55% male) performed a DLJL task where they landed from a 30-cm high box to 50% of their own body height and immediately jumped vertically for maximum height for LESS assessment. Participants completed three trials under two randomordered conditions: with and without footwear. Group mean LESS scores were greater (0.3 errors, p = 0.022) and jump heights were lower (0.6 cm, p = 0.029) in the footwear than barefoot condition, but differences were trivial and not clinically meaningful. Although the number of high-risk participants was comparable between groups (p = 1.000), categorisation was inconsistent for 16.25% of individuals and the occurrence of four of the 17 specific landing errors significantly differed between conditions. Based on the study findings, footwear does not appear to meaningfully influence mean LESS scores, risk categorisation, or jump height at a group level. At an individual level, footwear can affect risk categorisation and landing strategies. Hence, use of a consistent protocol is recommended in clinical setting, and use of footwear is advised for assessing injury risk given the predictive value of the LESS barefoot is unknown.
In Chapter Three, 76 participants (51% male) performed a DLJL task where they landed from a 30-cm high box to 50% of their own body height and immediately jumped vertically for maximum height for LESS assessment. Participants completed three trials under two randomordered conditions: with and without an overhead goal. Mean LESS scores were greater (0.3 errors, p < 0.001) with the overhead goal, but this small difference was not clinically meaningful. Similarly, although the number of high-risk participants was significantly greater with the overhead goal (p = 0.039), the 9.2% difference was trivial. Participants jumped 2.7 cm higher with the overhead goal (p < 0.001) without affecting the occurrence of any specific LESS errors. Based on the results found, performing the LESS with an overhead goal enhances sport specificity and elicits greater vertical jump performances with minimal change in landing errors and risk categorisation. Adding an overhead goal to LESS might enhance its suitability for injury risk screening, although the predictive value of LESS with an overhead goal needs confirmation.

text: Effect of footwear and overhead goal on the Landing Error Scoring System A large proportion of the population undertakes sport and exercise either for recreation or competition. However, with increased participation in sport and exercise, the occurrence of sport-related injuries also increases. Anterior Cruciate Ligament (ACL) injuries are injuries to the knee and one of the most common injuries in a sporting environment that occur from noncontact mechanisms. The Landing Error Scoring System (LESS) is a common screening tool used to identify individuals who are more at risk of sustaining an ACL injury based on double-leg jump-landing (DLJL) kinematics. Despite the LESS demonstrating good-toexcellent reliability and predictive value, the LESS is sensitive to various factors (e.g., protocol alterations) and has been criticised for lacking sport specificity. Despite studies conducting the LESS both with and without footwear and using an overhead goal during jumping tasks to increase sport specificity, to date, there is no research comparing LESS outcomes based on the presence/absence of footwear or presence/absence of an overhead goal.
Chapter One summarises literature surrounding sports injuries, ACL injuries, injury aetiology, injury prevention models, and injury risk screening tools used in sports with a focus on the LESS. Footwear and cueing in sports are also addressed. Chapter Two is experimental and compares overall LESS scores, group- and individual-level injury risk categorisation, specific LESS errors, and jump heights between footwear and no footwear conditions. Chapter Three is experimental and compares these same outcome measures between overhead goal and no goal conditions. Chapter Four summarises the key findings from the two experimental chapters, and addresses the limitations, strengths, and future research directions arising from this Thesis.
In Chapter Two, 80 participants (55% male) performed a DLJL task where they landed from a 30-cm high box to 50% of their own body height and immediately jumped vertically for maximum height for LESS assessment. Participants completed three trials under two randomordered conditions: with and without footwear. Group mean LESS scores were greater (0.3 errors, p = 0.022) and jump heights were lower (0.6 cm, p = 0.029) in the footwear than barefoot condition, but differences were trivial and not clinically meaningful. Although the number of high-risk participants was comparable between groups (p = 1.000), categorisation was inconsistent for 16.25% of individuals and the occurrence of four of the 17 specific landing errors significantly differed between conditions. Based on the study findings, footwear does not appear to meaningfully influence mean LESS scores, risk categorisation, or jump height at a group level. At an individual level, footwear can affect risk categorisation and landing strategies. Hence, use of a consistent protocol is recommended in clinical setting, and use of footwear is advised for assessing injury risk given the predictive value of the LESS barefoot is unknown.
In Chapter Three, 76 participants (51% male) performed a DLJL task where they landed from a 30-cm high box to 50% of their own body height and immediately jumped vertically for maximum height for LESS assessment. Participants completed three trials under two randomordered conditions: with and without an overhead goal. Mean LESS scores were greater (0.3 errors, p < 0.001) with the overhead goal, but this small difference was not clinically meaningful. Similarly, although the number of high-risk participants was significantly greater with the overhead goal (p = 0.039), the 9.2% difference was trivial. Participants jumped 2.7 cm higher with the overhead goal (p < 0.001) without affecting the occurrence of any specific LESS errors. Based on the results found, performing the LESS with an overhead goal enhances sport specificity and elicits greater vertical jump performances with minimal change in landing errors and risk categorisation. Adding an overhead goal to LESS might enhance its suitability for injury risk screening, although the predictive value of LESS with an overhead goal needs confirmation.

year: 2022

title: PIWeCS: enhancing human/machine agency in an interactive composition system

authors: Whalley, Ian

abstract: This paper focuses on the infrastructure and aesthetic approach used in PIWeCS: a Public Space Interactive Web-based Composition System. The concern was to increase the sense of dialogue between human and machine agency in an interactive work by adapting Paine's (2002) notion of a conversational model of interaction as a ‘complex system’. The machine implementation of PIWeCS is achieved through integrating intelligent agent programming with MAX/MSP. Human input is through a web infrastructure. The conversation is initiated and continued by participants through arrangements and composition based on short performed samples of traditional New Zealand Maori instruments. The system allows the extension of a composition through the electroacoustic manipulation of the source material.

text: PIWeCS: enhancing human/machine agency in an interactive composition system This paper focuses on the infrastructure and aesthetic approach used in PIWeCS: a Public Space Interactive Web-based Composition System. The concern was to increase the sense of dialogue between human and machine agency in an interactive work by adapting Paine's (2002) notion of a conversational model of interaction as a ‘complex system’. The machine implementation of PIWeCS is achieved through integrating intelligent agent programming with MAX/MSP. Human input is through a web infrastructure. The conversation is initiated and continued by participants through arrangements and composition based on short performed samples of traditional New Zealand Maori instruments. The system allows the extension of a composition through the electroacoustic manipulation of the source material.

year: 2004