title: Reinforcement Learning for Racecar Control

authors: Cleland, Benjamin George

subjects: Artificial Intelligence

abstract: This thesis investigates the use of reinforcement learning to learn to drive a racecar in the simulated environment of the Robot Automobile Racing Simulator. Real-life race driving is known to be difficult for humans, and expert human drivers use complex sequences of actions. There are a large number of variables, some of which change stochastically and all of which may affect the outcome. This makes  driving  a promising domain for testing and developing Machine Learning techniques that have the potential to be robust enough to work in the real world. Therefore the principles of the algorithms from this work may be applicable to a range of problems.

The investigation starts by finding a suitable data structure to represent the information learnt. This is tested using supervised learning. Reinforcement learning is added and roughly tuned, and the supervised learning is then removed. A simple tabular representation is found satisfactory, and this avoids difficulties with more complex methods and allows the investigation to concentrate on the essentials of learning. Various reward sources are tested and a combination of three are found to produce the best performance. Exploration of the problem space is investigated. Results show exploration is essential but controlling how much is done is also important. It turns out the learning episodes need to be very long and because of this the task needs to be treated as continuous by using discounting to limit the size of the variables stored. Eligibility traces are used with success to make the learning more efficient. The tabular representation is made more compact by hashing and more accurate by using smaller buckets. This slows the learning but produces better driving. The improvement given by a rough form of generalisation indicates the replacement of the tabular method by a function approximator is warranted. These results show reinforcement learning can work within the Robot Automobile Racing Simulator, and lay the foundations for building a more efficient and competitive agent.

text: Reinforcement Learning for Racecar Control This thesis investigates the use of reinforcement learning to learn to drive a racecar in the simulated environment of the Robot Automobile Racing Simulator. Real-life race driving is known to be difficult for humans, and expert human drivers use complex sequences of actions. There are a large number of variables, some of which change stochastically and all of which may affect the outcome. This makes  driving  a promising domain for testing and developing Machine Learning techniques that have the potential to be robust enough to work in the real world. Therefore the principles of the algorithms from this work may be applicable to a range of problems.

The investigation starts by finding a suitable data structure to represent the information learnt. This is tested using supervised learning. Reinforcement learning is added and roughly tuned, and the supervised learning is then removed. A simple tabular representation is found satisfactory, and this avoids difficulties with more complex methods and allows the investigation to concentrate on the essentials of learning. Various reward sources are tested and a combination of three are found to produce the best performance. Exploration of the problem space is investigated. Results show exploration is essential but controlling how much is done is also important. It turns out the learning episodes need to be very long and because of this the task needs to be treated as continuous by using discounting to limit the size of the variables stored. Eligibility traces are used with success to make the learning more efficient. The tabular representation is made more compact by hashing and more accurate by using smaller buckets. This slows the learning but produces better driving. The improvement given by a rough form of generalisation indicates the replacement of the tabular method by a function approximator is warranted. These results show reinforcement learning can work within the Robot Automobile Racing Simulator, and lay the foundations for building a more efficient and competitive agent.

year: 2006

title: The role of the passenger in everyday driving: Understanding how passengers assist adult drivers

authors: Reader, Shawn Jason

abstract: Previous research has indicated that adult drivers are at a lower risk of crashes when accompanied by at least one passenger. One potential reason for this is that passengers can assist drivers on the road via expanding situation awareness, reducing cognitive load and encouraging responsible driving behaviour. Both passengers and drivers report that passengers can be helpful on the road. However, no study has investigated if and how passengers assist adult drivers during everyday driving. The objective of this study was to use passenger and driver conversation to examine ways passengers assist drivers, how what drivers ask for compares to what passengers provide unprompted and how driver and passenger perceptions compare to what actually happens on the road. Twenty drivers aged between 25 and 65 were recruited for this study, along with passengers that they regularly drove with. Participants were recorded completing an on-road navigational task, and subsequently interviewed about their views and experiences with passenger interaction. On the road, passengers did assist drivers by navigating, pointing out hazards, watching for traffic at intersections, expanding the driver’s situation awareness, performing secondary tasks on behalf of drivers and giving feedback or criticism of driver behaviour. While patterns of assistance varied over different driver-passenger pairs, passengers initiated a higher proportion of assistance on average than drivers. During the interview, participants identified that passenger assistance is highly dependent on contextual factors. Participants discussed several ways that passengers assist drivers during everyday driving including increasing driver situation awareness, keeping the driver awake and alert, encouraging responsible behaviour and performing secondary tasks on the driver’s behalf. This is the first study to demonstrate the various ways that passengers assist adult drivers on the road. Passenger assistance does occur during everyday driving and can help to explain the reduction in crash risk associated with driving with a passenger. Further exploration of these findings could further increase our understanding of how passengers assist drivers, what forms of assistance are beneficial and how passengers can be encouraged to be better co-drivers and increase road safety.

text: The role of the passenger in everyday driving: Understanding how passengers assist adult drivers Previous research has indicated that adult drivers are at a lower risk of crashes when accompanied by at least one passenger. One potential reason for this is that passengers can assist drivers on the road via expanding situation awareness, reducing cognitive load and encouraging responsible driving behaviour. Both passengers and drivers report that passengers can be helpful on the road. However, no study has investigated if and how passengers assist adult drivers during everyday driving. The objective of this study was to use passenger and driver conversation to examine ways passengers assist drivers, how what drivers ask for compares to what passengers provide unprompted and how driver and passenger perceptions compare to what actually happens on the road. Twenty drivers aged between 25 and 65 were recruited for this study, along with passengers that they regularly drove with. Participants were recorded completing an on-road navigational task, and subsequently interviewed about their views and experiences with passenger interaction. On the road, passengers did assist drivers by navigating, pointing out hazards, watching for traffic at intersections, expanding the driver’s situation awareness, performing secondary tasks on behalf of drivers and giving feedback or criticism of driver behaviour. While patterns of assistance varied over different driver-passenger pairs, passengers initiated a higher proportion of assistance on average than drivers. During the interview, participants identified that passenger assistance is highly dependent on contextual factors. Participants discussed several ways that passengers assist drivers during everyday driving including increasing driver situation awareness, keeping the driver awake and alert, encouraging responsible behaviour and performing secondary tasks on the driver’s behalf. This is the first study to demonstrate the various ways that passengers assist adult drivers on the road. Passenger assistance does occur during everyday driving and can help to explain the reduction in crash risk associated with driving with a passenger. Further exploration of these findings could further increase our understanding of how passengers assist drivers, what forms of assistance are beneficial and how passengers can be encouraged to be better co-drivers and increase road safety.

year: 2019

title: Interpreting & practicing kaupapa Māori research in a community setting: The in’s and out’s

authors: Thompson, Keri

subjects: Māori

abstract: Pou Tuia Rangahau is a unique community based research unit based within a kaupapa Māori organisation. Kaupapa Māori methodologies are utilised, with the importance of these methodologies being that Māori are defining the process, doing the research for and about Māori, with the eventual outcome being meaningful to Māori. This paper is placed within this wider context of Kaupapa Māori and how it applies to the practice of research in particular. We discuss how we interpret and practice Kaupapa Māori Research (KMR) within an urban community based organisation by highlighting a particular piece of research that was undertaken by Pou Tuia Rangahau, the Research Unit of Te Runanga O Kirikiriroa Trust Inc.

text: Interpreting & practicing kaupapa Māori research in a community setting: The in’s and out’s Pou Tuia Rangahau is a unique community based research unit based within a kaupapa Māori organisation. Kaupapa Māori methodologies are utilised, with the importance of these methodologies being that Māori are defining the process, doing the research for and about Māori, with the eventual outcome being meaningful to Māori. This paper is placed within this wider context of Kaupapa Māori and how it applies to the practice of research in particular. We discuss how we interpret and practice Kaupapa Māori Research (KMR) within an urban community based organisation by highlighting a particular piece of research that was undertaken by Pou Tuia Rangahau, the Research Unit of Te Runanga O Kirikiriroa Trust Inc.

year: 2008