Robust sensory systems for dynamic environments
Creators: name: DELANEY, James
Publisher: name: RMIT University
Year: 2024
Subjects: classificationCode: 460608
subject: Mobile computing
classificationCode: 460609
subject: Networking and communications

description: Communication is the fundamental process by which information can be conveyed between entities, whether they are human, digital systems, or the like. The ability to coordinate and collaborate the efforts of multiple entities to achieve a common goal makes communication such a powerful and indispensable tool. Without it, the civilisation we have today would be non-existent and impossible to achieve. 

The collaborative and coordinated efforts of human entities operating in emergency rescue, disaster relief, and tactical environments are underpinned by effective communication. In these scenarios, on-the-ground situations can be highly dynamic and characterised by frequent and significant changes in the environment, goals or tasks, and the presence of adversarial actors. Thus, a high level of situational awareness and frequently updated information being available to commanders and operations control centres, regarding these environments and the personnel operating in them, is imperative for ensuring successful missions. 

To achieve this level of situational awareness, field sensing devices can convey information such as location, health and usage monitoring of equipment, environmental and atmospheric conditions, operating personnel vital signs, and data from audio or visual sensors. Relaying this information from devices attached or carried by operators, in most cases, must be achieved via wireless data links to a remote command and control centre. While satellite and other long-range, high-powered wireless technologies can easily transmit this information, field devices often have constraints on size, power, and weight. This means that lower-power, and consequently shorter range, wireless solutions have to be utilised for communicating via intermediary access points deployed atop deployment vehicles or temporary communication infrastructure. However, conditions in the situated environments can make wireless communication difficult through high node mobility and varied or unknown physical environment structure and characteristics. As such, robust and reliable links are essential for successful sensory and situational awareness communications in these scenarios.

In this thesis, we focus on the wireless links between devices equipped by deployed operating personnel and intermediary access point infrastructure. We use multi-radio (MR) platforms, wireless devices with multiple and diverse transceiver technologies, to produce robust and reliable links through adaptive control of available transceivers. Primarily, we study the application of intelligent control techniques, namely reinforcement learning (RL), for adaptive control over diverse transceivers to achieve the desired robustness and reliability under dynamic environmental conditions. 

The first part of this research develops a modelling and simulation approach for MR platforms and dynamic wireless communication environments, enabling further investigation of the proposed intelligent control techniques. Following this, we conduct an initial investigation by applying RL alongside other control methods to a MR problem; emulating the environments and conditions in emergency, disaster relief, and tactical scenarios. This initial work forms a benchmarking exercise where we compare alternative approaches and solution methodologies in terms of implementation and design efforts, communication performance, and online radio control policies. In doing so, we identify critical considerations for RL-based control in terms of both practical implementation and system design, including a comparison between exploration strategies and their impact on online performance. We show that the applied RL-based methods require less domain-specific design and utilise a simplified implementation framework to achieve favourable online radio selection and power control performance over existing methods. Furthermore, the simplified implementation of RL-based methods places a lower computational burden on constrained MR devices deployed in the field. 

The third aspect of this research further investigates the inherent multi-objective nature of wireless communications and the potential abilities of MR platforms to address this, using multi-objective RL as a control technique. We apply an explicit multi-objective framework, propose an extension to an existing algorithm to implement an adaptive exploration strategy, and investigate the range of potential objective trade-offs using this framework. The online performance of the multi-objective RL control agents is compared to optimal agent behaviour under two exploration strategies. We also propose a method to quantify the online radio control performance of the MR system with respect to optimal agent behaviour, using it to compare between different RL frameworks and exploration methods, and verify parameter selection. This verifies the improvements in online performance using this multi-objective framework and initiates discussion around the importance of studying online behaviour for RL-based MR control agents in these scenarios. Finally, we implement the proposed multi-objective RL techniques in a hardware testbed and conduct data and performance validation experiments. The outcomes are discussed and a range of design recommendations for future work are proposed.