# In-Home Evaluation of the Neo Care Artificial Intelligence Sound-Based Fall Detection System
## 4 Discussion  
This study rigorously evaluated the Neo Care Monitoring System, examining both its accuracy in real-world settings and its user experience, including perceived benefits and drawbacks and future utility. The system was frequently activated, but showed only moderate accuracy, primarily due to difficulties in differentiating everyday sounds from noises that signified genuine concerns, such as falls. User interviews reinforced the need for better calibration to improve this accuracy. The Neo's strengths included providing a sense of security and serving as a wearable-free safety monitoring alternative. However, the system also faced significant challenges, including misinterpreting everyday sounds, failure to detect some falls, and experiencing technical issues, like going offline. Users suggested several improvements, such as adding visual cues for hearing impaired users, voice-activated triggers, and an outdoor version. The majority of participants stated they would be willing to continue using the Neo after these adjustments and saw its potential utility in aged care facilities. Our study found that the Neo had a relatively high sensitivity of 75% for detecting falls, but struggled with low specificity, leading to frequent false positives. While high sensitivity is essential for not missing critical events like falls, it often came at the expense of specificity, causing the system to frequently misinterpret normal activities as falls. These findings are notably similar to those of [PERSON] et al. (2018), who studied a tri-axial accelerometer wearable in 19 older adults under free-living conditions. They reported an 80% sensitivity rate, detecting 8 out of 10 actual falls, but also had a high false positive rate of up to 0.3 false alarms per hour of wear. On the other hand, [PERSON] et al. 's study with 18 participants using an accelerometer in free-living conditions reported a low sensitivity of 25% but a high specificity of 91% ([PERSON] et al., 2018), underscoring the trade-off between sensitivity and specificity in monitoring devices. Two previous studies also evaluated fall detection using sound signals ([PERSON] et al., 2018) or smart speakers ([PERSON] et al., 2018). A previous study that evaluated a novel system utilizing both passive and active acoustic signals, implemented with a multi-modal classification framework, achieved 99% accuracy in typical conditions and over 90% accuracy in noisy or non-line-of-sight environments ([PERSON] et al., 2018). [PERSON] et al. (2018) evaluated a wearable device worn on the chest and a smart speaker or IoT device for confirmatory purposes. The results demonstrated that the system exhibited high sensitivity (0.94-0.96), specificity (0.95-0.97), and accuracy (0.95-0.96) in accurately detecting falls that occurred during transitions from sitting to standing, standing to sitting, and while in a standing or seated position. However, this was a lab-based study and not conducted in a real-world home setting. Interestingly, our study observed a notably lower performance compared with previous research. For example, specificity in the present study was 6.8%, suggesting potential variations in device performance and user population. Interpreting this lower specificity rate may imply that the Neo could be more effective in accurately identifying fall-related events while minimizing unnecessary alerts, potentially improving user experience and caregiver burden. Our study addresses the growing call for real-world testing of fall detection systems. Interestingly, most studies of fall detection systems are conducted in laboratory settings, typically with young participants who simulate falls. Highlighting this point, a systematic review of fall detection devices identified 92 studies, of which just 7% involved older adults being monitored in a real-world setting [31]. When tested under artificial conditions, the studies tend to reach far favorable results. For example, the same system described in the [PERSON] et al.