Miracle-dz/UniFAR-FTUs huggingface dataset:
The Robot in Our Path: Investigating the Perceptions of People with Motor Disabilities on Navigating Public Space Alongside Sidewalk Robots

Sidewalk robots are becoming increasingly common worldwide, yet their operation on public walkways presents challenges for pedestrians. This is especially true for people with motor disabilities (PWMD) who already manage obstacles such as inadequate ramps and public incivility. The addition of sidewalk robots could further intensify these difficulties, which poses an urgent need to examine how the design of sidewalk robots may influence the daily navigation experiences of PWMD. This poster illustrates findings from semi-structured interviews with ten PWMD, providing insights into their perspectives on the presence of sidewalk robots. The study uncovers potential conflicts in shared sidewalk use and the adaptive actions PWMD described needing to undertake in response. Interviewees raised concerns about whether the robots could accommodate the needs of PWMD, as compared to people walking on foot, and the repercussions of any shortcomings in this regard. Our research also examines tensions stemming from different robotic design choices, indicating the necessity for more accessible public robot designs. We further delve into PWMD’s interaction needs and modalities for routine operation and in the event of robot malfunction. As cities increasingly allow for the deployment of robots in public spaces, this work seeks to inform equitable design and deployment guidelines for sidewalk robots and calls for further research into the implications of the rise of public robots for the diverse populations that make up any given municipality.
Azimuth: Designing Accessible Dashboards for Screen Reader Users

Dashboards are frequently used to monitor and share data across a breadth of domains including business, finance, sports, public policy, and healthcare, just to name a few. The combination of different components (e.g., key performance indicators, charts, filtering widgets) and the interactivity between components makes dashboards powerful interfaces for data monitoring and analysis. However, these very characteristics also often make dashboards inaccessible to blind and low vision (BLV) users. Through a co-design study with two screen reader users, we investigate challenges faced by BLV users and identify design goals to support effective screen reader-based interactions with dashboards. Operationalizing the findings from the co-design process, we present a prototype system, Azimuth, that generates dashboards optimized for screen reader-based navigation along with complementary descriptions to support dashboard comprehension and interaction. Based on a follow-up study with five BLV participants, we showcase how our generated dashboards support BLV users and enable them to perform both targeted and open-ended analysis. Reflecting on our design process and study feedback, we discuss opportunities for future work on supporting interactive data analysis, understanding dashboard accessibility at scale, and investigating alternative devices and modalities for designing accessible visualization dashboards.
MMT-GD: Multi-Modal Transformer with Graph Distillation for Cross-Cultural Humor Detection

In this paper, we present a solution for the Cross-Cultural Humor Detection (MuSe-Humor) sub-challenge, which is part of the Multimodal Sentiment Analys Challenge (MuSe) 2023. The MuSe-Humor task aims to detect humor from multimodal data, including video, audio, and text, in a cross-cultural context. The training data consists of German recordings, while the test data consists of English recordings. To tackle this sub-challenge, we propose a method called MMT-GD, which leverages a multimodal transformer model to effectively integrate the multimodal data. Additionally, we incorporate graph distillation to ensure that the fusion process captures discriminative features from each modality, avoiding excessive reliance on any single modality. Experimental results validate the effectiveness of our approach, achieving an Area Under the Curve (AUC) score of 0.8704 on the test set and securing the third position in the challenge.
Toward Programming Languages for Reasoning: Humans, Symbolic Systems, and AI Agents

Integration, composition, mechanization, and AI assisted development are the driving themes in the future of software development. At their core these concepts are rooted in the increasingly important role of computing in our world, the desire to deliver functionality faster, with higher quality, and to empower more people to benefit from programmatic automation. These themes, and how they impact the human developers driving them, are the foundations for the next generation of programming languages. At first glance the needs of mechanization tools, AI agents, and human developers along with the various goals around development velocity, software quality, and software democratization are a broad and seemingly diverse set of needs. However, at their core is a single challenge that, once resolved, enables us to make radical progress in all of these areas. Our hypothesis is that, fundamentally, software development is a problem of reasoning about code and semantics. This is true for human developers implementing a feature, symbolic tools building models of application behaviour, and even for language based AI agents as they perform tasks. While the particular aspects of reasoning that each agent struggles with varies to some degree, they share many common themes and, surprisingly, most mainstream languages extensively employ (anti)features that make this task harder or infeasible! This paper proposes a novel approach to this challenge – instead of new language features or logical constructs, that add more complexity to what is already a problem of complexity, we propose radical simplification in the form of the Bosque platform and language.
Deploying VizLens: Characterizing User Needs, Preferences, and Challenges of Physical Interfaces Usage in the Wild

Blind or Visually Impaired (BVI) people often encounter flat, inaccessible interfaces. Current solutions lack cost-effectiveness, portability, and robustness in real-world settings. We introduce VizLens, a fully-automated, full-stack mobile application powered by computer vision algorithms. The system is deployed and publicly available through the Apple App Store (https://vizlens.org/). From May to August 2023, we had 665 users, who uploaded 1,320 interface images. We aim to use it to study usage patterns and possible challenges BVI users may encounter with flat interfaces through a large-scale study in real-world settings. With in-depth analysis of user data and activity logs, our study will provide insights into BVI users’ interface interests, preferred assistance modes, and potential challenges due to system limitations or users’ diverse abilities. Our goal is to enhance the understanding of how BVI users interact with inaccessible, flat interfaces, and inform future assistive technology design.
JTMA: Joint Multimodal Feature Fusion and Temporal Multi-head Attention for Humor Detection

In this paper, we propose a model named Joint multimodal feature fusion and Temporal Multi-head Attention (JTMA) to solve the MuSe-Humor sub-challenge in Multimodal Sentiment Analysis Challenge 2023. The goal of MuSe-Humor sub-challenge is to predict whether humor occurs in the given dataset that includes data from multiple modalities (e.g., text, audio and video). The cross-cultural testing presents a new challenge that makes it different from the previous years. To solve the above problems, the proposed model JTMA firstly uses a 1-D CNN to aggregate temporal information within the unimodal feature. Then the interactions of inter-modality and intra-modality are performed by the multimodal feature encoder module. Finally, we integrate the high-level representations learned from multiple modalities to accurately predict humor. The effectiveness of our proposed model is demonstrated through experimental results obtained on the official test set. Our model achieves an impressive AUC score of 0.8889, surpassing the performance of all other participants in the competition, and securing the Top 1 ranking.
On the Applicability of Annotation-Based Source Code Modification in Kotlin (Work in Progress)

Annotations add metadata to source code entities such as classes or functions, which later can be processed by so-called annotation processors to, for example, modify the annotated program’s source code. While annotation processing has been well-explored in Java, the Kotlin community still lacks a comprehensive summary. Thus, in this paper, we summarize the main approaches available in Kotlin: (1) Compile-time annotation processing using (a) Kotlin Annotation Processing Tool (KAPT), (b) Kotlin Symbolic Processing (KSP), or (c) writing a custom Kotlin Compiler plugin; as well as (2) load-time code modification using an agent or a custom class loader. We provide proof-of-concept implementations, discuss advantages and disadvantages, and specifically focus on how well each approach supports modifying the annotated source code. This should help developers and researchers to better decide when to use which approach.
Divergent tumor and immune cell reprogramming underlying immunotherapy response and immune-related adverse events in lung squamous cell carcinoma

Background Lung squamous cell carcinoma (LUSC) remains a leading cause of cancer-related deaths with few therapeutic strategies. Immune checkpoint inhibitors (ICIs) have demonstrated promising efficacy in patients with LUSC. However, ICIs could also lead to a unique spectrum of immune-related adverse events (irAEs), which dampen the clinical outcome. In-depth characterization of the immune hallmarks of antitumor responses and irAEs remains an unmet need to maximize ICI-treatment benefits of patients. Methods We performed single-cell RNA sequencing (scRNA-seq) on pre-ICI and on-ICI treatment tumor biopsies. We used bulk RNA-seq data of matched pretreatment/on-treatment tumors and irAE affected organs to validate observations from scRNA-seq analysis. Two independent patient cohorts were collected to determine circulating tumor necrosis factor (TNF) protein expression levels. Results We found that increased proportions of a macrophage subcluster with highly expressed secreted phosphoprotein 1 (SPP1) and two tumor cell subclusters in irAE patients, whereas proportions of two cytotoxic CD8+ T cell subclusters were higher in patients with partial response (PR). TNF signaling pathway was conversely associated with treatment efficacy and irAE development in most macrophage and tumor cell subclusters. Cell–cell communications for TNF ligand-receptor pairs between macrophage/T cells and tumor cells were also bidirectionally remodeled in responders versus non-responders and irAE versus non-irAE patients. Bulk RNA-seq analysis on matched pretreatment/on-treatment tumors and irAE affected organs revealed remarkably enhanced macrophage abundance and TNF signaling pathway in on-treatment tumors and organs developed irAEs. Furthermore, we observed significantly increased circulating TNF protein in plasma or serum of irAE patients but not ICI responders, based on analysis of two independent LUSC patient cohorts and one published ICI patient cohort. Conclusions Our data depicts specific reprogramming of macrophage, T cells and tumor cells associated with ICI response and irAEs, elucidates divergent roles of TNF signaling in antitumor immunity and irAEs, and highlights the significance of TNF expression in irAE development in the LUSC setting.
A Scalable Real-time Semantic Segmentation Network for Autonomous Driving

Computational resources are limited on real-time embedded devices, so the available computing cost of deployment on the target platform must be considered. We develop a feature extraction module based on the MobileNet backbone that can be adjusted in terms of computational complexity and capacity using the depth multiplier parameter, classifier depth, and kernel depth. These three parameters allow us to control the count of channels within the network, effectively managing the model's capacity and computational requirements. To achieve semantic segmentation, we incorporate additional components, such as an extension module. This extension module typically includes 1x1 pointwise convolutional layers for pixel-level classification and a transposed convolutional layer for upsampling the output to the original input image size. By combining the feature extraction module with this extension module, we can create a complete architecture capable of performing semantic segmentation tasks. The feature extraction module provides the initial feature extraction and the extension module adds the necessary components for accurate pixel-wise classification and upsampling. Compared to Hardware-aware Neural Architecture Search (NAS), pruning, runtime pruning, and knowledge distillation methods, our model has several advantages in terms of modular design, structural controllability, ease of implementation, and cost-effectiveness. Our computational efficiency, as measured by FLOPS, is highly competitive. Our method is distinguished by solving the problem of MobileNet's inability to adjust the size and number of convolution kernels. It achieves this through adaptable parameter tuning, including MobileNet's depth multiplier, the kernel size in the FCN head's Separable Convolution layer, and the depth of the first Point-wise Convolution layer. These adjustments are customized to match hardware's max multiply-accumulates (MACs), optimizing network capacity and maximizing resource utilization.
KOGI: A Seamless Integration of ChatGPT into Jupyter Environments for Programming Education

The impact of ChatGPT has brought both anxiety and anticipation to schools and universities. Exploring a positive method to improve programming skills with ChatGPT is a new and pressing challenge. In pursuit of this goal, we have developed KOGI, a learning support system that integrates ChatGPT into the Jupyter environment. This paper demonstrates how KOGI enables students to receive timely advice from ChatGPT in response to errors and other questions they encounter. We immediately introduced KOGI in our two introductory courses: Algorithms and Data Science. The introduction of KOGI resulted in a significant decrease in the number of unresolved student errors. In addition, we report on student trends observed in the classroom regarding the type and frequency of help requested. Although our findings are preliminary, they are informative for programming instructors interested in using ChatGPT.
Cooperative Spectrum Sensing with Deep Q-Network for Multimedia Applications

With the gradually stricter requirement for multimedia applications, spectrum inefficiencies are urgent to be relieved by sensing and utilizing Spectrum Holes (SHs) over a wide spectrum. Cognitive Radio Sensor Network (CRSN) has drawn a lot of attention, which determines the state of Primary Users (PUs) by implementing Cooperative Spectrum Sensing (CSS), further overcoming various noise and fading issues in the radio environment. A survey on the application of Reinforcement Learning (RL) technology for CSS is conducted, especially through handling the performance optimization problem that cannot be achieved by traditional methods. Specifically, we transformed the traditional Fusion Center (FC) into an intelligent Agent that is responsible for making fusion decisions based on the results of Energy Detection (ED) technology. In this way, through learning from experience, the system performance in global probabilities can be improved by making fusion decisions as accurately as possible. Compared with traditional methods, comparison studies demonstrate the effectiveness of the proposed method in improving the CSS system performances, as well as its robustness in the face of various environments. The combination and complement of the traditional and the proposed scheme are also suggested in this paper.
Towards Safe HPC: Productivity and Performance via Rust Interfaces for a Distributed C++ Actors Library (Work in Progress)

In this work-in-progress research paper, we make the case for using Rust to develop applications in the High Performance Computing (HPC) domain which is critically dependent on native C/C++ libraries. This work explores one example of Safe HPC via the design of a Rust interface to an existing distributed C++ Actors library. This existing library has been shown to deliver high performance to C++ developers of irregular Partitioned Global Address Space (PGAS) applications. Our key contribution is a proof-of-concept framework to express parallel programs safe-ly in Rust (and potentially other languages/systems), along with a corresponding study of the problems solved by our runtime, the implementation challenges faced, and user productivity. We also conducted an early evaluation of our approach by converting C++ actor implementations of four applications taken from the Bale kernels to Rust Actors using our framework. Our results show that the productivity benefits of our approach are significant since our Rust-based approach helped catch bugs statically during application development, without degrading performance relative to the original C++ actor versions.
Building the Habit of Authoring Alt Text: Design for Making a Change

Alternative (alt) text is essential for the accessibility of images to screen reader users. Alt text can be authored by end users or generated automatically using machine learning. Automated methods, while fast and cheap, do not result in high quality alt text. Meanwhile, the large number of images with no alt text on social media indicates that many users do not author alt text for the images they upload. Little is known about whether design elements that motivate and support users are integrated into alt text user interfaces (UIs). We reviewed the current state of alt text UIs in literature, and characterized their design according to behavior change and accountability design factors. We found that the majority of reviewed UIs contained general prompts with no specific motivator or guidance. Many of the UIs did not incorporate design elements to encourage accountability or behavior change. We recommend that future alt text authoring tools incorporate design elements which decrease the user’s cognitive effort, employ motivational and well-timed triggers, and leverage user accountability through alt text visibility and its association with the user’s social presence.
A Gallery In My Hand: A Multi-Exhibition Investigation of Accessible and Inclusive Gallery Experiences for Blind and Low Vision Visitors

With their overwhelming reliance on visual arts and artefacts, art galleries and museums are often considered unapproachable spaces for people who are blind or have low vision (BLV). We present the results of an ongoing collaboration with an Australian regional gallery in their efforts to improve access by BLV people to their permanent collection and temporary exhibitions. We describe the approach taken to develop interpretive technology-mediated experiences for BLV visitors for their 2021 and 2022 flagship exhibitions, along with evaluation from the perspectives of both BLV visitors and the gallery staff. Outcomes include: broader awareness within the gallery for inclusive design practice and public engagement; better understandings of the needs of BLV visitors, in particular relating to confidence building and technology integration; understanding of gallery constraints and challenges; and a framework for the development of future inclusive gallery experiences.
Unified Shared Memory: Friend or Foe? Understanding the Implications of Unified Memory on Managed Heaps

Adopting heterogeneous execution on GPUs and FPGAs in managed runtime systems, such as Java, is a challenging task due to the complexities of the underlying virtual machine. The majority of the current work has been focusing on compiler toolchains to solve the challenge of transparent just-in-time compilation of different code segments onto the accelerators. However, apart from providing automatic code generation, another paramount challenge is the seamless interoperability between the host memory manager and the Garbage Collector (GC). Currently, heterogeneous programming models that run on top of managed runtime systems, such as Aparapi and TornadoVM, need to block the GC when running native code (e.g, JNI code) in order to prevent the GC from moving data while the native code is still running on the hardware accelerator. To tackle the inefficacy of locking the GC while the GPU operates, this paper proposes a novel Unified Memory (UM) memory allocator for heterogeneous programming frameworks for managed runtime systems. In this paper, we show how, by providing small changes to a Java runtime system, automatic memory management can be enhanced to perform object reclamation not only on the host, but also on the device. This is done by allocating the Java Virtual Machine's object heap in unified memory which is visible to all hardware accelerators. In this manner -although explicit data synchronization between the host and the device is still required to ensure data consistency- we enable transparent page migration of Java heap-allocated objects between the host and the accelerator, since our UM system is aware of pointers and object migration due to GC collections. This technique has been implemented in the context of MaxineVM, an open source research VM for Java written in Java. We evaluated our approach on a discrete and an integrated GPU, showcasing under which conditions UM can benefit execution across different benchmarks and configurations.We concluded that when hardware acceleration is not employed, UM does not pose significant overheads unless memory intensive workloads are encountered which can exhibit up to 12% (worst case) and 2% (average) slowdowns. In addition, if hardware acceleration is used, UM can achieve up to 9.3x speedup compared to the non-UM baseline implementation for integrated GPUs.
Scalable Spreadsheet-Driven End-User Applications with Incremental Computation

Spreadsheets are a popular programming model among both novice and advanced users. It allows for quick development of computational tasks over the user's data set. It has also been suggested as a model for lightweight, low-code development of small-scale personal and collaborative applications for managing data. A barrier is hit when trying to scale up these applications: as the volume of the data grows, performance of spreadsheet computations degrades noticeably, making their maintenance more difficult and frustrating. This paper explores one aspect in which spreadsheets can be boosted significantly - by making computations more incremental, such that small changes to the data would incur a respectively small cost of update. This is particularly interesting in the context of aggregate operations - applied to sets of data elements, producing a single value, such as their sum or average. We propose a compilation phase in which a compiler transforms a spreadsheet table into a program that accepts a stream of changes and produces a stream of updates. We show how to adapt classical results from finite differencing to this interesting modern scenario, and demonstrate that it yields asymptotic improvements to resource usage.