OpenRaiser/Intern-Atlas huggingface dataset:
Revealing Weaknesses of Vietnamese Language Models Through Unanswerable Questions in Machine Reading Comprehension
Venue: arXiv cs.CL 2023
Year: 2023

Although the curse of multilinguality significantly restricts the language abilities of multilingual models in monolingual settings, researchers now still have to rely on multilingual models to develop state-of-the-art systems in Vietnamese Machine Reading Comprehension. This difficulty in researching is because of the limited number of high-quality works in developing Vietnamese language models. In order to encourage more work in this research field, we present a comprehensive analysis of language weaknesses and strengths of current Vietnamese monolingual models using the downstream task of Machine Reading Comprehension. From the analysis results, we suggest new directions for developing Vietnamese language models. Besides this main contribution, we also successfully reveal the existence of artifacts in Vietnamese Machine Reading Comprehension benchmarks and suggest an urgent need for new high-quality benchmarks to track the progress of Vietnamese Machine Reading Comprehension. Moreover, we also introduced a minor but valuable modification to the process of annotating unanswerable questions for Machine Reading Comprehension from previous work. Our proposed modification helps improve the quality of unanswerable questions to a higher level of difficulty for Machine Reading Comprehension systems to solve.
Xplainer: From X-Ray Observations to Explainable Zero-Shot Diagnosis
Venue: arXiv cs.CV 2023
Year: 2023

Automated diagnosis prediction from medical images is a valuable resource to support clinical decision-making. However, such systems usually need to be trained on large amounts of annotated data, which often is scarce in the medical domain. Zero-shot methods address this challenge by allowing a flexible adaption to new settings with different clinical findings without relying on labeled data. Further, to integrate automated diagnosis in the clinical workflow, methods should be transparent and explainable, increasing medical professionals' trust and facilitating correctness verification. In this work, we introduce Xplainer, a novel framework for explainable zero-shot diagnosis in the clinical setting. Xplainer adapts the classification-by-description approach of contrastive vision-language models to the multi-label medical diagnosis task. Specifically, instead of directly predicting a diagnosis, we prompt the model to classify the existence of descriptive observations, which a radiologist would look for on an X-Ray scan, and use the descriptor probabilities to estimate the likelihood of a diagnosis. Our model is explainable by design, as the final diagnosis prediction is directly based on the prediction of the underlying descriptors. We evaluate Xplainer on two chest X-ray datasets, CheXpert and ChestX-ray14, and demonstrate its effectiveness in improving the performance and explainability of zero-shot diagnosis. Our results suggest that Xplainer provides a more detailed understanding of the decision-making process and can be a valuable tool for clinical diagnosis.
Dual-Quaternion Interpolation
Venue: arXiv cs.CG 2023
Year: 2023

Transformations in the field of computer graphics and geometry are one of the most important concepts for efficient manipulation and control of objects in 2-dimensional and 3-dimensional space. Transformations take many forms each with their advantages and disadvantages. A particularly powerful tool for representing transforms in a unified form are dual-quaternions. A benefit of this unified form is the interpolation properties, which address a range of limitations (compact form that allows a rotational and translational components to be coupled). In this article, we examine various dual-quaternion interpolation options that achieve different trade-offs between computational cost, aesthetic factors and coupling dependency. Surprisingly, despite dual-quaternions being a common tool in graphics libraries, there are limited details on the interpolation details. Here we attempt to explain interpolation concept, elaborating on underpinning theories, while explaining concepts and bespoke modifications for added control.
Optimization and Optimizers for Adversarial Robustness
Venue: arXiv cs.LG 2023
Year: 2023

Empirical robustness evaluation (RE) of deep learning models against adversarial perturbations entails solving nontrivial constrained optimization problems. Existing numerical algorithms that are commonly used to solve them in practice predominantly rely on projected gradient, and mostly handle perturbations modeled by the $\ell_1$, $\ell_2$ and $\ell_\infty$ distances. In this paper, we introduce a novel algorithmic framework that blends a general-purpose constrained-optimization solver PyGRANSO with Constraint Folding (PWCF), which can add more reliability and generality to the state-of-the-art RE packages, e.g., AutoAttack. Regarding reliability, PWCF provides solutions with stationarity measures and feasibility tests to assess the solution quality. For generality, PWCF can handle perturbation models that are typically inaccessible to the existing projected gradient methods; the main requirement is the distance metric to be almost everywhere differentiable. Taking advantage of PWCF and other existing numerical algorithms, we further explore the distinct patterns in the solutions found for solving these optimization problems using various combinations of losses, perturbation models, and optimization algorithms. We then discuss the implications of these patterns on the current robustness evaluation and adversarial training.
Deep RL with Hierarchical Action Exploration for Dialogue Generation
Venue: arXiv cs.CL 2023
Year: 2023

Traditionally, approximate dynamic programming is employed in dialogue generation with greedy policy improvement through action sampling, as the natural language action space is vast. However, this practice is inefficient for reinforcement learning (RL) due to the sparsity of eligible responses with high action values, which leads to weak improvement sustained by random sampling. This paper presents theoretical analysis and experiments that reveal the performance of the dialogue policy is positively correlated with the sampling size. To overcome this limitation, we introduce a novel dual-granularity Q-function that explores the most promising response category to intervene in the sampling process. Our approach extracts actions based on a grained hierarchy, thereby achieving the optimum with fewer policy iterations. Additionally, we use offline RL and learn from multiple reward functions designed to capture emotional nuances in human interactions. Empirical studies demonstrate that our algorithm outperforms baselines across automatic metrics and human evaluations. Further testing reveals that our algorithm exhibits both explainability and controllability and generates responses with higher expected rewards.
Attention! Dynamic Epistemic Logic Models of (In)attentive Agents
Venue: arXiv cs.AI 2023
Year: 2023

Attention is the crucial cognitive ability that limits and selects what information we observe. Previous work by Bolander et al. (2016) proposes a model of attention based on dynamic epistemic logic (DEL) where agents are either fully attentive or not attentive at all. While introducing the realistic feature that inattentive agents believe nothing happens, the model does not represent the most essential aspect of attention: its selectivity. Here, we propose a generalization that allows for paying attention to subsets of atomic formulas. We introduce the corresponding logic for propositional attention, and show its axiomatization to be sound and complete. We then extend the framework to account for inattentive agents that, instead of assuming nothing happens, may default to a specific truth-value of what they failed to attend to (a sort of prior concerning the unattended atoms). This feature allows for a more cognitively plausible representation of the inattentional blindness phenomenon, where agents end up with false beliefs due to their failure to attend to conspicuous but unexpected events. Both versions of the model define attention-based learning through appropriate DEL event models based on a few and clear edge principles. While the size of such event models grow exponentially both with the number of agents and the number of atoms, we introduce a new logical language for describing event models syntactically and show that using this language our event models can be represented linearly in the number of agents and atoms. Furthermore, representing our event models using this language is achieved by a straightforward formalisation of the aforementioned edge principles.
Forecasting Workload in Cloud Computing: Towards Uncertainty-Aware Predictions and Transfer Learning
Venue: arXiv cs.DC 2023
Year: 2023

Predicting future resource demand in Cloud Computing is essential for optimizing the trade-off between serving customers' requests efficiently and minimizing the provisioning cost. Modelling prediction uncertainty is also desirable to better inform the resource decision-making process, but research in this field is under-investigated. In this paper, we propose univariate and bivariate Bayesian deep learning models that provide predictions of future workload demand and its uncertainty. We run extensive experiments on Google and Alibaba clusters, where we first train our models with datasets from different cloud providers and compare them with LSTM-based baselines. Results show that modelling the uncertainty of predictions has a positive impact on performance, especially on service level metrics, because uncertainty quantification can be tailored to desired target service levels that are critical in cloud applications. Moreover, we investigate whether our models benefit transfer learning capabilities across different domains, i.e. dataset distributions. Experiments on the same workload datasets reveal that acceptable transfer learning performance can be achieved within the same provider (because distributions are more similar). Also, domain knowledge does not transfer when the source and target domains are very different (e.g. from different providers), but this performance degradation can be mitigated by increasing the training set size of the source domain.
Trust Explanations to Do What They Say
Venue: arXiv cs.HC 2023
Year: 2023

How much are we to trust a decision made by an AI algorithm? Trusting an algorithm without cause may lead to abuse, and mistrusting it may similarly lead to disuse. Trust in an AI is only desirable if it is warranted; thus, calibrating trust is critical to ensuring appropriate use. In the name of calibrating trust appropriately, AI developers should provide contracts specifying use cases in which an algorithm can and cannot be trusted. Automated explanation of AI outputs is often touted as a method by which trust can be built in the algorithm. However, automated explanations arise from algorithms themselves, so trust in these explanations is similarly only desirable if it is warranted. Developers of algorithms explaining AI outputs (xAI algorithms) should provide similar contracts, which should specify use cases in which an explanation can and cannot be trusted.
Toxicity and Cultural Entrenchment in Peer-Production Communities: Toward a Handbook on Intelligent System Design
Venue: arXiv cs.HC 2023
Year: 2023

Toxicity and abuse are common in online peer-production communities. The social structure of peer-production communities that aim to produce accurate and trustworthy information require some conflict and gate-keeping to spur content production and curation. However, conflict and gate-keeping often devolve into hierarchical power structures which punish newcomers and lock out marginalized groups through entrenched cultural norms. Community administrators often focus on content quality, rather than consideration for all user safety, to promote community growth and survival. Once toxic cultural norms dominate a peer-production community, it is very difficult for community administrators to stop these behaviors from undermining inclusive peer-production. We propose developing a "handbook of intelligent system design" that attempts to frame design protocols to better read user-community culture and accurately distinguish toxic negative interactions from beneficial conflict.
Artificial Intelligence for Sustainability: Facilitating Sustainable Smart Product-Service Systems with Computer Vision
Venue: arXiv cs.LG 2023
Year: 2023

The usage and impact of deep learning for cleaner production and sustainability purposes remain little explored. This work shows how deep learning can be harnessed to increase sustainability in production and product usage. Specifically, we utilize deep learning-based computer vision to determine the wear states of products. The resulting insights serve as a basis for novel product-service systems with improved integration and result orientation. Moreover, these insights are expected to facilitate product usage improvements and R&D innovations. We demonstrate our approach on two products: machining tools and rotating X-ray anodes. From a technical standpoint, we show that it is possible to recognize the wear state of these products using deep-learning-based computer vision. In particular, we detect wear through microscopic images of the two products. We utilize a U-Net for semantic segmentation to detect wear based on pixel granularity. The resulting mean dice coefficients of 0.631 and 0.603 demonstrate the feasibility of the proposed approach. Consequently, experts can now make better decisions, for example, to improve the machining process parameters. To assess the impact of the proposed approach on environmental sustainability, we perform life cycle assessments that show gains for both products. The results indicate that the emissions of CO2 equivalents are reduced by 12% for machining tools and by 44% for rotating anodes. This work can serve as a guideline and inspire researchers and practitioners to utilize computer vision in similar scenarios to develop sustainable smart product-service systems and enable cleaner production.
Metacognition and Motivation: The Role of Time-Awareness in Preparation for Future Learning
Venue: arXiv cs.HC 2023
Year: 2023

In this work, we investigate how two factors, metacognitive skills and motivation, would impact student learning across domains. More specifically, our primary goal is to identify the critical, yet robust, interaction patterns of these two factors that would contribute to students' performance in learning logic first and then their performance on a subsequent new domain, probability. We are concerned with two types of metacognitive skills: strategy-awareness and time-awareness, that is, which problem-solving strategy to use and when to use it. Our data were collected from 495 participants across three consecutive semesters, and our results show that the only students who consistently outperform their peers across both domains are those who are not only highly motivated but also strategy-aware and time-aware.
Developing IncidentUI -- A Ride Comfort and Disengagement Evaluation Application for Autonomous Vehicles
Venue: arXiv cs.HC 2023
Year: 2023

This report details the design, development, and implementation of IncidentUI, an Android tablet application designed to measure user-experienced ride comfort and record disengagement data for autonomous vehicles (AV) during test drives. The goal of our project was to develop an Android application to run on a peripheral tablet and communicate with the Drive Pegasus AGX, the AI Computing Platform for Nvidia's AV Level 2 Autonomy Solution Architecture [1], to detect AV disengagements and report ride comfort. We designed and developed an Android XML-based intuitive user interface for IncidentUI. The development of IncidentUI required a redesign of the system architecture by redeveloping the system communications protocol in Java and implementing the Protocol Buffers (Protobufs) in Java using the existing system Protobuf definitions. The final iteration of IncidentUI yielded the desired functionality while testing on an AV test drive. We also received positive feedback from Nvidia's AV Platform Team during our final IncidentUI demonstration.
Challenges in Designing Racially Inclusive Language Technologies
Venue: arXiv cs.HC 2023
Year: 2023

We take a critical lens toward the pursuit of racially inclusive language technologies and identify several areas necessitating future work. We discuss the potential harms of conversational technologies, outline three challenges that arise in inclusive design, and lastly, argue that conversational user interface designers and researchers should go beyond racially inclusive design toward anti-racist and race positive designs.
Augmented reality as a Thirdspace: Simultaneous experience of the physical and virtual
Venue: arXiv cs.HC 2023
Year: 2023

With the proliferation of devices that display augmented reality (AR), now is the time for scholars and practitioners to evaluate and engage critically with emerging applications of the medium. AR mediates the way users see their bodies, hear their environment and engage with places. Applied in various forms, including social media, e-commerce, gaming, enterprise and art, the medium facilitates a hybrid experience of physical and digital spaces. This article employs a model of real-and-imagined space from geographer Edward Soja to examine how the user of an AR app navigates the two intertwined spaces of physical and digital, experiencing what Soja calls a 'Third-space'. The article illustrates the potential for headset-based AR to engender such a Thirdspace through the author's practice-led research project, the installation Through the Wardrobe. This installation demonstrates how AR has the potential to shift the way that users view and interact with their world with artistic applications providing an opportunity to question assumptions of social norms, identity and uses of physical space.
Context, Utility and Influence of an Explanation
Venue: arXiv cs.HC 2023
Year: 2023

Contextual utility theory integrates context-sensitive factors into utility-based decision-making models. It stresses the importance of understanding individual decision-makers' preferences, values, and beliefs and the situational factors that affect them. Contextual utility theory benefits explainable AI. First, it can improve transparency and understanding of how AI systems affect decision-making. It can reveal AI model biases and limitations by considering personal preferences and context. Second, contextual utility theory can make AI systems more personalized and adaptable to users and stakeholders. AI systems can better meet user needs and values by incorporating demographic and cultural data. Finally, contextual utility theory promotes ethical AI development and social responsibility. AI developers can create ethical systems that benefit society by considering contextual factors like societal norms and values. This work, demonstrates how contextual utility theory can improve AI system transparency, personalization, and ethics, benefiting both users and developers.
Learning unidirectional coupling using echo-state network
Venue: arXiv cs.LG 2023
Year: 2023

Reservoir Computing has found many potential applications in the field of complex dynamics. In this article, we exploit the exceptional capability of the echo-state network (ESN) model to make it learn a unidirectional coupling scheme from only a few time series data of the system. We show that, once trained with a few example dynamics of a drive-response system, the machine is able to predict the response system's dynamics for any driver signal with the same coupling. Only a few time series data of an $A-B$ type drive-response system in training is sufficient for the ESN to learn the coupling scheme. After training even if we replace drive system $A$ with a different system $C$, the ESN can reproduce the dynamics of response system $B$ using the dynamics of new drive system $C$ only.
Skip Connections in Spiking Neural Networks: An Analysis of Their Effect on Network Training
Venue: arXiv cs.NE 2023
Year: 2023

Spiking neural networks (SNNs) have gained attention as a promising alternative to traditional artificial neural networks (ANNs) due to their potential for energy efficiency and their ability to model spiking behavior in biological systems. However, the training of SNNs is still a challenging problem, and new techniques are needed to improve their performance. In this paper, we study the impact of skip connections on SNNs and propose a hyperparameter optimization technique that adapts models from ANN to SNN. We demonstrate that optimizing the position, type, and number of skip connections can significantly improve the accuracy and efficiency of SNNs by enabling faster convergence and increasing information flow through the network. Our results show an average +8% accuracy increase on CIFAR-10-DVS and DVS128 Gesture datasets adaptation of multiple state-of-the-art models.
TinyML: Tools, Applications, Challenges, and Future Research Directions
Venue: arXiv cs.LG 2023
Year: 2023

In recent years, Artificial Intelligence (AI) and Machine learning (ML) have gained significant interest from both, industry and academia. Notably, conventional ML techniques require enormous amounts of power to meet the desired accuracy, which has limited their use mainly to high-capability devices such as network nodes. However, with many advancements in technologies such as the Internet of Things (IoT) and edge computing, it is desirable to incorporate ML techniques into resource-constrained embedded devices for distributed and ubiquitous intelligence. This has motivated the emergence of the TinyML paradigm which is an embedded ML technique that enables ML applications on multiple cheap, resource- and power-constrained devices. However, during this transition towards appropriate implementation of the TinyML technology, multiple challenges such as processing capacity optimization, improved reliability, and maintenance of learning models' accuracy require timely solutions. In this article, various avenues available for TinyML implementation are reviewed. Firstly, a background of TinyML is provided, followed by detailed discussions on various tools supporting TinyML. Then, state-of-art applications of TinyML using advanced technologies are detailed. Lastly, various research challenges and future directions are identified.
Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languages
Venue: arXiv cs.CL 2023
Year: 2023

While code-mixing is a common linguistic practice in many parts of the world, collecting high-quality and low-cost code-mixed data remains a challenge for natural language processing (NLP) research. The recent proliferation of Large Language Models (LLMs) compels one to ask: how capable are these systems in generating code-mixed data? In this paper, we explore prompting multilingual LLMs in a zero-shot manner to generate code-mixed data for seven languages in South East Asia (SEA), namely Indonesian, Malay, Chinese, Tagalog, Vietnamese, Tamil, and Singlish. We find that publicly available multilingual instruction-tuned models such as BLOOMZ and Flan-T5-XXL are incapable of producing texts with phrases or clauses from different languages. ChatGPT exhibits inconsistent capabilities in generating code-mixed texts, wherein its performance varies depending on the prompt template and language pairing. For instance, ChatGPT generates fluent and natural Singlish texts (an English-based creole spoken in Singapore), but for English-Tamil language pair, the system mostly produces grammatically incorrect or semantically meaningless utterances. Furthermore, it may erroneously introduce languages not specified in the prompt. Based on our investigation, existing multilingual LLMs exhibit a wide range of proficiency in code-mixed data generation for SEA languages. As such, we advise against using LLMs in this context without extensive human checks.