OpenRaiser/Intern-Atlas huggingface dataset:
FlatENN: Train Flat for Enhanced Fault Tolerance of Quantized Deep Neural Networks
Venue: arXiv cs.LG 2022
Year: 2022

Model compression via quantization and sparsity enhancement has gained an immense interest to enable the deployment of deep neural networks (DNNs) in resource-constrained edge environments. Although these techniques have shown promising results in reducing the energy, latency and memory requirements of the DNNs, their performance in non-ideal real-world settings (such as in the presence of hardware faults) is yet to be completely understood. In this paper, we investigate the impact of bit-flip and stuck-at faults on activation-sparse quantized DNNs (QDNNs). We show that a high level of activation sparsity comes at the cost of larger vulnerability to faults. For instance, activation-sparse QDNNs exhibit up to 17.32% lower accuracy than the standard QDNNs. We also establish that one of the major cause of the degraded accuracy is sharper minima in the loss landscape for activation-sparse QDNNs, which makes them more sensitive to perturbations in the weight values due to faults. Based on this observation, we propose the mitigation of the impact of faults by employing a sharpness-aware quantization (SAQ) training scheme. The activation-sparse and standard QDNNs trained with SAQ have up to 36.71% and 24.76% higher inference accuracy, respectively compared to their conventionally trained equivalents. Moreover, we show that SAQ-trained activation-sparse QDNNs show better accuracy in faulty settings than standard QDNNs trained conventionally. Thus the proposed technique can be instrumental in achieving sparsity-related energy/latency benefits without compromising on fault tolerance.
Reinforcement Learning with Success Induced Task Prioritization
Venue: arXiv cs.LG 2022
Year: 2022

Many challenging reinforcement learning (RL) problems require designing a distribution of tasks that can be applied to train effective policies. This distribution of tasks can be specified by the curriculum. A curriculum is meant to improve the results of learning and accelerate it. We introduce Success Induced Task Prioritization (SITP), a framework for automatic curriculum learning, where a task sequence is created based on the success rate of each task. In this setting, each task is an algorithmically created environment instance with a unique configuration. The algorithm selects the order of tasks that provide the fastest learning for agents. The probability of selecting any of the tasks for the next stage of learning is determined by evaluating its performance score in previous stages. Experiments were carried out in the Partially Observable Grid Environment for Multiple Agents (POGEMA) and Procgen benchmark. We demonstrate that SITP matches or surpasses the results of other curriculum design methods. Our method can be implemented with handful of minor modifications to any standard RL framework and provides useful prioritization with minimal computational overhead.
Kannudi -- A Reference Editor for Kannada
Venue: arXiv cs.HC 2022
Year: 2022

Kannudi is a reference editor for Kannada based on OPOK! and OHOK! principles, and domain knowledge. It introduces a method of input for Kannada, called OHOK!, that is, Ottu Haku Ottu Kodu! (apply pressure and give ottu). This is especially suited for pressure sensitive input devices, though the current online implementation uses the regular mechanical keyboard. OHOK! has three possible modes, namely, sva-ottu (self-conjunct), kandante (as you see), and andante (as you say). It may be noted that kandante mode does not follow the phonetic order. However, this mode may work well for those who are inclined to visualize as they type rather than vocalizing the sounds. Kannudi also demonstrates how domain knowledge can be effectively used to potentially increase speed, accuracy, and user friendliness. For example, selection of a default vowel, automatic shunyification, and arkification. Also implemented are four types Deletes that are necessary for phono-syllabic languages like Kannada.
Categorisation of future applications for Augmented Reality in human lunar exploration
Venue: arXiv cs.HC 2022
Year: 2022

The European Space Agency (ESA) has a clear mission to go forward to the Moon in preparation of human presence on Mars. One of the technologies looked at to increase safety and efficiency of astronauts in this context is Augmented Reality (AR). This technology allows digital visual information to be overlaid onto the user's environment through some type of display or projector. In recent years separate studies have been conducted to test the potential value of AR for astronauts by implementing a few functionalities on an AR display followed by testing in terrestrial analogue environments. One of the groups contributing to these investigations is Spaceship EAC (SSEAC). SSEAC is a group of interns and trainees at the European Astronaut Centre (EAC) focusing on emerging technologies for human space exploration. This paper presents an outcome of SSEAC's activities related to AR for lunar extravehicular activities (EVAs), in which an approach similar to design thinking was used to explore, identify, and structure the opportunities offered by this technology. The resulting categorization of AR use cases can be used to identify new functionalities to test through prototyping and usability tests and can also be used to relate individual studies to each other to gain insight into the overall potential value AR has to offer to human lunar exploration. The approach adopted in this paper is based on the Fuzzy Front End (FFE) model from the innovation management domain. Utilising a user-driven instead of technology-driven method resulted in findings that are relevant irrespective of the hardware and software implementation. Instead, the outcome is an overview of use cases in which some type of AR system could provide value by contributing to increased astronaut safety, efficiency and/or efficacy.
Ring That Bell: A Corpus and Method for Multimodal Metaphor Detection in Videos
Venue: arXiv cs.MM 2022
Year: 2022

We present the first openly available multimodal metaphor annotated corpus. The corpus consists of videos including audio and subtitles that have been annotated by experts. Furthermore, we present a method for detecting metaphors in the new dataset based on the textual content of the videos. The method achieves a high F1-score (62\%) for metaphorical labels. We also experiment with other modalities and multimodal methods; however, these methods did not out-perform the text-based model. In our error analysis, we do identify that there are cases where video could help in disambiguating metaphors, however, the visual cues are too subtle for our model to capture. The data is available on Zenodo.
Common Practices and Taxonomy in Deep Multi-view Fusion for Remote Sensing Applications
Venue: arXiv cs.CV 2022
Year: 2022

The advances in remote sensing technologies have boosted applications for Earth observation. These technologies provide multiple observations or views with different levels of information. They might contain static or temporary views with different levels of resolution, in addition to having different types and amounts of noise due to sensor calibration or deterioration. A great variety of deep learning models have been applied to fuse the information from these multiple views, known as deep multi-view or multi-modal fusion learning. However, the approaches in the literature vary greatly since different terminology is used to refer to similar concepts or different illustrations are given to similar techniques. This article gathers works on multi-view fusion for Earth observation by focusing on the common practices and approaches used in the literature. We summarize and structure insights from several different publications concentrating on unifying points and ideas. In this manuscript, we provide a harmonized terminology while at the same time mentioning the various alternative terms that are used in literature. The topics covered by the works reviewed focus on supervised learning with the use of neural network models. We hope this review, with a long list of recent references, can support future research and lead to a unified advance in the area.
Tracing the Origin of Adversarial Attack for Forensic Investigation and Deterrence
Venue: arXiv cs.CR 2022
Year: 2022

Deep neural networks are vulnerable to adversarial attacks. In this paper, we take the role of investigators who want to trace the attack and identify the source, that is, the particular model which the adversarial examples are generated from. Techniques derived would aid forensic investigation of attack incidents and serve as deterrence to potential attacks. We consider the buyers-seller setting where a machine learning model is to be distributed to various buyers and each buyer receives a slightly different copy with same functionality. A malicious buyer generates adversarial examples from a particular copy $\mathcal{M}_i$ and uses them to attack other copies. From these adversarial examples, the investigator wants to identify the source $\mathcal{M}_i$. To address this problem, we propose a two-stage separate-and-trace framework. The model separation stage generates multiple copies of a model for a same classification task. This process injects unique characteristics into each copy so that adversarial examples generated have distinct and traceable features. We give a parallel structure which embeds a ``tracer'' in each copy, and a noise-sensitive training loss to achieve this goal. The tracing stage takes in adversarial examples and a few candidate models, and identifies the likely source. Based on the unique features induced by the noise-sensitive loss function, we could effectively trace the potential adversarial copy by considering the output logits from each tracer. Empirical results show that it is possible to trace the origin of the adversarial example and the mechanism can be applied to a wide range of architectures and datasets.
CarFi: Rider Localization Using Wi-Fi CSI
Venue: arXiv cs.NI 2022
Year: 2022

With the rise of hailing services, people are increasingly relying on shared mobility (e.g., Uber, Lyft) drivers to pick up for transportation. However, such drivers and riders have difficulties finding each other in urban areas as GPS signals get blocked by skyscrapers, in crowded environments (e.g., in stadiums, airports, and bars), at night, and in bad weather. It wastes their time, creates a bad user experience, and causes more CO2 emissions due to idle driving. In this work, we explore the potential of Wi-Fi to help drivers to determine the street side of the riders. Our proposed system is called CarFi that uses Wi-Fi CSI from two antennas placed inside a moving vehicle and a data-driven technique to determine the street side of the rider. By collecting real-world data in realistic and challenging settings by blocking the signal with other people and other parked cars, we see that CarFi is 95.44% accurate in rider-side determination in both line of sight (LoS) and non-line of sight (nLoS) conditions, and can be run on an embedded GPU in real-time.
Information Flow Tracking Methods for Protecting Cyber-Physical Systems against Hardware Trojans -- a Survey
Venue: arXiv cs.CR 2022
Year: 2022

Cyber-physical systems (CPS) provide profitable surfaces for hardware attacks such as hardware Trojans. Hardware Trojans can implement stealthy attacks such as leaking critical information, taking control of devices or harm humans. In this article we review information flow tracking (IFT) methods for protecting CPS against hardware Trojans, and discuss their current limitations. IFT methods are a promising approach for the detection of hardware Trojans in complex systems because the detection mechanism does not necessarily rely on potential Trojan behavior. However, in order to maximize the benefits research should focus more on black-box design models and consider real-world attack scenarios.
mRpostman: An IMAP Client for R
Venue: arXiv cs.NI 2022
Year: 2022

Internet Message Access Protocol (IMAP) clients are a common feature in several programming languages. Despite having some packages for electronic messages retrieval, the R language, until recently, lacked a broader solution, capable of coping with different IMAP servers and providing a wide spectrum of features. mRpostman covers most of the IMAP 4rev1 functionalities by implementing tools for message searching, selective fetching of message attributes, mailbox management, attachment extraction, and several other IMAP features that can be executed in virtually any mail provider. By doing so, it enables users to perform data analysis based on e-mail content. The goal of this article is to showcase the toolkit provided with the mRpostman package, to describe its key features and provide some application examples.
Exceeding Computational Complexity Trial-and-Error Dynamic Action and Intelligence
Venue: arXiv cs.CC 2022
Year: 2022

Computational complexity is a core theory of computer science, which dictates the degree of difficulty of computation. There are many problems with high complexity that we have to deal, which is especially true for AI. This raises a big question: Is there a better way to deal with these highly complex problems other than bounded by computational complexity? We believe that ideas and methods from intelligence science can be applied to these problems and help us to exceed computational complexity. In this paper, we try to clarify concepts, and we propose definitions such as unparticularized computing, particularized computing, computing agents, and dynamic search. We also propose and discuss a framework, i.e., trial-and-error + dynamic search. Number Partition Problem is a well-known NP-complete problem, and we use this problem as an example to illustrate the ideas discussed.
Need of 6G for the Metaverse Realization
Venue: arXiv cs.NI 2022
Year: 2022

The concept of the Metaverse aims to bring a fully-fledged extended reality environment to provide next generation applications and services. Development of the Metaverse is backed by many technologies, including, 5G, artificial intelligence, edge computing and extended reality. The advent of 6G is envisaged to mark a significant milestone in the development of the Metaverse, facilitating near-zero-latency, a plethora of new services and upgraded real-world infrastructure. This paper establishes the advantages of providing the Metaverse services over 6G along with an overview of the demanded technical requirements. The paper provides an insight to the concepts of the Metaverse and the envisaged technical capabilities of 6G mobile networks. Then, the technical aspects covering 6G for the development of the Metaverse, ranging from validating digital assets, interoperability, and efficient user interaction in the Metaverse to related security and privacy aspects are elaborated. Subsequently, the role of 6G technologies towards enabling the Metaverse, including artificial intelligence, blockchain, open radio access networks, edge computing, cloudification and internet of everything. The paper also presents 6G integration challenges and outlines ongoing projects towards developing the Metaverse technologies to facilitate the Metaverse applications and services.
White-box Inference Attacks against Centralized Machine Learning and Federated Learning
Venue: arXiv cs.CR 2022
Year: 2022

With the development of information science and technology, various industries have generated massive amounts of data, and machine learning is widely used in the analysis of big data. However, if the privacy of machine learning applications' customers cannot be guaranteed, it will cause security threats and losses to users' personal privacy information and service providers. Therefore, the issue of privacy protection of machine learning has received wide attention. For centralized machine learning models, we evaluate the impact of different neural network layers, gradient, gradient norm, and fine-tuned models on member inference attack performance with prior knowledge; For the federated learning model, we discuss the location of the attacker in the target model and its attack mode. The results show that the centralized machine learning model shows more serious member information leakage in all aspects, and the accuracy of the attacker in the central parameter server is significantly higher than the local Inference attacks as participants.
BASPRO: a balanced script producer for speech corpus collection based on the genetic algorithm
Venue: arXiv cs.NE 2022
Year: 2022

The performance of speech-processing models is heavily influenced by the speech corpus that is used for training and evaluation. In this study, we propose BAlanced Script PROducer (BASPRO) system, which can automatically construct a phonetically balanced and rich set of Chinese sentences for collecting Mandarin Chinese speech data. First, we used pretrained natural language processing systems to extract ten-character candidate sentences from a large corpus of Chinese news texts. Then, we applied a genetic algorithm-based method to select 20 phonetically balanced sentence sets, each containing 20 sentences, from the candidate sentences. Using BASPRO, we obtained a recording script called TMNews, which contains 400 ten-character sentences. TMNews covers 84% of the syllables used in the real world. Moreover, the syllable distribution has 0.96 cosine similarity to the real-world syllable distribution. We converted the script into a speech corpus using two text-to-speech systems. Using the designed speech corpus, we tested the performances of speech enhancement (SE) and automatic speech recognition (ASR), which are one of the most important regression- and classification-based speech processing tasks, respectively. The experimental results show that the SE and ASR models trained on the designed speech corpus outperform their counterparts trained on a randomly composed speech corpus.
Methodological reflections for AI alignment research using human feedback
Venue: arXiv cs.HC 2022
Year: 2022

The field of artificial intelligence (AI) alignment aims to investigate whether AI technologies align with human interests and values and function in a safe and ethical manner. AI alignment is particularly relevant for large language models (LLMs), which have the potential to exhibit unintended behavior due to their ability to learn and adapt in ways that are difficult to predict. In this paper, we discuss methodological challenges for the alignment problem specifically in the context of LLMs trained to summarize texts. In particular, we focus on methods for collecting reliable human feedback on summaries to train a reward model which in turn improves the summarization model. We conclude by suggesting specific improvements in the experimental design of alignment studies for LLMs' summarization capabilities.
Creating awareness about security and safety on highways to mitigate wildlife-vehicle collisions by detecting and recognizing wildlife fences using deep learning and drone technology
Venue: arXiv cs.CV 2022
Year: 2022

In South Africa, it is a common practice for people to leave their vehicles beside the road when traveling long distances for a short comfort break. This practice might increase human encounters with wildlife, threatening their security and safety. Here we intend to create awareness about wildlife fencing, using drone technology and computer vision algorithms to recognize and detect wildlife fences and associated features. We collected data at Amakhala and Lalibela private game reserves in the Eastern Cape, South Africa. We used wildlife electric fence data containing single and double fences for the classification task. Additionally, we used aerial and still annotated images extracted from the drone and still cameras for the segmentation and detection tasks. The model training results from the drone camera outperformed those from the still camera. Generally, poor model performance is attributed to (1) over-decompression of images and (2) the ability of drone cameras to capture more details on images for the machine learning model to learn as compared to still cameras that capture only the front view of the wildlife fence. We argue that our model can be deployed on client-edge devices to inform people about the presence and significance of wildlife fencing, which minimizes human encounters with wildlife, thereby mitigating wildlife-vehicle collisions.
A Review of Scene Representations for Robot Manipulators
Venue: arXiv cs.RO 2022
Year: 2022

For a robot to act intelligently, it needs to sense the world around it. Increasingly, robots build an internal representation of the world from sensor readings. This representation can then be used to inform downstream tasks, such as manipulation, collision avoidance, or human interaction. In practice, scene representations vary widely depending on the type of robot, the sensing modality, and the task that the robot is designed to do. This review provides an overview of the scene representations used for robot manipulators (robot arms). We focus primarily on representations which are built from real world sensing and are used to inform some downstream robotics task.
Inching Towards Automated Understanding of the Meaning of Art: An Application to Computational Analysis of Mondrian's Artwork
Venue: arXiv cs.CV 2022
Year: 2022

Deep Neural Networks (DNNs) have been successfully used in classifying digital images but have been less successful in classifying images with meanings that are not linear combinations of their visualized features, like images of artwork. Moreover, it is unknown what additional features must be included into DNNs, so that they can possibly classify using features beyond visually displayed features, like color, size, and form. Non-displayed features are important in abstract representations, reasoning, and understanding ambiguous expressions, which are arguably topics less studied by current AI methods. This paper attempts to identify capabilities that are related to semantic processing, a current limitation of DNNs. The proposed methodology identifies the missing capabilities by comparing the process of understanding Mondrian's paintings with the process of understanding electronic circuit designs, another creative problem solving instance. The compared entities are cognitive architectures that attempt to loosely mimic cognitive activities. The paper offers a detailed presentation of the characteristics of the architectural components, like goals, concepts, ideas, rules, procedures, beliefs, expectations, and outcomes. To explain the usefulness of the methodology, the paper discusses a new, three-step computational method to distinguish Mondrian's paintings from other artwork. The method includes in a backward order the cognitive architecture's components that operate only with the characteristics of the available data.
Subspace Coding for Error Control and Interference Mitigation in Routing Solutions with Cooperative Destination Nodes
Venue: arXiv cs.IT 2022
Year: 2022

Random Linear Network Coding (RLNC) is a transmission scheme that opts for linear combinations of the transmitted packets at a subset of the intermediate nodes. This scheme is usually considered when Network Coding (NC) is desired over non-coherent networks. In order to integrate error correction in RLNC, subspace codes have been proposed. The codewords in those codes are unchanged under rank-preserving row operations, making them convenient for RLNC. In this paper, we investigate the use of those codes for the interference channel in a system model consisting of an array of SISO communication systems with cooperative destination nodes. This system model is deemed as a simplified model of a network with a routing based transmission scheme. Results have indicated that the use of subspace codes have allowed for better performance in terms of the decoding failure probability.