OpenRaiser/Intern-Atlas huggingface dataset:
Self-Supervised Implicit Attention: Guided Attention by The Model Itself
Venue: arXiv cs.CV 2022
Year: 2022

We propose Self-Supervised Implicit Attention (SSIA), a new approach that adaptively guides deep neural network models to gain attention by exploiting the properties of the models themselves. SSIA is a novel attention mechanism that does not require any extra parameters, computation, or memory access costs during inference, which is in contrast to existing attention mechanism. In short, by considering attention weights as higher-level semantic information, we reconsidered the implementation of existing attention mechanisms and further propose generating supervisory signals from higher network layers to guide lower network layers for parameter updates. We achieved this by building a self-supervised learning task using the hierarchical features of the network itself, which only works at the training stage. To verify the effectiveness of SSIA, we performed a particular implementation (called an SSIA block) in convolutional neural network models and validated it on several image classification datasets. The experimental results show that an SSIA block can significantly improve the model performance, even outperforms many popular attention methods that require additional parameters and computation costs, such as Squeeze-and-Excitation and Convolutional Block Attention Module. Our implementation will be available on GitHub.
Towards ML Methods for Biodiversity: A Novel Wild Bee Dataset and Evaluations of XAI Methods for ML-Assisted Rare Species Annotations
Venue: arXiv cs.AI 2022
Year: 2022

Insects are a crucial part of our ecosystem. Sadly, in the past few decades, their numbers have worryingly decreased. In an attempt to gain a better understanding of this process and monitor the insects populations, Deep Learning may offer viable solutions. However, given the breadth of their taxonomy and the typical hurdles of fine grained analysis, such as high intraclass variability compared to low interclass variability, insect classification remains a challenging task. There are few benchmark datasets, which impedes rapid development of better AI models. The annotation of rare species training data, however, requires expert knowledge. Explainable Artificial Intelligence (XAI) could assist biologists in these annotation tasks, but choosing the optimal XAI method is difficult. Our contribution to these research challenges is threefold: 1) a dataset of thoroughly annotated images of wild bees sampled from the iNaturalist database, 2) a ResNet model trained on the wild bee dataset achieving classification scores comparable to similar state-of-the-art models trained on other fine-grained datasets and 3) an investigation of XAI methods to support biologists in annotation tasks.
Legal Provocations for HCI in the Design and Development of Trustworthy Autonomous Systems
Venue: arXiv cs.HC 2022
Year: 2022

We consider a series of legal provocations emerging from the proposed European Union AI Act 2021 (AIA) and how they open up new possibilities for HCI in the design and development of trustworthy autonomous systems. The AIA continues the by design trend seen in recent EU regulation of emerging technologies. The AIA targets AI developments that pose risks to society and citizens fundamental rights, introducing mandatory design and development requirements for high-risk AI systems (HRAIS). These requirements regulate different stages of the AI development cycle including ensuring data quality and governance strategies, mandating testing of systems, ensuring appropriate risk management, designing for human oversight, and creating technical documentation. These requirements open up new opportunities for HCI that reach beyond established concerns with the ethics and explainability of AI and situate AI development in human-centered processes and methods of design to enable compliance with regulation and foster societal trust in AI.
Evaluating object detector ensembles for improving the robustness of artifact detection in endoscopic video streams
Venue: arXiv cs.CV 2022
Year: 2022

In this contribution we use an ensemble deep-learning method for combining the prediction of two individual one-stage detectors (i.e., YOLOv4 and Yolact) with the aim to detect artefacts in endoscopic images. This ensemble strategy enabled us to improve the robustness of the individual models without harming their real-time computation capabilities. We demonstrated the effectiveness of our approach by training and testing the two individual models and various ensemble configurations on the "Endoscopic Artifact Detection Challenge" dataset. Extensive experiments show the superiority, in terms of mean average precision, of the ensemble approach over the individual models and previous works in the state of the art.
In-Range Farthest Point Queries and Related Problem in High Dimensions
Venue: arXiv cs.CG 2022
Year: 2022

Range-aggregate query is an important type of queries with numerous applications. It aims to obtain some structural information (defined by an aggregate function $F(\cdot)$) of the points (from a point set $P$) inside a given query range $B$. In this paper, we study the range-aggregate query problem in high dimensional space for two aggregate functions: (1) $F(P \cap B)$ is the farthest point in $P \cap B$ to a query point $q$ in $\mathbb{R}^d$ and (2) $F(P \cap B)$ is the minimum enclosing ball (MEB) of $P \cap B$. For problem (1), called In-Range Farthest Point (IFP) Query, we develop a bi-criteria approximation scheme: For any $ε>0$ that specifies the approximation ratio of the farthest distance and any $γ>0$ that measures the "fuzziness" of the query range, we show that it is possible to pre-process $P$ into a data structure of size $\tilde{O}_{ε,γ}(dn^{1+ρ})$ in $\tilde{O}_{ε,γ}(dn^{1+ρ})$ time such that given any $\mathbb{R}^d$ query ball $B$ and query point $q$, it outputs in $\tilde{O}_{ε,γ}(dn^ρ)$ time a point $p$ that is a $(1-ε)$-approximation of the farthest point to $q$ among all points lying in a $(1+γ)$-expansion $B(1+γ)$ of $B$, where $0<ρ<1$ is a constant depending on $ε$ and $γ$ and the hidden constants in big-O notations depend only on $ε$, $γ$ and $\text{Polylog}(nd)$. For problem (2), we show that the IFP result can be applied to develop query scheme with similar time and space complexities to achieve a $(1+ε)$-approximation for MEB.
Epistemic Deep Learning
Venue: arXiv cs.LG 2022
Year: 2022

The belief function approach to uncertainty quantification as proposed in the Demspter-Shafer theory of evidence is established upon the general mathematical models for set-valued observations, called random sets. Set-valued predictions are the most natural representations of uncertainty in machine learning. In this paper, we introduce a concept called epistemic deep learning based on the random-set interpretation of belief functions to model epistemic learning in deep neural networks. We propose a novel random-set convolutional neural network for classification that produces scores for sets of classes by learning set-valued ground truth representations. We evaluate different formulations of entropy and distance measures for belief functions as viable loss functions for these random-set networks. We also discuss methods for evaluating the quality of epistemic predictions and the performance of epistemic random-set neural networks. We demonstrate through experiments that the epistemic approach produces better performance results when compared to traditional approaches of estimating uncertainty.
The SIGMORPHON 2022 Shared Task on Morpheme Segmentation
Venue: arXiv cs.CL 2022
Year: 2022

The SIGMORPHON 2022 shared task on morpheme segmentation challenged systems to decompose a word into a sequence of morphemes and covered most types of morphology: compounds, derivations, and inflections. Subtask 1, word-level morpheme segmentation, covered 5 million words in 9 languages (Czech, English, Spanish, Hungarian, French, Italian, Russian, Latin, Mongolian) and received 13 system submissions from 7 teams and the best system averaged 97.29% F1 score across all languages, ranging English (93.84%) to Latin (99.38%). Subtask 2, sentence-level morpheme segmentation, covered 18,735 sentences in 3 languages (Czech, English, Mongolian), received 10 system submissions from 3 teams, and the best systems outperformed all three state-of-the-art subword tokenization methods (BPE, ULM, Morfessor2) by 30.71% absolute. To facilitate error analysis and support any type of future studies, we released all system predictions, the evaluation script, and all gold standard datasets.
Energy Saving Techniques for Energy Constrained CMOS Circuits and Systems
Venue: arXiv cs.AR 2022
Year: 2022

Portable devices like smartphones, tablets, wearable electronic devices, medical implants, wireless sensor nodes, and Internet-of-Things (IoT) devices have tremendous constraints on their energy consumption. Adding more functionalities onto the portable devices increases its energy consumption significantly. However, the energy capacity of the battery does not increase proportionally. Hence, to overcome the constraints on energy consumption, two main approaches are being undertaken by the designers to integrate more functionalities onto the energy-constrained systems. One approach involves reducing the inherent energy consumption of circuits, and the other involves harvesting energy from the ambient sources and utilizing it to power the circuits. In this thesis, both the approaches mentioned above are followed in developing energy-saving techniques for energy-constrained CMOS circuits and systems. Switched-capacitor-assisted power gating technique is proposed to reduce the leakage current of large digital circuits. Benefits in overall energy saving using nano-electromechanical switches (NEMS) for power gating is quantified on a system-on chip for a mobile platform made using a 14 nm gate length FinFET. For reducing the quiescent current in the analog front-end circuits, this thesis proposes a novel technique of realizing discrete-time (D-T) signal amplification using nano-electromechanical switches (NEMS). Apart from the three energy reduction techniques mentioned above, harvesting energy from mechanical vibrations using an efficient technique of power extraction from the piezoelectric transducer is proposed.
Experimental Validation of Spectral-Spatial Power Evolution Design Using Raman Amplifiers
Venue: arXiv cs.LG 2022
Year: 2022

We experimentally validate a machine learning-enabled Raman amplification framework, capable of jointly shaping the signal power evolution in two domains: frequency and fiber distance. The proposed experiment addresses the amplification in the whole C-band, by optimizing four first-order counter-propagating Raman pumps.
Structured Video Tokens @ Ego4D PNR Temporal Localization Challenge 2022
Venue: arXiv cs.CV 2022
Year: 2022

This technical report describes the SViT approach for the Ego4D Point of No Return (PNR) Temporal Localization Challenge. We propose a learning framework StructureViT (SViT for short), which demonstrates how utilizing the structure of a small number of images only available during training can improve a video model. SViT relies on two key insights. First, as both images and videos contain structured information, we enrich a transformer model with a set of \emph{object tokens} that can be used across images and videos. Second, the scene representations of individual frames in video should "align" with those of still images. This is achieved via a "Frame-Clip Consistency" loss, which ensures the flow of structured information between images and videos. SViT obtains strong performance on the challenge test set with 0.656 absolute temporal localization error.
Improving Diversity with Adversarially Learned Transformations for Domain Generalization
Venue: arXiv cs.LG 2022
Year: 2022

To be successful in single source domain generalization, maximizing diversity of synthesized domains has emerged as one of the most effective strategies. Many of the recent successes have come from methods that pre-specify the types of diversity that a model is exposed to during training, so that it can ultimately generalize well to new domains. However, naïve diversity based augmentations do not work effectively for domain generalization either because they cannot model large domain shift, or because the span of transforms that are pre-specified do not cover the types of shift commonly occurring in domain generalization. To address this issue, we present a novel framework that uses adversarially learned transformations (ALT) using a neural network to model plausible, yet hard image transformations that fool the classifier. This network is randomly initialized for each batch and trained for a fixed number of steps to maximize classification error. Further, we enforce consistency between the classifier's predictions on the clean and transformed images. With extensive empirical analysis, we find that this new form of adversarial transformations achieve both objectives of diversity and hardness simultaneously, outperforming all existing techniques on competitive benchmarks for single source domain generalization. We also show that ALT can naturally work with existing diversity modules to produce highly distinct, and large transformations of the source domain leading to state-of-the-art performance.
Deep Learning and Handheld Augmented Reality Based System for Optimal Data Collection in Fault Diagnostics Domain
Venue: arXiv cs.AI 2022
Year: 2022

Compared to current AI or robotic systems, humans navigate their environment with ease, making tasks such as data collection trivial. However, humans find it harder to model complex relationships hidden in the data. AI systems, especially deep learning (DL) algorithms, impressively capture those complex relationships. Symbiotically coupling humans and computational machines' strengths can simultaneously minimize the collected data required and build complex input-to-output mapping models. This paper enables this coupling by presenting a novel human-machine interaction framework to perform fault diagnostics with minimal data. Collecting data for diagnosing faults for complex systems is difficult and time-consuming. Minimizing the required data will increase the practicability of data-driven models in diagnosing faults. The framework provides instructions to a human user to collect data that mitigates the difference between the data used to train and test the fault diagnostics model. The framework is composed of three components: (1) a reinforcement learning algorithm for data collection to develop a training dataset, (2) a deep learning algorithm for diagnosing faults, and (3) a handheld augmented reality application for data collection for testing data. The proposed framework has provided above 100\% precision and recall on a novel dataset with only one instance of each fault condition. Additionally, a usability study was conducted to gauge the user experience of the handheld augmented reality application, and all users were able to follow the provided steps.
Robust Attack Graph Generation
Venue: arXiv cs.LG 2022
Year: 2022

We present a method to learn automaton models that are more robust to input modifications. It iteratively aligns sequences to a learned model, modifies the sequences to their aligned versions, and re-learns the model. Automaton learning algorithms are typically very good at modeling the frequent behavior of a software system. Our solution can be used to also learn the behavior present in infrequent sequences, as these will be aligned to the frequent ones represented by the model. We apply our method to the SAGE tool for modeling attacker behavior from intrusion alerts. In experiments, we demonstrate that our algorithm learns models that can handle noise such as added and removed symbols from sequences. Furthermore, it learns more concise models that fit better to the training data.
Beyond Adult and COMPAS: Fairness in Multi-Class Prediction
Venue: arXiv cs.LG 2022
Year: 2022

We consider the problem of producing fair probabilistic classifiers for multi-class classification tasks. We formulate this problem in terms of "projecting" a pre-trained (and potentially unfair) classifier onto the set of models that satisfy target group-fairness requirements. The new, projected model is given by post-processing the outputs of the pre-trained classifier by a multiplicative factor. We provide a parallelizable iterative algorithm for computing the projected classifier and derive both sample complexity and convergence guarantees. Comprehensive numerical comparisons with state-of-the-art benchmarks demonstrate that our approach maintains competitive performance in terms of accuracy-fairness trade-off curves, while achieving favorable runtime on large datasets. We also evaluate our method at scale on an open dataset with multiple classes, multiple intersectional protected groups, and over 1M samples.
Neural Network Normal Estimation and Bathymetry Reconstruction from Sidescan Sonar
Venue: arXiv cs.RO 2022
Year: 2022

Sidescan sonar intensity encodes information about the changes of surface normal of the seabed. However, other factors such as seabed geometry as well as its material composition also affect the return intensity. One can model these intensity changes in a forward direction from the surface normals from bathymetric map and physical properties to the measured intensity or alternatively one can use an inverse model which starts from the intensities and models the surface normals. Here we use an inverse model which leverages deep learning's ability to learn from data; a convolutional neural network is used to estimate the surface normal from the sidescan. Thus the internal properties of the seabed are only implicitly learned. Once this information is estimated, a bathymetric map can be reconstructed through an optimization framework that also includes altimeter readings to provide a sparse depth profile as a constraint. Implicit neural representation learning was recently proposed to represent the bathymetric map in such an optimization framework. In this article, we use a neural network to represent the map and optimize it under constraints of altimeter points and estimated surface normal from sidescan. By fusing multiple observations from different angles from several sidescan lines, the estimated results are improved through optimization. We demonstrate the efficiency and scalability of the approach by reconstructing a high-quality bathymetry using sidescan data from a large sidescan survey. We compare the proposed data-driven inverse model approach of modeling a sidescan with a forward Lambertian model. We assess the quality of each reconstruction by comparing it with data constructed from a multibeam sensor.
Efficient Approximation of Expected Hypervolume Improvement using Gauss-Hermite Quadrature
Venue: arXiv cs.LG 2022
Year: 2022

Many methods for performing multi-objective optimisation of computationally expensive problems have been proposed recently. Typically, a probabilistic surrogate for each objective is constructed from an initial dataset. The surrogates can then be used to produce predictive densities in the objective space for any solution. Using the predictive densities, we can compute the expected hypervolume improvement (EHVI) due to a solution. Maximising the EHVI, we can locate the most promising solution that may be expensively evaluated next. There are closed-form expressions for computing the EHVI, integrating over the multivariate predictive densities. However, they require partitioning the objective space, which can be prohibitively expensive for more than three objectives. Furthermore, there are no closed-form expressions for a problem where the predictive densities are dependent, capturing the correlations between objectives. Monte Carlo approximation is used instead in such cases, which is not cheap. Hence, the need to develop new accurate but cheaper approximation methods remains. Here we investigate an alternative approach toward approximating the EHVI using Gauss-Hermite quadrature. We show that it can be an accurate alternative to Monte Carlo for both independent and correlated predictive densities with statistically significant rank correlations for a range of popular test problems.
Roadblocks to Attracting Students to Software Testing Careers: Comparisons of Replicated Studies
Venue: arXiv cs.SE 2022
Year: 2022

Context. Recently, a family of studies highlighted the unpopularity of software testing careers among undergraduate students in software engineering and computer science courses. The original study and its replications explored the perception of students in universities in four countries (Cana-da, China, India, and Malaysia), and indicated that most students do not consider a career in software testing as an option after graduation. This scenario represents a problem for the software industry since the lack of skilled testing professionals might decrease the quality of software projects and increase the number of unsuccessful projects. Goal. The present study aims to replicate, in Brazil, the studies conducted in the other four countries to establish comparisons and support the development of strategies to improve the visibility and importance of software testing among undergraduate students across the globe. Method. We followed the same protocol in the original study to collect data using a questionnaire and analyzed the answers using descriptive statistics and qualitative data analysis. Results. Our findings indicate similarities among the results obtained in Brazil in comparison to those obtained from other countries. We observed that students are not motivated to follow a testing career in the software industry based on a belief that testing activities lack challenges and opportunities for continuous learning. Conclusions. In summary, students seem to be interested in learning more about software testing. However, the lack of discussions about the theme in software development courses, as well as the limited offer of courses focused on software quality at the university level reduce the visibility of this area, which causes a decrease in the interest in this career.
PeQuENet: Perceptual Quality Enhancement of Compressed Video with Adaptation- and Attention-based Network
Venue: arXiv cs.CV 2022
Year: 2022

In this paper we propose a generative adversarial network (GAN) framework to enhance the perceptual quality of compressed videos. Our framework includes attention and adaptation to different quantization parameters (QPs) in a single model. The attention module exploits global receptive fields that can capture and align long-range correlations between consecutive frames, which can be beneficial for enhancing perceptual quality of videos. The frame to be enhanced is fed into the deep network together with its neighboring frames, and in the first stage features at different depths are extracted. Then extracted features are fed into attention blocks to explore global temporal correlations, followed by a series of upsampling and convolution layers. Finally, the resulting features are processed by the QP-conditional adaptation module which leverages the corresponding QP information. In this way, a single model can be used to enhance adaptively to various QPs without requiring multiple models specific for every QP value, while having similar performance. Experimental results demonstrate the superior performance of the proposed PeQuENet compared with the state-of-the-art compressed video quality enhancement algorithms.