OpenRaiser/Intern-Atlas huggingface dataset:
Do Neural Networks Trained with Topological Features Learn Different Internal Representations?
Venue: arXiv cs.LG 2022
Year: 2022

There is a growing body of work that leverages features extracted via topological data analysis to train machine learning models. While this field, sometimes known as topological machine learning (TML), has seen some notable successes, an understanding of how the process of learning from topological features differs from the process of learning from raw data is still limited. In this work, we begin to address one component of this larger issue by asking whether a model trained with topological features learns internal representations of data that are fundamentally different than those learned by a model trained with the original raw data. To quantify ``different'', we exploit two popular metrics that can be used to measure the similarity of the hidden representations of data within neural networks, neural stitching and centered kernel alignment. From these we draw a range of conclusions about how training with topological features does and does not change the representations that a model learns. Perhaps unsurprisingly, we find that structurally, the hidden representations of models trained and evaluated on topological features differ substantially compared to those trained and evaluated on the corresponding raw data. On the other hand, our experiments show that in some cases, these representations can be reconciled (at least to the degree required to solve the corresponding task) using a simple affine transformation. We conjecture that this means that neural networks trained on raw data may extract some limited topological features in the process of making predictions.
Revisiting Attention Weights as Explanations from an Information Theoretic Perspective
Venue: arXiv cs.CL 2022
Year: 2022

Attention mechanisms have recently demonstrated impressive performance on a range of NLP tasks, and attention scores are often used as a proxy for model explainability. However, there is a debate on whether attention weights can, in fact, be used to identify the most important inputs to a model. We approach this question from an information theoretic perspective by measuring the mutual information between the model output and the hidden states. From extensive experiments, we draw the following conclusions: (i) Additive and Deep attention mechanisms are likely to be better at preserving the information between the hidden states and the model output (compared to Scaled Dot-product); (ii) ablation studies indicate that Additive attention can actively learn to explain the importance of its input hidden representations; (iii) when attention values are nearly the same, the rank order of attention values is not consistent with the rank order of the mutual information(iv) Using Gumbel-Softmax with a temperature lower than one, tends to produce a more skewed attention score distribution compared to softmax and hence is a better choice for explainable design; (v) some building blocks are better at preserving the correlation between the ordered list of mutual information and attention weights order (for e.g., the combination of BiLSTM encoder and Additive attention). Our findings indicate that attention mechanisms do have the potential to function as a shortcut to model explanations when they are carefully combined with other model elements.
(When) Are Contrastive Explanations of Reinforcement Learning Helpful?
Venue: arXiv cs.LG 2022
Year: 2022

Global explanations of a reinforcement learning (RL) agent's expected behavior can make it safer to deploy. However, such explanations are often difficult to understand because of the complicated nature of many RL policies. Effective human explanations are often contrastive, referencing a known contrast (policy) to reduce redundancy. At the same time, these explanations also require the additional effort of referencing that contrast when evaluating an explanation. We conduct a user study to understand whether and when contrastive explanations might be preferable to complete explanations that do not require referencing a contrast. We find that complete explanations are generally more effective when they are the same size or smaller than a contrastive explanation of the same policy, and no worse when they are larger. This suggests that contrastive explanations are not sufficient to solve the problem of effectively explaining reinforcement learning policies, and require additional careful study for use in this context.
The Birds Need Attention Too: Analysing usage of Self Attention in identifying bird calls in soundscapes
Venue: arXiv cs.MM 2022
Year: 2022

Birds are vital parts of ecosystems across the world and are an excellent measure of the quality of life on earth. Many bird species are endangered while others are already extinct. Ecological efforts in understanding and monitoring bird populations are important to conserve their habitat and species, but this mostly relies on manual methods in rough terrains. Recent advances in Machine Learning and Deep Learning have made automatic bird recognition in diverse environments possible. Birdcall recognition till now has been performed using convolutional neural networks. In this work, we try and understand how self-attention can aid in this endeavor. With that we build an pre-trained Attention-based Spectrogram Transformer baseline for BirdCLEF 2022 and compare the results against the pre-trained Convolution-based baseline. Our results show that the transformer models outperformed the convolutional model and we further validate our results by building baselines and analyzing the results for the previous year BirdCLEF 2021 challenge. Source code available at https://github.com/ck090/BirdCLEF-22
Aligning Learners' Expectations and Performance by Learning Analytics Systemwith a Predictive Model
Venue: arXiv cs.HC 2022
Year: 2022

Learning analytics (LA) is data collection, analysis, and representation of data about learners in order to improve their learning and performance. Furthermore, LA opens the door to opportunities for self-regulated learning in higher education, a circular process in which learners activate and sustain behaviours that are oriented toward their personal learning goals. The potentials of LA and self-regulated learning are huge; however, they are not yet widely applied in higher education institutions. Slovenian higher education institutions have lagged behind other European countries in LA adoption. Our research aims to fill this gap by using a qualitatively and quantitatively led workflow for building a requirement-oriented LA solution, consisting of empirically gathering the students' expectations of LA and presenting a dashboard solution. Translated Student Expectations of Learning Analytics Questionnaire and focus groups were used to gather expectations from learners. Based on them, a user interface utilizing LA and grade prediction with an AI model was implemented for a selected course. The interface includes early grade prediction, peer comparison, and historical data overview. Grade prediction is based on a machine learning model built on users' interaction in the virtual learning environment, demographic data and lab grades. First, classification is used to determine students at risk of failing - its precision reaches 98% after the first month of the course. Second, the exact grade is predicted with the Decision Tree Regressor, reaching a mean absolute error of 11.2grade points (on a 100 points scale) after the first month. The proposed system's main benefit is the support for self-regulation of the learning process during the semester, possibly motivating students to adjust their learning strategies to prevent failing the course. Initial student evaluation showed positive results.
Generative Aspect-Based Sentiment Analysis with Contrastive Learning and Expressive Structure
Venue: arXiv cs.CL 2022
Year: 2022

Generative models have demonstrated impressive results on Aspect-based Sentiment Analysis (ABSA) tasks, particularly for the emerging task of extracting Aspect-Category-Opinion-Sentiment (ACOS) quadruples. However, these models struggle with implicit sentiment expressions, which are commonly observed in opinionated content such as online reviews. In this work, we introduce GEN-SCL-NAT, which consists of two techniques for improved structured generation for ACOS quadruple extraction. First, we propose GEN-SCL, a supervised contrastive learning objective that aids quadruple prediction by encouraging the model to produce input representations that are discriminable across key input attributes, such as sentiment polarity and the existence of implicit opinions and aspects. Second, we introduce GEN-NAT, a new structured generation format that better adapts autoregressive encoder-decoder models to extract quadruples in a generative fashion. Experimental results show that GEN-SCL-NAT achieves top performance across three ACOS datasets, averaging 1.48% F1 improvement, with a maximum 1.73% increase on the LAPTOP-L1 dataset. Additionally, we see significant gains on implicit aspect and opinion splits that have been shown as challenging for existing ACOS approaches.
Robot Operating System 2: Design, Architecture, and Uses In The Wild
Venue: arXiv cs.RO 2022
Year: 2022

The next chapter of the robotics revolution is well underway with the deployment of robots for a broad range of commercial use-cases. Even in a myriad of applications and environments, there exists a common vocabulary of components that robots share - the need for a modular, scalable, and reliable architecture; sensing; planning; mobility; and autonomy. The Robot Operating System (ROS) was an integral part of the last chapter, demonstrably expediting robotics research with freely-available components and a modular framework. However, ROS 1 was not designed with many necessary production-grade features and algorithms. ROS 2 and its related projects have been redesigned from the ground up to meet the challenges set forth by modern robotic systems in new and exploratory domains at all scales. In this review, we highlight the philosophical and architectural changes of ROS 2 powering this new chapter in the robotics revolution. We also show through case studies the influence ROS 2 and its adoption has had on accelerating real robot systems to reliable deployment in an assortment of challenging environments.
Improving Children's Speech Recognition by Fine-tuning Self-supervised Adult Speech Representations
Venue: arXiv cs.CL 2022
Year: 2022

Children's speech recognition is a vital, yet largely overlooked domain when building inclusive speech technologies. The major challenge impeding progress in this domain is the lack of adequate child speech corpora; however, recent advances in self-supervised learning have created a new opportunity for overcoming this problem of data scarcity. In this paper, we leverage self-supervised adult speech representations and use three well-known child speech corpora to build models for children's speech recognition. We assess the performance of fine-tuning on both native and non-native children's speech, examine the effect of cross-domain child corpora, and investigate the minimum amount of child speech required to fine-tune a model which outperforms a state-of-the-art adult model. We also analyze speech recognition performance across children's ages. Our results demonstrate that fine-tuning with cross-domain child corpora leads to relative improvements of up to 46.08% and 45.53% for native and non-native child speech respectively, and absolute improvements of 14.70% and 31.10%. We also show that with as little as 5 hours of transcribed children's speech, it is possible to fine-tune a children's speech recognition system that outperforms a state-of-the-art adult model fine-tuned on 960 hours of adult speech.
Edge2Vec: A High Quality Embedding for the Jigsaw Puzzle Problem
Venue: arXiv cs.CV 2022
Year: 2022

Pairwise compatibility measure (CM) is a key component in solving the jigsaw puzzle problem (JPP) and many of its recently proposed variants. With the rapid rise of deep neural networks (DNNs), a trade-off between performance (i.e., accuracy) and computational efficiency has become a very significant issue. Whereas an end-to-end DNN-based CM model exhibits high performance, it becomes virtually infeasible on very large puzzles, due to its highly intensive computation. On the other hand, exploiting the concept of embeddings to alleviate significantly the computational efficiency, has resulted in degraded performance, according to recent studies. This paper derives an advanced CM model (based on modified embeddings and a new loss function, called hard batch triplet loss) for closing the above gap between speed and accuracy; namely a CM model that achieves SOTA results in terms of performance and efficiency combined. We evaluated our newly derived CM on three commonly used datasets, and obtained a reconstruction improvement of 5.8% and 19.5% for so-called Type-1 and Type-2 problem variants, respectively, compared to best known results due to previous CMs.
Massively Parallel Algorithms for $b$-Matching
Venue: arXiv cs.DS 2022
Year: 2022

This paper presents an $O(\log\log \bar{d})$ round massively parallel algorithm for $1+ε$ approximation of maximum weighted $b$-matchings, using near-linear memory per machine. Here $\bar{d}$ denotes the average degree in the graph and $ε$ is an arbitrarily small positive constant. Recall that $b$-matching is the natural and well-studied generalization of the matching problem where different vertices are allowed to have multiple (and differing number of) incident edges in the matching. Concretely, each vertex $v$ is given a positive integer budget $b_v$ and it can have up to $b_v$ incident edges in the matching. Previously, there were known algorithms with round complexity $O(\log\log n)$, or $O(\log\log Δ)$ where $Δ$ denotes maximum degree, for $1+ε$ approximation of weighted matching and for maximal matching [Czumaj et al., STOC'18, Ghaffari et al. PODC'18; Assadi et al. SODA'19; Behnezhad et al. FOCS'19; Gamlath et al. PODC'19], but these algorithms do not extend to the more general $b$-matching problem.
Multi-Player Bandits Robust to Adversarial Collisions
Venue: arXiv cs.LG 2022
Year: 2022

Motivated by cognitive radios, stochastic Multi-Player Multi-Armed Bandits has been extensively studied in recent years. In this setting, each player pulls an arm, and receives a reward corresponding to the arm if there is no collision, namely the arm was selected by one single player. Otherwise, the player receives no reward if collision occurs. In this paper, we consider the presence of malicious players (or attackers) who obstruct the cooperative players (or defenders) from maximizing their rewards, by deliberately colliding with them. We provide the first decentralized and robust algorithm RESYNC for defenders whose performance deteriorates gracefully as $\tilde{O}(C)$ as the number of collisions $C$ from the attackers increases. We show that this algorithm is order-optimal by proving a lower bound which scales as $Ω(C)$. This algorithm is agnostic to the algorithm used by the attackers and agnostic to the number of collisions $C$ faced from attackers.
Relationship of the language distance to English ability of a country
Venue: arXiv cs.CL 2022
Year: 2022

Language difference is one of the factors that hinder the acquisition of second language skills. In this article, we introduce a novel solution that leverages the strength of deep neural networks to measure the semantic dissimilarity between languages based on their word distributions in the embedding space of the multilingual pre-trained language model (e.g.,BERT). Then, we empirically examine the effectiveness of the proposed semantic language distance (SLD) in explaining the consistent variation in English ability of countries, which is proxied by their performance in the Internet-Based Test of English as Foreign Language (TOEFL iBT). The experimental results show that the language distance demonstrates negative influence on a country's average English ability. Interestingly, the effect is more significant on speaking and writing subskills, which pertain to the productive aspects of language learning. Besides, we provide specific recommendations for future research directions.
Operation-level Concurrent Transaction Execution for Blockchains
Venue: arXiv cs.DC 2022
Year: 2022

Despite the success in various scenarios, blockchain systems, especially EVM-compatible ones that serially execute transactions, still face the significant challenge of limited throughput. Concurrent transaction execution is a promising technique to accelerate transaction processing and increase the overall throughput. Existing concurrency control algorithms, however, fail to obtain enough speedups in real-world blockchains due to the high-contention workloads. In this paper, we propose a novel operation-level concurrency control algorithm designed for blockchains. The core idea behind our algorithm is that only operations depending on conflicts should be executed serially, while all other conflict-free operations can be executed concurrently. Therefore, in contrast to the traditional approaches, which block or abort the entire transaction when encountering conflicts, our algorithm introduces a redo phase to resolve conflicts at the operation level by re-executing conflicting operations only. We also develop a set of data dependency tracking mechanisms to achieve precise identification and speedy re-execution for conflicting operations. We implement an open-source prototype based on Go Ethereum and evaluate it using real-world Ethereum blocks. The evaluation results show that our algorithm achieves an average speedup of 4.28$\times$. If combined with state prefetching techniques, our approach can further accelerate the transaction execution by 7.11$\times$.
Robust Quantum Circuit for Clique Problem with Intermediate Qudits
Venue: arXiv cs.ET 2022
Year: 2022

Clique problem has a wide range of applications due to its pattern matching ability. There are various formulation of clique problem like $k$-clique problem, maximum clique problem, etc. The $k$-Clique problem, determines whether an arbitrary network has a clique or not whereas maximum clique problem finds the largest clique in a graph. It is already exhibited in the literature that the $k$-clique or maximum clique problem (NP-problem) can be solved in an asymptotically faster manner by using quantum algorithms as compared to the conventional computing. Quantum computing with higher dimensions is gaining popularity due to its large storage capacity and computation power. In this article, we have shown an improved quantum circuit implementation for the $k$-clique problem and maximum clique problem (MCP) with the help of higher-dimensional intermediate temporary qudits for the first time to the best of our knowledge. The cost of state-of-the-art quantum circuit for $k$-clique problem is colossal due to a huge number of $n$-qubit Toffoli gates. We have exhibited an improved cost and depth over the circuit by applying a generalized $n$-qubit Toffoli gate decomposition with intermediate ququarts (4-dimensional qudits).
NeRFFaceEditing: Disentangled Face Editing in Neural Radiance Fields
Venue: arXiv cs.GR 2022
Year: 2022

Recent methods for synthesizing 3D-aware face images have achieved rapid development thanks to neural radiance fields, allowing for high quality and fast inference speed. However, existing solutions for editing facial geometry and appearance independently usually require retraining and are not optimized for the recent work of generation, thus tending to lag behind the generation process. To address these issues, we introduce NeRFFaceEditing, which enables editing and decoupling geometry and appearance in the pretrained tri-plane-based neural radiance field while retaining its high quality and fast inference speed. Our key idea for disentanglement is to use the statistics of the tri-plane to represent the high-level appearance of its corresponding facial volume. Moreover, we leverage a generated 3D-continuous semantic mask as an intermediary for geometry editing. We devise a geometry decoder (whose output is unchanged when the appearance changes) and an appearance decoder. The geometry decoder aligns the original facial volume with the semantic mask volume. We also enhance the disentanglement by explicitly regularizing rendered images with the same appearance but different geometry to be similar in terms of color distribution for each facial component separately. Our method allows users to edit via semantic masks with decoupled control of geometry and appearance. Both qualitative and quantitative evaluations show the superior geometry and appearance control abilities of our method compared to existing and alternative solutions.
Foveated Rendering: a State-of-the-Art Survey
Venue: arXiv cs.GR 2022
Year: 2022

Recently, virtual reality (VR) technology has been widely used in medical, military, manufacturing, entertainment, and other fields. These applications must simulate different complex material surfaces, various dynamic objects, and complex physical phenomena, increasing the complexity of VR scenes. Current computing devices cannot efficiently render these complex scenes in real time, and delayed rendering makes the content observed by the user inconsistent with the user's interaction, causing discomfort. Foveated rendering is a promising technique that can accelerate rendering. It takes advantage of human eyes' inherent features and renders different regions with different qualities without sacrificing perceived visual quality. Foveated rendering research has a history of 31 years and is mainly focused on solving the following three problems. The first is to apply perceptual models of the human visual system into foveated rendering. The second is to render the image with different qualities according to foveation principles. The third is to integrate foveated rendering into existing rendering paradigms to improve rendering performance. In this survey, we review foveated rendering research from 1990 to 2021. We first revisit the visual perceptual models related to foveated rendering. Subsequently, we propose a new foveated rendering taxonomy and then classify and review the research on this basis. Finally, we discuss potential opportunities and open questions in the foveated rendering field. We anticipate that this survey will provide new researchers with a high-level overview of the state of the art in this field, furnish experts with up-to-date information and offer ideas alongside a framework to VR display software and hardware designers and engineers.
Security Closure of IC Layouts Against Hardware Trojans
Venue: arXiv cs.CR 2022
Year: 2022

Due to cost benefits, supply chains of integrated circuits (ICs) are largely outsourced nowadays. However, passing ICs through various third-party providers gives rise to many threats, like piracy of IC intellectual property or insertion of hardware Trojans, i.e., malicious circuit modifications. In this work, we proactively and systematically harden the physical layouts of ICs against post-design insertion of Trojans. Toward that end, we propose a multiplexer-based logic-locking scheme that is (i) devised for layout-level Trojan prevention, (ii) resilient against state-of-the-art, oracle-less machine learning attacks, and (iii) fully integrated into a tailored, yet generic, commercial-grade design flow. Our work provides in-depth security and layout analysis on a challenging benchmark suite. We show that ours can render layouts resilient, with reasonable overheads, against Trojan insertion in general and also against second-order attacks (i.e., adversaries seeking to bypass the locking defense in an oracle-less setting). We release our layout artifacts for independent verification [29] and we will release our methodology's source code.