OpenRaiser/Intern-Atlas huggingface dataset:
Interpreting Song Lyrics with an Audio-Informed Pre-trained Language Model
Venue: arXiv cs.SD 2022
Year: 2022

Lyric interpretations can help people understand songs and their lyrics quickly, and can also make it easier to manage, retrieve and discover songs efficiently from the growing mass of music archives. In this paper we propose BART-fusion, a novel model for generating lyric interpretations from lyrics and music audio that combines a large-scale pre-trained language model with an audio encoder. We employ a cross-modal attention module to incorporate the audio representation into the lyrics representation to help the pre-trained language model understand the song from an audio perspective, while preserving the language model's original generative performance. We also release the Song Interpretation Dataset, a new large-scale dataset for training and evaluating our model. Experimental results show that the additional audio information helps our model to understand words and music better, and to generate precise and fluent interpretations. An additional experiment on cross-modal music retrieval shows that interpretations generated by BART-fusion can also help people retrieve music more accurately than with the original BART.
Learned Lossless JPEG Transcoding via Joint Lossy and Residual Compression
Venue: arXiv cs.CV 2022
Year: 2022

As a commonly-used image compression format, JPEG has been broadly applied in the transmission and storage of images. To further reduce the compression cost while maintaining the quality of JPEG images, lossless transcoding technology has been proposed to recompress the compressed JPEG image in the DCT domain. Previous works, on the other hand, typically reduce the redundancy of DCT coefficients and optimize the probability prediction of entropy coding in a hand-crafted manner that lacks generalization ability and flexibility. To tackle the above challenge, we propose the learned lossless JPEG transcoding framework via Joint Lossy and Residual Compression. Instead of directly optimizing the entropy estimation, we focus on the redundancy that exists in the DCT coefficients. To the best of our knowledge, we are the first to utilize the learned end-to-end lossy transform coding to reduce the redundancy of DCT coefficients in a compact representational domain. We also introduce residual compression for lossless transcoding, which adaptively learns the distribution of residual DCT coefficients before compressing them using context-based entropy coding. Our proposed transcoding architecture shows significant superiority in the compression of JPEG images thanks to the collaboration of learned lossy transform coding and residual entropy coding. Extensive experiments on multiple datasets have demonstrated that our proposed framework can achieve about 21.49% bits saving in average based on JPEG compression, which outperforms the typical lossless transcoding framework JPEG-XL by 3.51%.
SOniCS: Develop intuition on biomechanical systems through interactive error controlled simulations
Venue: arXiv cs.MS 2022
Year: 2022

This new approach allows the user to experiment with model choices easily and quickly without requiring in-depth expertise, as constitutive models can be modified by one line of code only. This ease in building new models makes SOniCS ideal to develop surrogate, reduced order models and to train machine learning algorithms for uncertainty quantification or to enable patient-specific simulations. SOniCS is thus not only a tool that facilitates the development of surgical training simulations but also, and perhaps more importantly, paves the way to increase the intuition of users or otherwise non-intuitive behaviors of (bio)mechanical systems. The plugin uses new developments of the FEniCSx project enabling automatic generation with FFCx of finite element tensors such as the local residual vector and Jacobian matrix. We validate our approach with numerical simulations such as manufactured solutions, cantilever beams, and benchmarks provided by FEBio. We reach machine precision accuracy and demonstrate the use of the plugin for a real-time haptic simulation involving a surgical tool controlled by the user in contact with a hyperelastic liver. We include complete examples showing the use of our plugin for simulations involving Saint Venant-Kirchhoff, Neo-Hookean, Mooney-Rivlin, and Holzapfel Ogden anisotropic models as supplementary material.
Team Precoding Towards Scalable Cell-free Massive MIMO Networks
Venue: arXiv cs.IT 2022
Year: 2022

In a recent work, we studied a novel precoding design for cell-free networks called team minimum mean-square error (TMMSE) precoding, which rigorously generalizes centralized MMSE precoding to distributed operations based on transmitter-specific channel state information (CSI). Despite its flexibility in handling different cooperation regimes at the CSI sharing level, TMMSE precoding assumes network-wide sharing of the data bearing signals, and hence it is inherently not scalable. In this work, inspired by recent advances on scalable cell-free architectures based on user-centric network clustering techniques, we address this issue by proposing a novel version of the TMMSE precoding design covering partial message sharing. The obtained framework is then successfully applied to derive a variety of novel, optimal, and efficient precoding schemes for a user-centric cell-free network deployed using multiple radio stripes. Numerical simulations of a typical industrial internet-of-things scenario corroborate the gains of TMMSE precoding over competing schemes in terms of spectral efficiency under different power constraints. Although presented in the context of downlink precoding, the results of this paper may be applied also on the uplink.
Bugs in the Data: How ImageNet Misrepresents Biodiversity
Venue: arXiv cs.CV 2022
Year: 2022

ImageNet-1k is a dataset often used for benchmarking machine learning (ML) models and evaluating tasks such as image recognition and object detection. Wild animals make up 27% of ImageNet-1k but, unlike classes representing people and objects, these data have not been closely scrutinized. In the current paper, we analyze the 13,450 images from 269 classes that represent wild animals in the ImageNet-1k validation set, with the participation of expert ecologists. We find that many of the classes are ill-defined or overlapping, and that 12% of the images are incorrectly labeled, with some classes having >90% of images incorrect. We also find that both the wildlife-related labels and images included in ImageNet-1k present significant geographical and cultural biases, as well as ambiguities such as artificial animals, multiple species in the same image, or the presence of humans. Our findings highlight serious issues with the extensive use of this dataset for evaluating ML systems, the use of such algorithms in wildlife-related tasks, and more broadly the ways in which ML datasets are commonly created and curated.
A Survey of Open Source Automation Tools for Data Science Predictions
Venue: arXiv cs.LG 2022
Year: 2022

We present an expository overview of technical and cultural challenges to the development and adoption of automation at various stages in the data science prediction lifecycle, restricting focus to supervised learning with structured datasets. In addition, we review popular open source Python tools implementing common solution patterns for the automation challenges and highlight gaps where we feel progress still demands to be made.
Benchmarking Human Face Similarity Using Identical Twins
Venue: arXiv cs.CV 2022
Year: 2022

The problem of distinguishing identical twins and non-twin look-alikes in automated facial recognition (FR) applications has become increasingly important with the widespread adoption of facial biometrics. Due to the high facial similarity of both identical twins and look-alikes, these face pairs represent the hardest cases presented to facial recognition tools. This work presents an application of one of the largest twin datasets compiled to date to address two FR challenges: 1) determining a baseline measure of facial similarity between identical twins and 2) applying this similarity measure to determine the impact of doppelgangers, or look-alikes, on FR performance for large face datasets. The facial similarity measure is determined via a deep convolutional neural network. This network is trained on a tailored verification task designed to encourage the network to group together highly similar face pairs in the embedding space and achieves a test AUC of 0.9799. The proposed network provides a quantitative similarity score for any two given faces and has been applied to large-scale face datasets to identify similar face pairs. An additional analysis which correlates the comparison score returned by a facial recognition tool and the similarity score returned by the proposed network has also been performed.
Polyhedral Specification and Code Generation of Sparse Tensor Contraction with Co-Iteration
Venue: arXiv cs.PL 2022
Year: 2022

This paper presents a code generator for sparse tensor contraction computations. It leverages a mathematical representation of loop nest computations in the sparse polyhedral framework (SPF), which extends the polyhedral model to support non-affine computations, such as arise in sparse tensors. SPF is extended to perform layout specification, optimization, and code generation of sparse tensor code: 1) we develop a polyhedral layout specification that decouples iteration spaces for layout and computation; and, 2) we develop efficient co-iteration of sparse tensors by combining polyhedra scanning over the layout of one sparse tensor with the synthesis of code to find corresponding elements in other tensors through an SMT solver. We compare the generated code with that produced by a state-of-the-art tensor compiler, TACO. We achieve on average 1.63$\times$ faster parallel performance than TACO on sparse-sparse co-iteration and describe how to improve that to 2.72$\times$ average speedup by switching the find algorithms. We also demonstrate that decoupling iteration spaces of layout and computation enables additional layout and computation combinations to be supported.
Autonomous social robot navigation in unknown urban environments using semantic segmentation
Venue: arXiv cs.RO 2022
Year: 2022

For autonomous robots navigating in urban environments, it is important for the robot to stay on the designated path of travel (i.e., the footpath), and avoid areas such as grass and garden beds, for safety and social conformity considerations. This paper presents an autonomous navigation approach for unknown urban environments that combines the use of semantic segmentation and LiDAR data. The proposed approach uses the segmented image mask to create a 3D obstacle map of the environment, from which, the boundaries of the footpath is computed. Compared to existing methods, our approach does not require a pre-built map and provides a 3D understanding of the safe region of travel, enabling the robot to plan any path through the footpath. Experiments comparing our method with two alternatives using only LiDAR or only semantic segmentation show that overall our proposed approach performs significantly better with greater than 91% success rate outdoors, and greater than 66% indoors. Our method enabled the robot to remain on the safe path of travel at all times, and reduced the number of collisions.
Rail break and derailment prediction using Probabilistic Graphical Modelling
Venue: arXiv cs.LG 2022
Year: 2022

Rail breaks are one of the most common causes of derailments internationally. This is no different for the South African Iron Ore line. Many rail breaks occur as a heavy-haul train passes over a crack, large defect or defective weld. In such cases, it is usually too late for the train to slow down in time to prevent a de-railment. Knowing the risk of a rail break occurring associated with a train passing over a section of rail allows for better implementation of maintenance initiatives and mitigating measures. In this paper the Ore Line's specific challenges are discussed and the currently available data that can be used to create a rail break risk prediction model is reviewed. The development of a basic rail break risk prediction model for the Ore Line is then presented. Finally the insight gained from the model is demonstrated by means of discussing various scenarios of various rail break risk. In future work, we are planning on extending this basic model to allow input from live monitoring systems such as the ultrasonic broken rail detection system.
TEP-GNN: Accurate Execution Time Prediction of Functional Tests using Graph Neural Networks
Venue: arXiv cs.SE 2022
Year: 2022

Predicting the performance of production code prior to actually executing or benchmarking it is known to be highly challenging. In this paper, we propose a predictive model, dubbed TEP-GNN, which demonstrates that high-accuracy performance prediction is possible for the special case of predicting unit test execution times. TEP-GNN uses FA-ASTs, or flow-augmented ASTs, as a graph-based code representation approach, and predicts test execution times using a powerful graph neural network (GNN) deep learning model. We evaluate TEP-GNN using four real-life Java open source programs, based on 922 test files mined from the projects' public repositories. We find that our approach achieves a high Pearson correlation of 0.789, considerable outperforming a baseline deep learning model. However, we also find that more work is needed for trained models to generalize to unseen projects. Our work demonstrates that FA-ASTs and GNNs are a feasible approach for predicting absolute performance values, and serves as an important intermediary step towards being able to predict the performance of arbitrary code prior to execution.
Design of an Efficient CSI Feedback Mechanism in Massive MIMO Systems: A Machine Learning Approach using Empirical Data
Venue: arXiv cs.IT 2022
Year: 2022

Massive multiple-input multiple-output (mMIMO) regime reaps the benefits of spatial diversity and multiplexing gains, subject to precise channel state information (CSI) acquisition. In the current communication architecture, the downlink CSI is estimated by the user equipment (UE) via dedicated pilots and then fed back to the gNodeB (gNB). The feedback information is compressed with the goal of reducing over-the-air overhead. This compression increases the inaccuracy of acquired CSI, thus degrading the overall spectral efficiency. This paper proposes a computationally inexpensive machine learning (ML)-based CSI feedback algorithm, which exploits twin channel predictors. The proposed approach can work for both time-division duplex (TDD) and frequency-division duplex (FDD) systems, and it allows to reduce feedback overhead and improves the acquired CSI accuracy. To observe real benefits, we demonstrate the performance of the proposed approach using the empirical data recorded at the Nokia campus in Stuttgart, Germany. Numerical results show the effectiveness of the proposed approach in terms of reducing overhead, minimizing quantization errors, increasing spectral efficiency, cosine similarity, and precoding gain compared to the traditional CSI feedback mechanism.
Data Augmentation for Graph Data: Recent Advancements
Venue: arXiv cs.LG 2022
Year: 2022

Graph Neural Network (GNNs) based methods have recently become a popular tool to deal with graph data because of their ability to incorporate structural information. The only hurdle in the performance of GNNs is the lack of labeled data. Data Augmentation techniques for images and text data can not be used for graph data because of the complex and non-euclidean structure of graph data. This gap has forced researchers to shift their focus towards the development of data augmentation techniques for graph data. Most of the proposed Graph Data Augmentation (GDA) techniques are task-specific. In this paper, we survey the existing GDA techniques based on different graph tasks. This survey not only provides a reference to the research community of GDA but also provides the necessary information to the researchers of other domains.
Identity-Sensitive Knowledge Propagation for Cloth-Changing Person Re-identification
Venue: arXiv cs.CV 2022
Year: 2022

Cloth-changing person re-identification (CC-ReID), which aims to match person identities under clothing changes, is a new rising research topic in recent years. However, typical biometrics-based CC-ReID methods often require cumbersome pose or body part estimators to learn cloth-irrelevant features from human biometric traits, which comes with high computational costs. Besides, the performance is significantly limited due to the resolution degradation of surveillance images. To address the above limitations, we propose an effective Identity-Sensitive Knowledge Propagation framework (DeSKPro) for CC-ReID. Specifically, a Cloth-irrelevant Spatial Attention module is introduced to eliminate the distraction of clothing appearance by acquiring knowledge from the human parsing module. To mitigate the resolution degradation issue and mine identity-sensitive cues from human faces, we propose to restore the missing facial details using prior facial knowledge, which is then propagated to a smaller network. After training, the extra computations for human parsing or face restoration are no longer required. Extensive experiments show that our framework outperforms state-of-the-art methods by a large margin. Our code is available at https://github.com/KimbingNg/DeskPro.
Prediction of the energy and exergy performance of F135 PW100 turbofan engine via deep learning
Venue: arXiv cs.LG 2022
Year: 2022

In the present study, the effects of flight-Mach number, flight altitude, fuel types, and intake air temperature on thrust specific fuel consumption, thrust, intake air mass flow rate, thermal and propulsive efficiecies, as well as the exergetic efficiency and the exergy destruction rate in F135 PW100 engine are investigated. Based on the results obtained in the first phase, to model the thermodynamic performance of the aforementioned engine cycle, Flight-Mach number and flight altitude are considered to be 2.5 and 30,000 m, respectively; due to the operational advantage of supersonic flying at high altitude flight conditions, and the higher thrust of hydrogen fuel. Accordingly, in the second phase, taking into account the mentioned flight conditions, an intelligent model has been obtained to predict output parameters (i.e., thrust, thrust specific fuel consumption, and overall exergetic efficiency) using the deep learning method. In the attained deep neural model, the pressure ratio of the high-pressure turbine, fan pressure ratio, turbine inlet temperature, intake air temperature, and bypass ratio are considered input parameters. The provided datasets are randomly divided into two sets: the first one contains 6079 samples for model training and the second set contains 1520 samples for testing. In particular, the Adam optimization algorithm, the cost function of the mean square error, and the active function of rectified linear unit are used to train the network. The results show that the error percentage of the deep neural model is equal to 5.02%, 1.43%, and 2.92% to predict thrust, thrust specific fuel consumption, and overall exergetic efficiency, respectively, which indicates the success of the attained model in estimating the output parameters of the present problem.
Towards Benchmarking Explainable Artificial Intelligence Methods
Venue: arXiv cs.AI 2022
Year: 2022

The currently dominating artificial intelligence and machine learning technology, neural networks, builds on inductive statistical learning. Neural networks of today are information processing systems void of understanding and reasoning capabilities, consequently, they cannot explain promoted decisions in a humanly valid form. In this work, we revisit and use fundamental philosophy of science theories as an analytical lens with the goal of revealing, what can be expected, and more importantly, not expected, from methods that aim to explain decisions promoted by a neural network. By conducting a case study we investigate a selection of explainability method's performance over two mundane domains, animals and headgear. Through our study, we lay bare that the usefulness of these methods relies on human domain knowledge and our ability to understand, generalise and reason. The explainability methods can be useful when the goal is to gain further insights into a trained neural network's strengths and weaknesses. If our aim instead is to use these explainability methods to promote actionable decisions or build trust in ML-models they need to be less ambiguous than they are today. In this work, we conclude from our study, that benchmarking explainability methods, is a central quest towards trustworthy artificial intelligence and machine learning.
A Feedforward Unitary Equivariant Neural Network
Venue: arXiv cs.LG 2022
Year: 2022

We devise a new type of feedforward neural network. It is equivariant with respect to the unitary group $U(n)$. The input and output can be vectors in $\mathbb{C}^n$ with arbitrary dimension $n$. No convolution layer is required in our implementation. We avoid errors due to truncated higher order terms in Fourier-like transformation. The implementation of each layer can be done efficiently using simple calculations. As a proof of concept, we have given empirical results on the prediction of the dynamics of atomic motion to demonstrate the practicality of our approach.
Multi-Scale Multi-Target Domain Adaptation for Angle Closure Classification
Venue: arXiv cs.CV 2022
Year: 2022

Deep learning (DL) has made significant progress in angle closure classification with anterior segment optical coherence tomography (AS-OCT) images. These AS-OCT images are often acquired by different imaging devices/conditions, which results in a vast change of underlying data distributions (called "data domains"). Moreover, due to practical labeling difficulties, some domains (e.g., devices) may not have any data labels. As a result, deep models trained on one specific domain (e.g., a specific device) are difficult to adapt to and thus may perform poorly on other domains (e.g., other devices). To address this issue, we present a multi-target domain adaptation paradigm to transfer a model trained on one labeled source domain to multiple unlabeled target domains. Specifically, we propose a novel Multi-scale Multi-target Domain Adversarial Network (M2DAN) for angle closure classification. M2DAN conducts multi-domain adversarial learning for extracting domain-invariant features and develops a multi-scale module for capturing local and global information of AS-OCT images. Based on these domain-invariant features at different scales, the deep model trained on the source domain is able to classify angle closure on multiple target domains even without any annotations in these domains. Extensive experiments on a real-world AS-OCT dataset demonstrate the effectiveness of the proposed method.