Reviewerly/exHarmony huggingface dataset:
AIMNet: Adaptive Image-Tag Merging Network For Automatic Medical Report Generation

In recent years, medical report generation has received increasing research interest with the goal of automatically generating long and coherent descriptive paragraphs that can de-scribe in detail the observations of normal and abnormal regions in the input medical images. Unlike general image captioning tasks, medical report generation is more challenging for data-driven neural models. This is mainly due to severe visual and textual data biases. To address these problems, we propose an Adaptive Image-Tag Merging Network (AIMNet) that first predicts the tags of diseases from the input image, and then adaptively merges the visual information of the input image and disease information from the disease tags to learn the disease-oriented visual features that can better represent abnormal regions of the input image, and thus can be used to alleviate data bias problems. The experiments and analyses on the public MIMIC-CXR and IU-Xray datasets show that our proposed AIMNet achieves the state-of-the-art results under all metrics and significantly outperforms previous models on the MIMIC-CXR and IU-Xray datasets with relatively 15.1% and 6.5% margins in terms of BLEU-4 score.
Competitive Multi-Agent Reinforcement Learning with Self-Supervised Representation

We present MASRL: Competitive Multi-Agent Self-supervised representations for Reinforcement Learning in the multi-agent competitive environment. MASRL introduces a simple but effective self-supervised task: predicting a learning agent’s opponent’s future move. In doing this, the agent learns a stronger representation from this additional signal, focusing not only on itself but also on its opponent. By understanding and anticipating the opponent’s future moves, MASRL allows the learning agent to develop effective strategies for opponent exploitation. Our method stabilizes training, improves sample efficiency, and allows the agent to generalize and adapt its playing strategy to other unseen expert opponents. On the Multi-Agent Atari benchmark, MASRL achieves remarkable performance, outperforming other strong baselines. Examples of demo videos can be found at: https://sites.google.com/view/compmarl
Proto2Proto: Can you recognize the car, the way I do?

Prototypical methods have recently gained a lot of attention due to their intrinsic interpretable nature, which is obtained through the prototypes. With growing use cases of model reuse and distillation, there is a need to also study transfer of interpretability from one model to another. We present Proto2Proto, a novel method to transfer interpretability of one prototypical part network to another via knowledge distillation. Our approach aims to add interpretability to the "dark" knowledge transferred from the teacher to the shallower student model. We propose two novel losses: "Global Explanation" loss and "Patch-Prototype Correspondence" loss to facilitate such a transfer. Global Explanation loss forces the student prototypes to be close to teacher prototypes, and Patch-Prototype Correspondence loss enforces the local representations of the student to be similar to that of the teacher. Further, we propose three novel metrics to evaluate the student's proximity to the teacher as measures of interpretability transfer in our settings. We qualitatively and quantitatively demonstrate the effectiveness of our method on CUB-200-2011 and Stanford Cars datasets. Our experiments show that the proposed method indeed achieves interpretability transfer from teacher to student while simultaneously exhibiting competitive performance.
New approach to computer-aided learning based on digital library user behavior

The author introduces the mathematical model of recurrent neural network with external memory. It is intended for predicting efficient education trajectory in digital information environments, e. g. digital libraries. The goal of computer-aided learning based on neural networks is to personalize user trajectories. In the study, user behavior is modeled for the more precise personalization in various aspects using recurrent neural networks. The method is designed for two types of recurrent neural networks, i. e. the classic one with sigmoidal activation function and that with LSTM (Long Short-Term Memory). The experiments demonstrated serious advantages of recurrent neural networks over analogous methods in predicting education trajectory. Thus, the proposed model is the more efficient in predictive accuracy (by 15–20% higher than analogous methods). Its prime application area is prediction of optimum user education trajectory in the digital information environment, and digital library, in particul
Neural Collapse Inspired Attraction-Repulsion-Balanced Loss for Imbalanced Learning

Class imbalance distribution widely exists in real-world engineering. However, the mainstream optimization algorithms that seek to minimize error will trap the deep learning model in sub-optimums when facing extreme class imbalance. It seriously harms the classification precision, especially on the minor classes. The essential reason is that the gradients of the classifier weights are imbalanced among the components from different classes. In this paper, we propose Attraction-Repulsion-Balanced Loss (ARB-Loss) to balance the different components of the gradients. We perform experiments on the large-scale classification and segmentation datasets and our ARB-Loss can achieve state-of-the-art performance via only one-stage training instead of 2-stage learning like nowadays SOTA works.
Facial Expression Manipulation for Personalized Facial Action Estimation

Limited sizes of annotated video databases of spontaneous facial expression, imbalanced action unit labels, and domain shift are three main obstacles in training models to detect facial actions and estimate their intensity. To address these problems, we propose an approach that incorporates facial expression generation for facial action unit intensity estimation. Our approach reconstructs the 3D shape of the face from each video frame, aligns the 3D mesh to a canonical view, and trains a GAN-based network to synthesize novel images with facial action units of interest. We leverage the synthetic images to achieve two goals: 1) generating AU-balanced databases, and 2) tackling domain shift with personalized networks. To generate a balanced database, we synthesize expressions with varying AU intensities and perform semantic resampling. Our experimental results on FERA17 show that networks trained on synthesized facial expressions outperform those trained on actual facial expressions and surpass current state-of-the-art approaches. To tackle domain shift, we propose personalizing pretrained networks. We generate synthetic expressions of each target subject with varying AU intensity labels and use the person-specific synthetic images to fine-tune pretrained networks. To evaluate performance of the personalized networks, we use DISFA and PAIN databases. Personalized networks, which require only a single image from each target subject to generate synthetic images, achieved significant improvement in generalizing to unseen domains.
kpfriends at SemEval-2022 Task 2: NEAMER -- Named Entity Augmented Multi-word Expression Recognizer

We present NEAMER -- Named Entity Augmented Multi-word Expression Recognizer. This system is inspired by non-compositionality characteristics shared between Named Entity and Idiomatic Expressions. We utilize transfer learning and locality features to enhance idiom classification task. This system is our submission for SemEval Task 2: Multilingual Idiomaticity Detection and Sentence Embedding Subtask A OneShot shared task. We achieve SOTA with F1 0.9395 during post-evaluation phase. We also observe improvement in training stability. Lastly, we experiment with non-compositionality knowledge transfer, cross-lingual fine-tuning and locality features, which we also introduce in this paper.
Quasi-orthogonality and intrinsic dimensions as measures of learning and generalisation

Finding best architectures of learning machines, such as deep neural networks, is a well-known technical and theoretical challenge. Recent work by Mellor et al (2021) showed that there may exist correlations between the accuracies of trained networks and the values of some easily computable measures defined on randomly initialised networks which may enable to search tens of thousands of neural architectures without training. Mellor et al used the Hamming distance evaluated over all ReLU neurons as such a measure. Motivated by these findings, in our work, we ask the question of the existence of other and perhaps more principled measures which could be used as determinants of success of a given neural architecture. In particular, we examine, if the dimensionality and quasi-orthogonality of neural networks' feature space could be correlated with the network's performance after training. We showed, using the setup as in Mellor et al, that dimensionality and quasi-orthogonality may jointly serve as network's performance discriminants. In addition to offering new opportunities to accelerate neural architecture search, our findings suggest important relationships between the networks' final performance and properties of their randomly initialised feature spaces: data dimension and quasi-orthogonality.
Improving Self-Supervised Learning-based MOS Prediction Networks

MOS (Mean Opinion Score) is a subjective method used for the evaluation of a system's quality. Telecommunications (for voice and video), and speech synthesis systems (for generated speech) are a few of the many applications of the method. While MOS tests are widely accepted, they are time-consuming and costly since human input is required. In addition, since the systems and subjects of the tests differ, the results are not really comparable. On the other hand, a large number of previous tests allow us to train machine learning models that are capable of predicting MOS value. By automatically predicting MOS values, both the aforementioned issues can be resolved. The present work introduces data-, training- and post-training specific improvements to a previous self-supervised learning-based MOS prediction model. We used a wav2vec 2.0 model pre-trained on LibriSpeech, extended with LSTM and non-linear dense layers. We introduced transfer learning, target data preprocessing a two- and three-phase training method with different batch formulations, dropout accumulation (for larger batch sizes) and quantization of the predictions. The methods are evaluated using the shared synthetic speech dataset of the first Voice MOS challenge.
IMDeception: Grouped Information Distilling Super-Resolution Network

Single-Image-Super-Resolution (SISR) is a classical computer vision problem that has benefited from the recent advancements in deep learning methods, especially the advancements of convolutional neural networks (CNN). Although state-of-the-art methods improve the performance of SISR on several datasets, direct application of these networks for practical use is still an issue due to heavy computational load. For this purpose, recently, researchers have focused on more efficient and high-performing network structures. Information multi-distilling network (IMDN) is one of the highly efficient SISR networks with high performance and low computational load. IMDN achieves this efficiency with various mechanisms such as Intermediate Information Collection (IIC), working in a global setting, Progressive Refinement Module (PRM), and Contrast Aware Channel Attention (CCA), employed in a local setting. These mechanisms, however, do not equally contribute to the efficiency and performance of IMDN. In this work, we propose the Global Progressive Refinement Module (GPRM) as a less parameter-demanding alternative to the IIC module for feature aggregation. To further decrease the number of parameters and floating point operations persecond (FLOPS), we also propose Grouped Information Distilling Blocks (GIDB). Using the proposed structures, we design an efficient SISR network called IMDeception. Experiments reveal that the proposed network performs on par with state-of-the-art models despite having a limited number of parameters and FLOPS. Furthermore, using grouped convolutions as a building block of GIDB increases room for further optimization during deployment. To show its potential, the proposed model was deployed on NVIDIA Jetson Xavier AGX and it has been shown that it can run in real-time on this edge device
Masked Image Modeling Advances 3D Medical Image Analysis

Recently, masked image modeling (MIM) has gained considerable attention due to its capacity to learn from vast amounts of unlabeled data and has been demonstrated to be effective on a wide variety of vision tasks involving natural images. Meanwhile, the potential of self-supervised learning in modeling 3D medical images is anticipated to be immense due to the high quantities of unlabeled images, and the expense and difficulty of quality labels. However, MIM's applicability to medical images remains uncertain. In this paper, we demonstrate that masked image modeling approaches can also advance 3D medical images analysis in addition to natural images. We study how masked image modeling strategies leverage performance from the viewpoints of 3D medical image segmentation as a representative downstream task: i) when compared to naive contrastive learning, masked image modeling approaches accelerate the convergence of supervised training even faster (1.40$\times$) and ultimately produce a higher dice score; ii) predicting raw voxel values with a high masking ratio and a relatively smaller patch size is non-trivial self-supervised pretext-task for medical images modeling; iii) a lightweight decoder or projection head design for reconstruction is powerful for masked image modeling on 3D medical images which speeds up training and reduce cost; iv) finally, we also investigate the effectiveness of MIM methods under different practical scenarios where different image resolutions and labeled data ratios are applied.
Neural-Network Decoders for Measurement Induced Phase Transitions

Open quantum systems have been shown to host a plethora of exotic dynamical phases. Measurement-induced entanglement phase transitions in monitored quantum systems are a striking example of this phenomena. However, naive realizations of such phase transitions requires an exponential number of repetitions of the experiment which is practically unfeasible on large systems. Recently, it has been proposed that these phase transitions can be probed locally via entangling reference qubits and studying their purification dynamics. In this work, we leverage modern machine learning tools to devise a neural network decoder to determine the state of the reference qubits conditioned on the measurement outcomes. We show that the entanglement phase transition manifests itself as a stark change in the learnability of the decoder function. We study the complexity and scalability of this approach and discuss how it can be utilized to detect entanglement phase transitions in generic experiments.
Counterfactual Learning To Rank for Utility-Maximizing Query Autocompletion

Conventional methods for query autocompletion aim to predict which completed query a user will select from a list. A shortcoming of this approach is that users often do not know which query will provide the best retrieval performance on the current information retrieval system, meaning that any query autocompletion methods trained to mimic user behavior can lead to suboptimal query suggestions. To overcome this limitation, we propose a new approach that explicitly optimizes the query suggestions for downstream retrieval performance. We formulate this as a problem of ranking a set of rankings, where each query suggestion is represented by the downstream item ranking it produces. We then present a learning method that ranks query suggestions by the quality of their item rankings. The algorithm is based on a counterfactual learning approach that is able to leverage feedback on the items (e.g., clicks, purchases) to evaluate query suggestions through an unbiased estimator, thus avoiding the assumption that users write or select optimal queries. We establish theoretical support for the proposed approach and provide learning-theoretic guarantees. We also present empirical results on publicly available datasets, and demonstrate real-world applicability using data from an online shopping store.
Towards Privacy-Preserving Neural Architecture Search

Machine learning promotes the continuous development of signal processing in various fields, including network traffic monitoring, EEG classification, face identification, and many more. However, massive user data collected for training deep learning models raises privacy concerns and increases the difficulty of manually adjusting the network structure. To address these issues, we propose a privacy-preserving neural architecture search (PP-NAS) framework based on secure multi-party computation to protect users' data and the model's parameters/hyper-parameters. PP-NAS outsources the NAS task to two non-colluding cloud servers for making full advantage of mixed protocols design. Complement to the existing PP machine learning frameworks, we redesign the secure ReLU and Max-pooling garbled circuits for significantly better efficiency ($3 \sim 436$ times speed-up). We develop a new alternative to approximate the Softmax function over secret shares, which bypasses the limitation of approximating exponential operations in Softmax while improving accuracy. Extensive analyses and experiments demonstrate PP-NAS's superiority in security, efficiency, and accuracy.
Blockchain-Oriented Software Variant Forks: A Preliminary Study

In collaborative social development platforms such as GitHub, forking a repository is a common activity. A variant fork wants to split the development from the original repository and grow towards a different direction. In this preliminary exploratory research, we analyze the possible reasons for creating a variant fork in blockchain-oriented software. By collecting repositories in GitHub, we created a dataset with repositories and their variants, from which we manually analyzed 86 variants. Based on the variants we studied, the main reason to create a variant in blockchain-oriented software is to support a different blockchain platform (65%).
An overview of the Eight International Olympiad in Cryptography "Non-Stop University CRYPTO"

Non-Stop University CRYPTO is the International Olympiad in Cryptography that was held for the eight time in 2021. Hundreds of university and school students, professionals from 33 countries worked on mathematical problems in cryptography during a week. The aim of the Olympiad is to attract attention to curious and even open scientific problems of modern cryptography. In this paper, problems and their solutions of the Olympiad'2021 are presented. We consider 19 problems of varying difficulty and topics: ciphers, online machines, passwords, binary strings, permutations, quantum circuits, historical ciphers, elliptic curves, masking, implementation on a chip, etc. We discuss several open problems on quantum error correction, finding special permutations and s-Boolean sharing of a function, obtaining new bounds on the distance to affine vectorial functions.
Quantum Error Detection Without Using Ancilla Qubits

In this paper, we describe and experimentally demonstrate an error detection scheme that does not employ ancilla qubits or mid-circuit measurements. This is achieved by expanding the Hilbert space where a single logical qubit is encoded using several physical qubits. For example, one possible two qubit encoding identifies $|0\rangle_L=|01\rangle$ and $|1\rangle_L=|10\rangle$. If during the final measurement a $|11\rangle$ or $|00\rangle$ is observed an error is declared and the run is not included in subsequent analysis. We provide codewords for a simple bit-flip encoding, a way to encode the states, a way to implement logical $U_3$ and logical $C_x$ gates, and a description of which errors can be detected. We then run Greenberger-Horne-Zeilinger circuits on the transmon based IBM quantum computers, with an input space of $N\in\{2,3,4,5\}$ logical qubits and $Q\in\{1,2,3,4,5\}$ physical qubits per logical qubit. The results are compared relative to $Q=1$ with and without error detection and we find a significant improvement for $Q\in\{2,3,4\}$.
A global analysis of metrics used for measuring performance in natural language processing

Measuring the performance of natural language processing models is challenging. Traditionally used metrics, such as BLEU and ROUGE, originally devised for machine translation and summarization, have been shown to suffer from low correlation with human judgment and a lack of transferability to other tasks and languages. In the past 15 years, a wide range of alternative metrics have been proposed. However, it is unclear to what extent this has had an impact on NLP benchmarking efforts. Here we provide the first large-scale cross-sectional analysis of metrics used for measuring performance in natural language processing. We curated, mapped and systematized more than 3500 machine learning model performance results from the open repository 'Papers with Code' to enable a global and comprehensive analysis. Our results suggest that the large majority of natural language processing metrics currently used have properties that may result in an inadequate reflection of a models' performance. Furthermore, we found that ambiguities and inconsistencies in the reporting of metrics may lead to difficulties in interpreting and comparing model performances, impairing transparency and reproducibility in NLP research.
DIMIX: DIminishing MIXing for Sloppy Agents

We study non-convex distributed optimization problems where a set of agents collaboratively solve a separable optimization problem that is distributed over a time-varying network. The existing methods to solve these problems rely on (at most) one time-scale algorithms, where each agent performs a diminishing or constant step-size gradient descent at the average estimate of the agents in the network. However, if possible at all, exchanging exact information, that is required to evaluate these average estimates, potentially introduces a massive communication overhead. Therefore, a reasonable practical assumption to be made is that agents only receive a rough approximation of the neighboring agents' information. To address this, we introduce and study a \textit{two time-scale} decentralized algorithm with a broad class of \textit{lossy} information sharing methods (that includes noisy, quantized, and/or compressed information sharing) over \textit{time-varying} networks. In our method, one time-scale suppresses the (imperfect) incoming information from the neighboring agents, and one time-scale operates on local cost functions' gradients. We show that with a proper choices for the step-sizes' parameters, the algorithm achieves a convergence rate of $\mathcal{O}({T}^{-1/3 + \epsilon})$ for non-convex distributed optimization problems over time-varying networks, for any $\epsilon>0$. Our simulation results support the theoretical results of the paper.