Reviewerly/exHarmony huggingface dataset:
Analysis of Sparse Subspace Clustering: Experiments and Random Projection

Clustering can be defined as the process of assembling objects into a number of groups whose elements are similar to each other in some manner. As a technique that is used in many domains, such as face clustering, plant categorization, image segmentation, document classification, clustering is considered one of the most important unsupervised learning problems. Scientists have surveyed this problem for years and developed different techniques that can solve it, such as k-means clustering. We analyze one of these techniques: a powerful clustering algorithm called Sparse Subspace Clustering. We demonstrate several experiments using this method and then introduce a new approach that can reduce the computational time required to perform sparse subspace clustering.
COOL, a Context Outlooker, and its Application to Question Answering and other Natural Language Processing Tasks

Vision outlooker improves the performance of vision transformers, which implements a self-attention mechanism by adding an outlook attention, a form of local attention. In natural language processing, as has been the case in computer vision and other domains, transformer-based models constitute the state-of-the-art for most processing tasks. In this domain, too, many authors have argued and demonstrated the importance of local context. We present an outlook attention mechanism, COOL, for natural language processing. COOL, added on top of the self-attention layers of a transformer-based model, encodes local syntactic context considering word proximity and more pair-wise constraints than dynamic convolution used by existing approaches. A comparative empirical performance evaluation of an implementation of COOL with different transformer-based models confirms the opportunity for improvement over a baseline using the original models alone for various natural language processing tasks, including question answering. The proposed approach achieves competitive performance with existing state-of-the-art methods on some tasks.
Transducer-based language embedding for spoken language identification

The acoustic and linguistic features are important cues for the spoken language identification (LID) task. Recent advanced LID systems mainly use acoustic features that lack the usage of explicit linguistic feature encoding. In this paper, we propose a novel transducer-based language embedding approach for LID tasks by integrating an RNN transducer model into a language embedding framework. Benefiting from the advantages of the RNN transducer's linguistic representation capability, the proposed method can exploit both phonetically-aware acoustic features and explicit linguistic features for LID tasks. Experiments were carried out on the large-scale multilingual LibriSpeech and VoxLingua107 datasets. Experimental results showed the proposed method significantly improves the performance on LID tasks with 12% to 59% and 16% to 24% relative improvement on in-domain and cross-domain datasets, respectively.
Heterogeneous Autoencoder Empowered by Quadratic Neurons

Inspired by the complexity and diversity of biological neurons, a quadratic neuron is proposed to replace the inner product in the current neuron with a simplified quadratic function. Employing such a novel type of neurons offers a new perspective on developing deep learning. When analyzing quadratic neurons, we find that there exists a function such that a heterogeneous network can approximate it well with a polynomial number of neurons but a purely conventional or quadratic network needs an exponential number of neurons to achieve the same level of error. Encouraged by this inspiring theoretical result on heterogeneous networks, we directly integrate conventional and quadratic neurons in an autoencoder to make a new type of heterogeneous autoencoders. Anomaly detection experiments confirm that heterogeneous autoencoders perform competitively compared to other state-of-the-art models.
Data-augmented cross-lingual synthesis in a teacher-student framework

Cross-lingual synthesis can be defined as the task of letting a speaker generate fluent synthetic speech in another language. This is a challenging task, and resulting speech can suffer from reduced naturalness, accented speech, and/or loss of essential voice characteristics. Previous research shows that many models appear to have insufficient generalization capabilities to perform well on every of these cross-lingual aspects. To overcome these generalization problems, we propose to apply the teacher-student paradigm to cross-lingual synthesis. While a teacher model is commonly used to produce teacher forced data, we propose to also use it to produce augmented data of unseen speaker-language pairs, where the aim is to retain essential speaker characteristics. Both sets of data are then used for student model training, which is trained to retain the naturalness and prosodic variation present in the teacher forced data, while learning the speaker identity from the augmented data. Some modifications to the student model are proposed to make the separation of teacher forced and augmented data more straightforward. Results show that the proposed approach improves the retention of speaker characteristics in the speech, while managing to retain high levels of naturalness and prosodic variation.
Super Resolution for Turbulent Flows in 2D: Stabilized Physics Informed Neural Networks

We propose a new design of a neural network for solving a zero shot super resolution problem for turbulent flows. We embed Luenberger-type observer into the network's architecture to inform the network of the physics of the process, and to provide error correction and stabilization mechanisms. In addition, to compensate for decrease of observer's performance due to the presence of unknown destabilizing forcing, the network is designed to estimate the contribution of the unknown forcing implicitly from the data over the course of training. By running a set of numerical experiments, we demonstrate that the proposed network does recover unknown forcing from data and is capable of predicting turbulent flows in high resolution from low resolution noisy observations.
Second-order gauge-invariant formalism for the cosmological observables: Complete verification of their gauge-invariance

Accounting for all the relativistic effects, we have developed the fully nonlinear gauge-invariant formalism for describing the cosmological observables and presented the second-order perturbative expressions associated with light propagation and observations without choosing a gauge condition. For the first time, we have performed a complete verification of the validity of our second-order expressions by comparing their gauge-transformation properties from two independent methods: one directly obtained from their expressions in terms of metric perturbations and the other expected from their nonlinear relations. The expressions for the cosmological observables such as galaxy clustering and the luminosity distance are invariant under diffeomorphism and gauge-invariant at the observed position. We compare our results to the previous work and discuss the differences in the perturbative expressions. Our second-order gauge-invariant formalism constitutes a major step forward in the era of precision cosmology and its applications in the future will play a crucial role for going beyond the power spectrum and probing the early universe.
Synthetic Graph Generation to Benchmark Graph Learning

Graph learning algorithms have attained state-of-the-art performance on many graph analysis tasks such as node classification, link prediction, and clustering. It has, however, become hard to track the field's burgeoning progress. One reason is due to the very small number of datasets used in practice to benchmark the performance of graph learning algorithms. This shockingly small sample size (~10) allows for only limited scientific insight into the problem. In this work, we aim to address this deficiency. We propose to generate synthetic graphs, and study the behaviour of graph learning algorithms in a controlled scenario. We develop a fully-featured synthetic graph generator that allows deep inspection of different models. We argue that synthetic graph generations allows for thorough investigation of algorithms and provides more insights than overfitting on three citation datasets. In the case study, we show how our framework provides insight into unsupervised and supervised graph neural network models.
C-MORE: Pretraining to Answer Open-Domain Questions by Consulting Millions of References

We consider the problem of pretraining a twostage open-domain question answering (QA) system (retriever + reader) with strong transfer capabilities.The key challenge is how to construct a large amount of high-quality question-answer-context triplets without taskspecific annotations.Specifically, the triplets should align well with downstream tasks by: (i) covering a wide range of domains (for opendomain applications), (ii) linking a question to its semantically relevant context with supporting evidence (for training the retriever), and (iii) identifying the correct answer in the context (for training the reader).Previous pretraining approaches generally fall short of one or more of these requirements.In this work, we automatically construct a large-scale corpus that meets all three criteria by consulting millions of references cited within Wikipedia.The well-aligned pretraining signals benefit both the retriever and the reader significantly.Our pretrained retriever leads to 2%-10% absolute gains in top-20 accuracy.And with our pretrained reader, the entire system improves by up to 4% in exact match.
Unsupervised Learning of Accurate Siamese Tracking

Unsupervised learning has been popular in various computer vision tasks, including visual object tracking. However, prior unsupervised tracking approaches rely heavily on spatial supervision from template-search pairs and are still unable to track objects with strong variation over a long time span. As unlimited self-supervision signals can be obtained by tracking a video along a cycle in time, we investigate evolving a Siamese tracker by tracking videos forward-backward. We present a novel unsupervised tracking framework, in which we can learn temporal correspondence both on the classification branch and regression branch. Specifically, to propagate reliable template feature in the forward propagation process so that the tracker can be trained in the cycle, we first propose a consistency propagation transformation. We then identify an ill-posed penalty problem in conventional cycle training in backward propagation process. Thus, a differentiable region mask is proposed to select features as well as to implicitly penalize tracking errors on intermediate frames. Moreover, since noisy labels may degrade training, we propose a mask-guided loss reweighting strategy to assign dynamic weights based on the quality of pseudo labels. In extensive experiments, our tracker outperforms preceding unsupervised methods by a substantial margin, performing on par with supervised methods on large-scale datasets such as TrackingNet and LaSOT. Code is available at https://github.com/FlorinShum/ULAST.
Towards Generalizeable Semantic Product Search by Text Similarity Pre-training on Search Click Logs

Recently, semantic search has been successfully applied to e-commerce product search and the learned semantic space(s) for query and product encoding are expected to generalize to unseen queries or products.Yet, whether generalization can conveniently emerge has not been thoroughly studied in the domain thus far.In this paper, we examine several general-domain and domain-specific pre-trained Roberta variants and discover that general-domain finetuning does not help generalization, which aligns with the discovery of prior art.Proper domain-specific fine-tuning with clickstream data can lead to better model generalization, based on a bucketed analysis of a publicly available manual annotated query-product pair data.
HiMODE: A Hybrid Monocular Omnidirectional Depth Estimation Model

Monocular omnidirectional depth estimation is receiving considerable research attention due to its broad applications for sensing 360{\deg} surroundings. Existing approaches in this field suffer from limitations in recovering small object details and data lost during the ground-truth depth map acquisition. In this paper, a novel monocular omnidirectional depth estimation model, namely HiMODE is proposed based on a hybrid CNN+Transformer (encoder-decoder) architecture whose modules are efficiently designed to mitigate distortion and computational cost, without performance degradation. Firstly, we design a feature pyramid network based on the HNet block to extract high-resolution features near the edges. The performance is further improved, benefiting from a self and cross attention layer and spatial/temporal patches in the Transformer encoder and decoder, respectively. Besides, a spatial residual block is employed to reduce the number of parameters. By jointly passing the deep features extracted from an input image at each backbone block, along with the raw depth maps predicted by the transformer encoder-decoder, through a context adjustment layer, our model can produce resulting depth maps with better visual quality than the ground-truth. Comprehensive ablation studies demonstrate the significance of each individual module. Extensive experiments conducted on three datasets; Stanford3D, Matterport3D, and SunCG, demonstrate that HiMODE can achieve state-of-the-art performance for 360{\deg} monocular depth estimation.
R(Det)^2: Randomized Decision Routing for Object Detection

In the paradigm of object detection, the decision head is an important part, which affects detection performance significantly. Yet how to design a high-performance decision head remains to be an open issue. In this paper, we propose a novel approach to combine decision trees and deep neural networks in an end-to-end learning manner for object detection. First, we disentangle the decision choices and prediction values by plugging soft decision trees into neural networks. To facilitate effective learning, we propose randomized decision routing with node selective and associative losses, which can boost the feature representative learning and network decision simultaneously. Second, we develop the decision head for object detection with narrow branches to generate the routing probabilities and masks, for the purpose of obtaining divergent decisions from different nodes. We name this approach as the randomized decision routing for object detection, abbreviated as R(Det)$^2$. Experiments on MS-COCO dataset demonstrate that R(Det)$^2$ is effective to improve the detection performance. Equipped with existing detectors, it achieves $1.4\sim 3.6$\% AP improvement.
Partial Relaxed Optimal Transport for Denoised Recommendation

The interaction data used by recommender systems (RSs) inevitably include noises resulting from mistaken or exploratory clicks, especially under implicit feedbacks. Without proper denoising, RS models cannot effectively capture users' intrinsic preferences and the true interactions between users and items. To address such noises, existing methods mostly rely on auxiliary data which are not always available. In this work, we ground on Optimal Transport (OT) to globally match a user embedding space and an item embedding space, allowing both non-deep and deep RS models to discriminate intrinsic and noisy interactions without supervision. Specifically, we firstly leverage the OT framework via Sinkhorn distance to compute the continuous many-to-many user-item matching scores. Then, we relax the regularization in Sinkhorn distance to achieve a closed-form solution with a reduced time complexity. Finally, to consider individual user behaviors for denoising, we develop a partial OT framework to adaptively relabel user-item interactions through a personalized thresholding mechanism. Extensive experiments show that our framework can significantly boost the performances of existing RS models.
Improved Lower Bounds on the Domination Number of Hypercubes and Binary Codes with Covering Radius One

A dominating set on an $n $-dimensional hypercube is equivalent to a binary covering code of length $n $ and covering radius 1. It is still an open problem to determine the domination number $\gamma(Q_n)$ for $ n\geq10$ and $ n\ne2^{k},2^{k}-1 $ ($k\in\mathbb{N} $). When $n$ is a multiple of 6, the best known lower bound is $\gamma(Q_n)\geq \frac{2^n}{n}$, given by Van Wee (1988). In this article, we present a new method using congruence properties due to Laurent Habsieger (1997) and obtain an improved lower bound $\gamma(Q_n)\geq \frac{(n-2)2^n }{n^2-2n-2}$ when $n$ is a multiple of 6.
Scientific Paper Recommendation Systems: a Literature Review of recent Publications

Scientific writing builds upon already published papers. Manual identification of publications to read, cite or consider as related papers relies on a researcher's ability to identify fitting keywords or initial papers from which a literature search can be started. The rapidly increasing amount of papers has called for automatic measures to find the desired relevant publications, so-called paper recommendation systems. As the number of publications increases so does the amount of paper recommendation systems. Former literature reviews focused on discussing the general landscape of approaches throughout the years and highlight the main directions. We refrain from this perspective, instead we only consider a comparatively small time frame but analyse it fully. In this literature review we discuss used methods, datasets, evaluations and open challenges encountered in all works first released between January 2019 and October 2021. The goal of this survey is to provide a comprehensive and complete overview of current paper recommendation systems.
Automation of developing adaptive e-learning using blockchain technology

The low percentage of completion of online courses in the framework of the modern mass online education may be related to the lack of adaptation for a specific user of an online course of a learning path unifying all the stu-dents. The solution to the problem can be adaptive learning, which modifies the learning process for different students depending on their characteristics and learning conditions. The purpose of the study is to automate the process of developing adaptive e-learning by using the blockchain technology. The results of a study, in which adaptive learning was organized on the basis of the Felder-Silverman learning styles model was used as input data for the experiment. Applying the mechanism of smart contracts, the blockchain technology registered learning events, which made it possible to determine the learning style of users directly during the learning process. The chosen model of learning styles, as well as the number of categories of students, were configured exclusively in the smart contract code, which made it possible to talk about the universality and scalability of the process of organizing adaptive e-learning using the blockchain. The technology also ensured the reliability of data storage and traceability. It documents heterogeneous data in a single storage space adding a timemarker and the author's address to each recorded event. The proposed method also unifies the organization of adaptive e-learning, since it is suitable for different models of learning styles with different parameters. The automated process does not require the development of preliminary tests and is free from biased assessments from the students in the learning course
Research of information interaction between different subsystems based on CCSDS standards in object identification

Due to the increasing complexity of information processes, there is a need to develop models and methods of interaction between various subsystems, segments and, in relation to them, to develop methods for identifying ob-jects. The object of research is the process of accounting for the movement of material objects between different segments of complex systems, for example, between different segments of the international space station, between different segments of the material flow system or other segments of the Earth-Space system. Instruments and instrument complexes were chosen as material objects. Based on the performed study of data structure complexity levels, the necessity of development of identification systems for different segments of the material flow transportation system was substantiated. A new method is proposed and the results of practical experiments on the identification of material objects on the basis of CCSDS standard (CCSDS RFID Tag-Encoding Yellow Book) are considered. The hardware platform for accounting of tangible objects and information collection was selected radio-frequency readers and on-board computer networks represented by CCSDS 881.1-B-1 (Spacecraft onboard interface services - RFID tag encoding specification). RFID tags are used to identify tangible objects. To solve the problem of transferring information about material objects the developed model of information exchange and method of identification of material objects are presented, necessary fields and tag's memory bank structure are presented. In the process of describing the identification of material objects it is proposed to use a special alphabet ECMA-113 for data representation in the identifiers placed in the RFID tags. As part of the study, the simulation of information exchange between different subsystems during the transfer of information about material objects was conducted. The results of the experiments are presented, confirming that the universality of the alphabet lies in the possibility of simultaneous use of Latin and Cyrillic coding. For an estimation of efficiency of the offered models and modelling of a situation of an exchange of the information on material objects in wireless on-board networks separate independent subprogrammes of coding, decoding of identifiers of material objects have been developed and practical modelling of transfer of the big information volume between two various agencies (segments) is spent. The programmes were developed by the participants independently &#x0D; of each other. On the basis of fulfilled modeling of information interaction and information transfer on material objects it was found that the error of data recognition does not exceed 6-7% and it does not increase with the increasing number of objects. The development results allowed to formulate a conclusion about the success of the experiment on the application of object identifiers based on CCSDS for airborne wireless networks, for the use of radio-frequency identification systems.