OpenRaiser/Intern-Atlas huggingface dataset:
Text AutoAugment: Learning Compositional Augmentation Policy for Text Classification
Venue: arXiv cs.CL 2021
Year: 2021

Data augmentation aims to enrich training samples for alleviating the overfitting issue in low-resource or class-imbalanced situations. Traditional methods first devise task-specific operations such as Synonym Substitute, then preset the corresponding parameters such as the substitution rate artificially, which require a lot of prior knowledge and are prone to fall into the sub-optimum. Besides, the number of editing operations is limited in the previous methods, which decreases the diversity of the augmented data and thus restricts the performance gain. To overcome the above limitations, we propose a framework named Text AutoAugment (TAA) to establish a compositional and learnable paradigm for data augmentation. We regard a combination of various operations as an augmentation policy and utilize an efficient Bayesian Optimization algorithm to automatically search for the best policy, which substantially improves the generalization capability of models. Experiments on six benchmark datasets show that TAA boosts classification accuracy in low-resource and class-imbalanced regimes by an average of 8.8% and 9.7%, respectively, outperforming strong baselines.
A Gradient Sampling Algorithm for Stratified Maps with Applications to Topological Data Analysis
Venue: arXiv cs.CG 2021
Year: 2021

We introduce a novel gradient descent algorithm extending the well-known Gradient Sampling methodology to the class of stratifiably smooth objective functions, which are defined as locally Lipschitz functions that are smooth on some regular pieces-called the strata-of the ambient Euclidean space. For this class of functions, our algorithm achieves a sub-linear convergence rate. We then apply our method to objective functions based on the (extended) persistent homology map computed over lower-star filters, which is a central tool of Topological Data Analysis. For this, we propose an efficient exploration of the corresponding stratification by using the Cayley graph of the permutation group. Finally, we provide benchmark and novel topological optimization problems, in order to demonstrate the utility and applicability of our framework.
Latin writing styles analysis with Machine Learning: New approach to old questions
Venue: arXiv cs.CL 2021
Year: 2021

In the Middle Ages texts were learned by heart and spread using oral means of communication from generation to generation. Adaptation of the art of prose and poems allowed keeping particular descriptions and compositions characteristic for many literary genres. Taking into account such a specific construction of literature composed in Latin, we can search for and indicate the probability patterns of familiar sources of specific narrative texts. Consideration of Natural Language Processing tools allowed us the transformation of textual objects into numerical ones and then application of machine learning algorithms to extract information from the dataset. We carried out the task consisting of the practical use of those concepts and observation to create a tool for analyzing narrative texts basing on open-source databases. The tool focused on creating specific search tools resources which could enable us detailed searching throughout the text. The main objectives of the study take into account finding similarities between sentences and between documents. Next, we applied machine learning algorithms on chosen texts to calculate specific features of them (for instance authorship or centuries) and to recognize sources of anonymous texts with a certain percentage.
Grassmannian Constellation Design for Noncoherent MIMO Systems Using Autoencoders
Venue: arXiv cs.IT 2021
Year: 2021

In this letter, we propose an autoencoder (AE) for designing Grassmannian constellations in noncoherent (NC) multiple-input multiple-output (MIMO) systems. To guarantee the properties of Grassmannian constellations, the proposed AE constructs the transmitted symbols following an unitary space-time modulation. It penalizes the difference between input and output symbols in terms of cross entropy during the training, which is regarded as a generic optimization method. The constellations learned by the proposed AE have substantial symbol error rate (SER) performance gains compared to the non-Grassmannian constellations and conventionally constructed Grassmannian constellations in high SNR regime. The resulting Grassmannian constellation of the proposed AE achieves higher diversity than the non-Grassmannian constellation in i.i.d. Rayleigh channels. Moreover, the proposed approach can be adaptive to different channel statistics by training with corresponding channel realizations.
Developing Products Update-Alert System for e-Commerce Websites Users Using HTML Data and Web Scraping Technique
Venue: arXiv cs.IR 2021
Year: 2021

Websites are regarded as domains of limitless information which anyone and everyone can access. The new trend of technology put us to change the way we are doing our business. The Internet now is fastly becoming a new place for business and the advancement in this technology gave rise to the number of e-commerce websites. This made the lifestyle of marketers/vendors, retailers and consumers (collectively regarded as users in this paper) easy, because it provides easy platforms to sale/order items through the internet. This also requires that the users will have to spend a lot of time and effort to search for the best product deals, products updates and offers on e-commerce websites. They have to filter and compare search results by themselves which takes a lot of time and there are chances of ambiguous results. In this paper, we applied web crawling and scraping methods on an e-commerce website to get HTML data for identifying products updates based on the current time. The HTML data is preprocessed to extract details of the products such as name, price, post date and time, etc. to serve as useful information for users.
Clock Skew Compensation Algorithm Immune to Floating-Point Precision Loss
Venue: arXiv cs.NI 2021
Year: 2021

We propose a novel clock skew compensation algorithm based on Bresenham's line drawing algorithm. The proposed algorithm can avoid the effect of limited floating-point precision (e.g., 32-bit single precision) on clock skew compensation and thereby provide high-precision time synchronization even with resource-constrained sensor nodes in wireless sensor networks.
Supporting CUDA for an extended RISC-V GPU architecture
Venue: arXiv cs.PL 2021
Year: 2021

With the rapid development of scientific computation, more and more researchers and developers are committed to implementing various workloads/operations on different devices. Among all these devices, NVIDIA GPU is the most popular choice due to its comprehensive documentation and excellent development tools. As a result, there are abundant resources for hand-writing high-performance CUDA codes. However, CUDA is mainly supported by only commercial products and there has been no support for open-source H/W platforms. RISC-V is the most popular choice for hardware ISA, thanks to its elegant design and open-source license. In this project, we aim to utilize these existing CUDA codes with RISC-V devices. More specifically, we design and implement a pipeline that can execute CUDA source code on an RISC-V GPU architecture. We have succeeded in executing CUDA kernels with several important features, like multi-thread and atomic instructions, on an RISC-V GPU architecture.
Regional Adversarial Training for Better Robust Generalization
Venue: arXiv cs.CV 2021
Year: 2021

Adversarial training (AT) has been demonstrated as one of the most promising defense methods against various adversarial attacks. To our knowledge, existing AT-based methods usually train with the locally most adversarial perturbed points and treat all the perturbed points equally, which may lead to considerably weaker adversarial robust generalization on test data. In this work, we introduce a new adversarial training framework that considers the diversity as well as characteristics of the perturbed points in the vicinity of benign samples. To realize the framework, we propose a Regional Adversarial Training (RAT) defense method that first utilizes the attack path generated by the typical iterative attack method of projected gradient descent (PGD), and constructs an adversarial region based on the attack path. Then, RAT samples diverse perturbed training points efficiently inside this region, and utilizes a distance-aware label smoothing mechanism to capture our intuition that perturbed points at different locations should have different impact on the model performance. Extensive experiments on several benchmark datasets show that RAT consistently makes significant improvement on standard adversarial training (SAT), and exhibits better robust generalization.
Binaural Audio Generation via Multi-task Learning
Venue: arXiv cs.SD 2021
Year: 2021

We present a learning-based approach for generating binaural audio from mono audio using multi-task learning. Our formulation leverages additional information from two related tasks: the binaural audio generation task and the flipped audio classification task. Our learning model extracts spatialization features from the visual and audio input, predicts the left and right audio channels, and judges whether the left and right channels are flipped. First, we extract visual features using ResNet from the video frames. Next, we perform binaural audio generation and flipped audio classification using separate subnetworks based on visual features. Our learning method optimizes the overall loss based on the weighted sum of the losses of the two tasks. We train and evaluate our model on the FAIR-Play dataset and the YouTube-ASMR dataset. We perform quantitative and qualitative evaluations to demonstrate the benefits of our approach over prior techniques.
Co-Separable Nonnegative Matrix Factorization
Venue: arXiv cs.LG 2021
Year: 2021

Nonnegative matrix factorization (NMF) is a popular model in the field of pattern recognition. It aims to find a low rank approximation for nonnegative data M by a product of two nonnegative matrices W and H. In general, NMF is NP-hard to solve while it can be solved efficiently under separability assumption, which requires the columns of factor matrix are equal to columns of the input matrix. In this paper, we generalize separability assumption based on 3-factor NMF M=P_1SP_2, and require that S is a sub-matrix of the input matrix. We refer to this NMF as a Co-Separable NMF (CoS-NMF). We discuss some mathematics properties of CoS-NMF, and present the relationships with other related matrix factorizations such as CUR decomposition, generalized separable NMF(GS-NMF), and bi-orthogonal tri-factorization (BiOR-NM3F). An optimization model for CoS-NMF is proposed and alternated fast gradient method is employed to solve the model. Numerical experiments on synthetic datasets, document datasets and facial databases are conducted to verify the effectiveness of our CoS-NMF model. Compared to state-of-the-art methods, CoS-NMF model performs very well in co-clustering task, and preserves a good approximation to the input data matrix as well.
Neural News Recommendation with Collaborative News Encoding and Structural User Encoding
Venue: arXiv cs.CL 2021
Year: 2021

Automatic news recommendation has gained much attention from the academic community and industry. Recent studies reveal that the key to this task lies within the effective representation learning of both news and users. Existing works typically encode news title and content separately while neglecting their semantic interaction, which is inadequate for news text comprehension. Besides, previous models encode user browsing history without leveraging the structural correlation of user browsed news to reflect user interests explicitly. In this work, we propose a news recommendation framework consisting of collaborative news encoding (CNE) and structural user encoding (SUE) to enhance news and user representation learning. CNE equipped with bidirectional LSTMs encodes news title and content collaboratively with cross-selection and cross-attention modules to learn semantic-interactive news representations. SUE utilizes graph convolutional networks to extract cluster-structural features of user history, followed by intra-cluster and inter-cluster attention modules to learn hierarchical user interest representations. Experiment results on the MIND dataset validate the effectiveness of our model to improve the performance of news recommendation. Our code is released at https://github.com/Veason-silverbullet/NNR.
Better Self-training for Image Classification through Self-supervision
Venue: arXiv cs.CV 2021
Year: 2021

Self-training is a simple semi-supervised learning approach: Unlabelled examples that attract high-confidence predictions are labelled with their predictions and added to the training set, with this process being repeated multiple times. Recently, self-supervision -- learning without manual supervision by solving an automatically-generated pretext task -- has gained prominence in deep learning. This paper investigates three different ways of incorporating self-supervision into self-training to improve accuracy in image classification: self-supervision as pretraining only, self-supervision performed exclusively in the first iteration of self-training, and self-supervision added to every iteration of self-training. Empirical results on the SVHN, CIFAR-10, and PlantVillage datasets, using both training from scratch, and Imagenet-pretrained weights, show that applying self-supervision only in the first iteration of self-training can greatly improve accuracy, for a modest increase in computation time.
Deep Learning-based mitosis detection in breast cancer histologic samples
Venue: arXiv cs.CV 2021
Year: 2021

This is the submission for mitosis detection in the context of the MIDOG 2021 challenge. It is based on the two-stage objection model Faster RCNN as well as DenseNet as a backbone for the neural network architecture. It achieves a F1-score of 0.6645 on the Preliminary Test Phase Leaderboard.
Towards a Reference Architecture for Future Industrial Internet of Things Networks
Venue: arXiv cs.NI 2021
Year: 2021

With the continuing decrease of sensor technology prices as well as the increase of communication and analytical capabilities of modern internet of things devices, the continuously generated amount of data is constantly growing. Various use cases show the untapped potential of this data for new business models. However, conventional industrial IT networks of traditional manufacturing companies can hardly meet the modern requirements emerging with today's and future industrial internet of things applications. Outdated and rigid network infrastructures are one of the main reasons for hesitant innovation efforts and cross-organizational collaborations as well as the slow adoption of modern business models by traditional manufacturing companies. Following the design science research paradigm, our work contributes by elaborating on a comprehensive list of requirements for future industrial internet of things networks from a theoretical and practical perspective as well as a proposed reference architecture acting as a blueprint for future implementations.
A Survey on Open Set Recognition
Venue: arXiv cs.CV 2021
Year: 2021

Open Set Recognition (OSR) is about dealing with unknown situations that were not learned by the models during training. In this paper, we provide a survey of existing works about OSR and distinguish their respective advantages and disadvantages to help out new researchers interested in the subject. The categorization of OSR models is provided along with an extensive summary of recent progress. Additionally, the relationships between OSR and its related tasks including multi-class classification and novelty detection are analyzed. It is concluded that OSR can appropriately deal with unknown instances in the real-world where capturing all possible classes in the training data is not practical. Lastly, applications of OSR are highlighted and some new directions for future research topics are suggested.
CE-Dedup: Cost-Effective Convolutional Neural Nets Training based on Image Deduplication
Venue: arXiv cs.CV 2021
Year: 2021

Attributed to the ever-increasing large image datasets, Convolutional Neural Networks (CNNs) have become popular for vision-based tasks. It is generally admirable to have larger-sized datasets for higher network training accuracies. However, the impact of dataset quality has not to be involved. It is reasonable to assume the near-duplicate images exist in the datasets. For instance, the Street View House Numbers (SVHN) dataset having cropped house plate digits from 0 to 9 are likely to have repetitive digits from the same/similar house plates. Redundant images may take up a certain portion of the dataset without consciousness. While contributing little to no accuracy improvement for the CNNs training, these duplicated images unnecessarily pose extra resource and computation consumption. To this end, this paper proposes a framework to assess the impact of the near-duplicate images on CNN training performance, called CE-Dedup. Specifically, CE-Dedup associates a hashing-based image deduplication approach with downstream CNNs-based image classification tasks. CE-Dedup balances the tradeoff between a large deduplication ratio and a stable accuracy by adjusting the deduplication threshold. The effectiveness of CE-Dedup is validated through extensive experiments on well-known CNN benchmarks. On one hand, while maintaining the same validation accuracy, CE-Dedup can reduce the dataset size by 23%. On the other hand, when allowing a small validation accuracy drop (by 5%), CE-Dedup can trim the dataset size by 75%.
An Automated Approach for the Recognition of Bengali License Plates
Venue: arXiv cs.CV 2021
Year: 2021

Automatic Number Plate Recognition (ALPR) is a system for automatically identifying the license plates of any vehicle. This process is important for tracking, ticketing, and any billing system, among other things. With the use of information and communication technology (ICT), all systems are being automated, including the vehicle tracking system. This study proposes a hybrid method for detecting license plates using characters from them. Our captured image information was used for the recognition procedure in Bangladeshi vehicles, which is the topic of this study. Here, for license plate detection, the YOLO model was used where 81% was correctly predicted. And then, for license plate segmentation, Otsu's Thresholding was used and eventually, for character recognition, the CNN model was applied. This model will allow the vehicle's automated license plate detection system to avoid any misuse.
Binary self-dual codes of various lengths with new weight enumerators from a modified bordered construction and neighbours
Venue: arXiv cs.IT 2021
Year: 2021

In this work, we define a modification of a bordered construction for self-dual codes which utilises $λ$-circulant matrices. We provide the necessary conditions for the construction to produce self-dual codes over finite commutative Frobenius rings of characteristic 2. Using the modified construction together with the neighbour construction, we construct many binary self-dual codes of lengths 54, 68, 82 and 94 with weight enumerators that have previously not been known to exist.
BiHPF: Bilateral High-Pass Filters for Robust Deepfake Detection
Venue: arXiv cs.CV 2021
Year: 2021

The advancement in numerous generative models has a two-fold effect: a simple and easy generation of realistic synthesized images, but also an increased risk of malicious abuse of those images. Thus, it is important to develop a generalized detector for synthesized images of any GAN model or object category, including those unseen during the training phase. However, the conventional methods heavily depend on the training settings, which cause a dramatic decline in performance when tested with unknown domains. To resolve the issue and obtain a generalized detection ability, we propose Bilateral High-Pass Filters (BiHPF), which amplify the effect of the frequency-level artifacts that are known to be found in the synthesized images of generative models. Numerous experimental results validate that our method outperforms other state-of-the-art methods, even when tested with unseen domains.
Multi-task learning from fixed-wing UAV images for 2D/3D city modeling
Venue: arXiv cs.CV 2021
Year: 2021

Single-task learning in artificial neural networks will be able to learn the model very well, and the benefits brought by transferring knowledge thus become limited. In this regard, when the number of tasks increases (e.g., semantic segmentation, panoptic segmentation, monocular depth estimation, and 3D point cloud), duplicate information may exist across tasks, and the improvement becomes less significant. Multi-task learning has emerged as a solution to knowledge-transfer issues and is an approach to scene understanding which involves multiple related tasks each with potentially limited training data. Multi-task learning improves generalization by leveraging the domain-specific information contained in the training data of related tasks. In urban management applications such as infrastructure development, traffic monitoring, smart 3D cities, and change detection, automated multi-task data analysis for scene understanding based on the semantic, instance, and panoptic annotation, as well as monocular depth estimation, is required to generate precise urban models. In this study, a common framework for the performance assessment of multi-task learning methods from fixed-wing UAV images for 2D/3D city modeling is presented.
Skim-Attention: Learning to Focus via Document Layout
Venue: arXiv cs.CL 2021
Year: 2021

Transformer-based pre-training techniques of text and layout have proven effective in a number of document understanding tasks. Despite this success, multimodal pre-training models suffer from very high computational and memory costs. Motivated by human reading strategies, this paper presents Skim-Attention, a new attention mechanism that takes advantage of the structure of the document and its layout. Skim-Attention only attends to the 2-dimensional position of the words in a document. Our experiments show that Skim-Attention obtains a lower perplexity than prior works, while being more computationally efficient. Skim-Attention can be further combined with long-range Transformers to efficiently process long documents. We also show how Skim-Attention can be used off-the-shelf as a mask for any Pre-trained Language Model, allowing to improve their performance while restricting attention. Finally, we show the emergence of a document structure representation in Skim-Attention.