OpenRaiser/Intern-Atlas huggingface dataset:
Bounded Fuzzy Possibilistic Method of Critical Objects Processing in Machine Learning
Venue: arXiv cs.LG 2020
Year: 2020

Unsatisfying accuracy of learning methods is mostly caused by omitting the influence of important parameters such as membership assignments, type of data objects, and distance or similarity functions. The proposed method, called Bounded Fuzzy Possibilistic Method (BFPM) addresses different issues that previous clustering or classification methods have not sufficiently considered in their membership assignments. In fuzzy methods, the object's memberships should sum to 1. Hence, any data object may obtain full membership in at most one cluster or class. Possibilistic methods relax this condition, but the method can be satisfied with the results even if just an arbitrary object obtains the membership from just one cluster, which prevents the objects' movement analysis. Whereas, BFPM differs from previous fuzzy and possibilistic approaches by removing these restrictions. Furthermore, BFPM provides the flexible search space for objects' movement analysis. Data objects are also considered as fundamental keys in learning methods, and knowing the exact type of objects results in providing a suitable environment for learning algorithms. The Thesis introduces a new type of object, called critical, as well as categorizing data objects into two different categories: structural-based and behavioural-based. Critical objects are considered as causes of miss-classification and miss-assignment in learning procedures. The Thesis also proposes new methodologies to study the behaviour of critical objects with the aim of evaluating objects' movements (mutation) from one cluster or class to another. The Thesis also introduces a new type of feature, called dominant, that is considered as one of the causes of miss-classification and miss-assignments. Then the Thesis proposes new sets of similarity functions, called Weighted Feature Distance (WFD) and Prioritized Weighted Feature Distance (PWFD).
Towards Purely Unsupervised Disentanglement of Appearance and Shape for Person Images Generation
Venue: arXiv cs.CV 2020
Year: 2020

There have been a fairly of research interests in exploring the disentanglement of appearance and shape from human images. Most existing endeavours pursuit this goal by either using training images with annotations or regulating the training process with external clues such as human skeleton, body segmentation or cloth patches etc. In this paper, we aim to address this challenge in a more unsupervised manner---we do not require any annotation nor any external task-specific clues. To this end, we formulate an encoder-decoder-like network to extract both the shape and appearance features from input images at the same time, and train the parameters by three losses: feature adversarial loss, color consistency loss and reconstruction loss. The feature adversarial loss mainly impose little to none mutual information between the extracted shape and appearance features, while the color consistency loss is to encourage the invariance of person appearance conditioned on different shapes. More importantly, our unsupervised (Unsupervised learning has many interpretations in different tasks. To be clear, in this paper, we refer unsupervised learning as learning without task-specific human annotations, pairs or any form of weak supervision.) framework utilizes learned shape features as masks which are applied to the input itself in order to obtain clean appearance features. Without using fixed input human skeleton, our network better preserves the conditional human posture while requiring less supervision. Experimental results on DeepFashion and Market1501 demonstrate that the proposed method achieves clean disentanglement and is able to synthesis novel images of comparable quality with state-of-the-art weakly-supervised or even supervised methods.
Contrastive Visual-Linguistic Pretraining
Venue: arXiv cs.CV 2020
Year: 2020

Several multi-modality representation learning approaches such as LXMERT and ViLBERT have been proposed recently. Such approaches can achieve superior performance due to the high-level semantic information captured during large-scale multimodal pretraining. However, as ViLBERT and LXMERT adopt visual region regression and classification loss, they often suffer from domain gap and noisy label problems, based on the visual features having been pretrained on the Visual Genome dataset. To overcome these issues, we propose unbiased Contrastive Visual-Linguistic Pretraining (CVLP), which constructs a visual self-supervised loss built upon contrastive learning. We evaluate CVLP on several down-stream tasks, including VQA, GQA and NLVR2 to validate the superiority of contrastive learning on multi-modality representation learning. Our code is available at: https://github.com/ArcherYunDong/CVLP-.
Fast-Convergent Federated Learning
Venue: arXiv cs.LG 2020
Year: 2020

Federated learning has emerged recently as a promising solution for distributing machine learning tasks through modern networks of mobile devices. Recent studies have obtained lower bounds on the expected decrease in model loss that is achieved through each round of federated learning. However, convergence generally requires a large number of communication rounds, which induces delay in model training and is costly in terms of network resources. In this paper, we propose a fast-convergent federated learning algorithm, called FOLB, which performs intelligent sampling of devices in each round of model training to optimize the expected convergence speed. We first theoretically characterize a lower bound on improvement that can be obtained in each round if devices are selected according to the expected improvement their local models will provide to the current global model. Then, we show that FOLB obtains this bound through uniform sampling by weighting device updates according to their gradient information. FOLB is able to handle both communication and computation heterogeneity of devices by adapting the aggregations according to estimates of device's capabilities of contributing to the updates. We evaluate FOLB in comparison with existing federated learning algorithms and experimentally show its improvement in trained model accuracy, convergence speed, and/or model stability across various machine learning tasks and datasets.
Deep Photometric Stereo for Non-Lambertian Surfaces
Venue: arXiv cs.CV 2020
Year: 2020

This paper addresses the problem of photometric stereo, in both calibrated and uncalibrated scenarios, for non-Lambertian surfaces based on deep learning. We first introduce a fully convolutional deep network for calibrated photometric stereo, which we call PS-FCN. Unlike traditional approaches that adopt simplified reflectance models to make the problem tractable, our method directly learns the mapping from reflectance observations to surface normal, and is able to handle surfaces with general and unknown isotropic reflectance. At test time, PS-FCN takes an arbitrary number of images and their associated light directions as input and predicts a surface normal map of the scene in a fast feed-forward pass. To deal with the uncalibrated scenario where light directions are unknown, we introduce a new convolutional network, named LCNet, to estimate light directions from input images. The estimated light directions and the input images are then fed to PS-FCN to determine the surface normals. Our method does not require a pre-defined set of light directions and can handle multiple images in an order-agnostic manner. Thorough evaluation of our approach on both synthetic and real datasets shows that it outperforms state-of-the-art methods in both calibrated and uncalibrated scenarios.
KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media
Venue: arXiv cs.CL 2020
Year: 2020

In this paper, we describe our approach to utilize pre-trained BERT models with Convolutional Neural Networks for sub-task A of the Multilingual Offensive Language Identification shared task (OffensEval 2020), which is a part of the SemEval 2020. We show that combining CNN with BERT is better than using BERT on its own, and we emphasize the importance of utilizing pre-trained language models for downstream tasks. Our system, ranked 4th with macro averaged F1-Score of 0.897 in Arabic, 4th with score of 0.843 in Greek, and 3rd with score of 0.814 in Turkish. Additionally, we present ArabicBERT, a set of pre-trained transformer language models for Arabic that we share with the community.
Neural-Symbolic Reasoning over Knowledge Graph for Multi-stage Explainable Recommendation
Venue: arXiv cs.IR 2020
Year: 2020

Recent work on recommender systems has considered external knowledge graphs as valuable sources of information, not only to produce better recommendations but also to provide explanations of why the recommended items were chosen. Pure rule-based symbolic methods provide a transparent reasoning process over knowledge graph but lack generalization ability to unseen examples, while deep learning models enhance powerful feature representation ability but are hard to interpret. Moreover, direct reasoning over large-scale knowledge graph can be costly due to the huge search space of pathfinding. We approach the problem through a novel coarse-to-fine neural symbolic reasoning method called NSER. It first generates a coarse-grained explanation to capture abstract user behavioral pattern, followed by a fined-grained explanation accompanying with explicit reasoning paths and recommendations inferred from knowledge graph. We extensively experiment on four real-world datasets and observe substantial gains of recommendation performance compared with state-of-the-art methods as well as more diversified explanations in different granularity.
Energy Efficiency Through Joint Routing and Function Placement in Different Modes of SDN/NFV Networks
Venue: arXiv cs.NI 2020
Year: 2020

Network function virtualization (NFV) and software defined networking (SDN) are two promising technologies to enable 5G and 6G services and achieve cost reduction, network scalability, and deployment flexibility. However, migration to full SDN/NFV networks in order to serve these services is a time consuming process and costly for mobile operators. This paper focuses on energy efficiency during the transition of mobile core networks (MCN) to full SDN/NFV networks, and explores how energy efficiency can be addressed during such migration. We propose a general system model containing a combination of legacy nodes and links, in addition to newly introduced NFV and SDN nodes. We refer to this system model as partial SDN and hybrid NFV MCN which can cover different modes of SDN and NFV implementations. Based on this framework, we formulate energy efficiency by considering joint routing and function placement in the network. Since this problem belongs to the class of non-linear integer programming problems, to solve it efficiently, we present a modified Viterbi algorithm (MVA) based on multi-stage graph modeling and a modified Dijkstra's algorithm. We simulate this algorithm for a number of network scenarios with different fractions of NFV and SDN nodes, and evaluate how much energy can be saved through such transition. Simulation results confirm the expected performance of the algorithm which saves up to 70% energy compared to network where all nodes are always on. Interestingly, the amount of energy saved by the proposed algorithm in the case of hybrid NFV and partial SDN networks can reach up to 60-90% of the saved energy in full NFV/SDN networks.
Dual Distribution Alignment Network for Generalizable Person Re-Identification
Venue: arXiv cs.IR 2020
Year: 2020

Domain generalization (DG) serves as a promising solution to handle person Re-Identification (Re-ID), which trains the model using labels from the source domain alone, and then directly adopts the trained model to the target domain without model updating. However, existing DG approaches are usually disturbed by serious domain variations due to significant dataset variations. Subsequently, DG highly relies on designing domain-invariant features, which is however not well exploited, since most existing approaches directly mix multiple datasets to train DG based models without considering the local dataset similarities, i.e., examples that are very similar but from different domains. In this paper, we present a Dual Distribution Alignment Network (DDAN), which handles this challenge by mapping images into a domain-invariant feature space by selectively aligning distributions of multiple source domains. Such an alignment is conducted by dual-level constraints, i.e., the domain-wise adversarial feature learning and the identity-wise similarity enhancement. We evaluate our DDAN on a large-scale Domain Generalization Re-ID (DG Re-ID) benchmark. Quantitative results demonstrate that the proposed DDAN can well align the distributions of various source domains, and significantly outperforms all existing domain generalization approaches.
Can We `Feel' the Temperature of Knowledge? Modelling Scientific Popularity Dynamics via Thermodynamics
Venue: arXiv cs.SI 2020
Year: 2020

Just like everything in the nature, scientific topics flourish and perish. While existing literature well captures article's life-cycle via citation patterns, little is known about how scientific popularity and impact evolves for a specific topic. It would be most intuitive if we could `feel' topic's activity just as we perceive the weather by temperature. Here, we conceive knowledge temperature to quantify topic overall popularity and impact through citation network dynamics. Knowledge temperature includes 2 parts. One part depicts lasting impact by assessing knowledge accumulation with an analogy between topic evolution and isobaric expansion. The other part gauges temporal changes in knowledge structure, an embodiment of short-term popularity, through the rate of entropy change with internal energy, 2 thermodynamic variables approximated via node degree and edge number. Our analysis of representative topics with size ranging from 1000 to over 30000 articles reveals that the key to flourishing is topics' ability in accumulating useful information for future knowledge generation. Topics particularly experience temperature surges when their knowledge structure is altered by influential articles. The spike is especially obvious when there appears a single non-trivial novel research focus or merging in topic structure. Overall, knowledge temperature manifests topics' distinct evolutionary cycles.
Outage Probability and Capacity Scaling Law of Multiple RIS-Aided Cooperative Networks
Venue: arXiv cs.IT 2020
Year: 2020

In this letter, we consider a dual-hop cooperative network assisted by multiple reconfigurable intelligent surfaces (RISs). Assuming that the RIS with the highest instantaneous end-to-end signal-to-noise ratio (SNR) is selected to aid the communication, the outage probability (OP) and average sum-rate are investigated. Specifically, an exact analysis for the OP is developed. In addition, relying on the extreme value theory, closed-form expressions for the asymptotic OP and asymptotic sum-rate are derived, based on which the capacity scaling law is established. Our results are corroborated through simulations and insightful discussions are provided. In particular, our analysis shows that the number of RISs as well as the number of reflecting elements play a crucial role in the capacity scaling law of multiple RIS-aided cooperative networks. Also, comparisons with relay-aided systems are carried out to demonstrate that the proposed system setup outperforms relaying schemes both in terms of the OP and average sum-rate.
Hermitian dual-containing constacyclic BCH codes and related quantum codes of length $\frac{q^{2m}-1}{q+1}$
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we study a family of constacyclic BCH codes over $\mathbb{F}_{q^2}$ of length $n=\frac{q^{2m}-1}{q+1}$, where $q$ is a prime power, and $m\geq2$ an even integer. The maximum design distance of narrow-sense Hermitian dual-containing constacyclic BCH codes of length $n$ is determined. Furthermore, the exact dimension of the constacyclic BCH codes with given design distance is computed. As a consequence, we are able to derive the parameters of quantum codes as a function of their design parameters of the associated constacyclic BCH codes. This improves the result by Yuan et al. (Des Codes Cryptogr 85(1): 179-190, 2017), showing that with the same lengths, except for three trivial cases ($q=2,3,4$), our resultant quantum codes can always yield strict dimension or minimum distance gains than the ones obtained by Yuan et al.. Moreover, fixing length $n=\frac{q^{2m}-1}{q+1}$, some constructed quantum codes have better parameters or are beneficial complements compared with some known results (Aly et al., IEEE Trans Inf Theory 53(3): 1183-1188, 2007, Li et al., Quantum Inf Process 18(5): 127, 2019, Wang et al., Quantum Inf Process 18(8): 323, 2019, Song et al., Quantum Inf Process 17(10): 1-24, 2018.).
NAYEL at SemEval-2020 Task 12: TF/IDF-Based Approach for Automatic Offensive Language Detection in Arabic Tweets
Venue: arXiv cs.CL 2020
Year: 2020

In this paper, we present the system submitted to "SemEval-2020 Task 12". The proposed system aims at automatically identify the Offensive Language in Arabic Tweets. A machine learning based approach has been used to design our system. We implemented a linear classifier with Stochastic Gradient Descent (SGD) as optimization algorithm. Our model reported 84.20%, 81.82% f1-score on development set and test set respectively. The best performed system and the system in the last rank reported 90.17% and 44.51% f1-score on test set respectively.
Algorithm Configurations of MOEA/D with an Unbounded External Archive
Venue: arXiv cs.NE 2020
Year: 2020

In the evolutionary multi-objective optimization (EMO) community, it is usually assumed that the final population is presented to the decision maker as the result of the execution of an EMO algorithm. Recently, an unbounded external archive was used to evaluate the performance of EMO algorithms in some studies where a pre-specified number of solutions are selected from all the examined non-dominated solutions. In this framework, which is referred to as the solution selection framework, the final population does not have to be a good solution set. Thus, the solution selection framework offers higher flexibility to the design of EMO algorithms than the final population framework. In this paper, we examine the design of MOEA/D under these two frameworks. First, we show that the performance of MOEA/D is improved by linearly changing the reference point specification during its execution through computational experiments with various combinations of initial and final specifications. Robust and high performance of the solution selection framework is observed. Then, we examine the use of a genetic algorithm-based offline hyper-heuristic method to find the best configuration of MOEA/D in each framework. Finally, we further discuss solution selection after the execution of an EMO algorithm in the solution selection framework.
Contraction Mapping of Feature Norms for Classifier Learning on the Data with Different Quality
Venue: arXiv cs.CV 2020
Year: 2020

The popular softmax loss and its recent extensions have achieved great success in the deep learning-based image classification. However, the data for training image classifiers usually has different quality. Ignoring such problem, the correct classification of low quality data is hard to be solved. In this paper, we discover the positive correlation between the feature norm of an image and its quality through careful experiments on various applications and various deep neural networks. Based on this finding, we propose a contraction mapping function to compress the range of feature norms of training images according to their quality and embed this contraction mapping function into softmax loss or its extensions to produce novel learning objectives. The experiments on various classification applications, including handwritten digit recognition, lung nodule classification, face verification and face recognition, demonstrate that the proposed approach is promising to effectively deal with the problem of learning on the data with different quality and leads to the significant and stable improvements in the classification accuracy.
Testing And Hardening IoT Devices Against the Mirai Botnet
Venue: arXiv cs.CY 2020
Year: 2020

A large majority of cheap Internet of Things (IoT) devices that arrive brand new, and are configured with out-of-the-box settings, are not being properly secured by the manufactures, and are vulnerable to existing malware lurking on the Internet. Among them is the Mirai botnet which has had its source code leaked to the world, allowing any malicious actor to configure and unleash it. A combination of software assets not being utilised safely and effectively are exposing consumers to a full compromise. We configured and attacked 4 different IoT devices using the Mirai libraries. Our experiments concluded that three out of the four devices were vulnerable to the Mirai malware and became infected when deployed using their default configuration. This demonstrates that the original security configurations are not sufficient to provide acceptable levels of protection for consumers, leaving their devices exposed and vulnerable. By analysing the Mirai libraries and its attack vectors, we were able to determine appropriate device configuration countermeasures to harden the devices against this botnet, which were successfully validated through experimentation.
Binary Search and First Order Gradient Based Method for Stochastic Optimization
Venue: arXiv cs.LG 2020
Year: 2020

In this paper, we present a novel stochastic optimization method, which uses the binary search technique with first order gradient based optimization method, called Binary Search Gradient Optimization (BSG) or BiGrad. In this optimization setup, a non-convex surface is treated as a set of convex surfaces. In BSG, at first, a region is defined, assuming region is convex. If region is not convex, then the algorithm leaves the region very fast and defines a new one, otherwise, it tries to converge at the optimal point of the region. In BSG, core purpose of binary search is to decide, whether region is convex or not in logarithmic time, whereas, first order gradient based method is primarily applied, to define a new region. In this paper, Adam is used as a first order gradient based method, nevertheless, other methods of this class may also be considered. In deep neural network setup, it handles the problem of vanishing and exploding gradient efficiently. We evaluate BSG on the MNIST handwritten digit, IMDB, and CIFAR10 data set, using logistic regression and deep neural networks. We produce more promising results as compared to other first order gradient based optimization methods. Furthermore, proposed algorithm generalizes significantly better on unseen data as compared to other methods.
Internal Quasiperiod Queries
Venue: arXiv cs.DS 2020
Year: 2020

Internal pattern matching requires one to answer queries about factors of a given string. Many results are known on answering internal period queries, asking for the periods of a given factor. In this paper we investigate (for the first time) internal queries asking for covers (also known as quasiperiods) of a given factor. We propose a data structure that answers such queries in $O(\log n \log \log n)$ time for the shortest cover and in $O(\log n (\log \log n)^2)$ time for a representation of all the covers, after $O(n \log n)$ time and space preprocessing.