OpenRaiser/Intern-Atlas huggingface dataset:
GL-GAN: Adaptive Global and Local Bilevel Optimization model of Image Generation
Venue: arXiv cs.CV 2020
Year: 2020

Although Generative Adversarial Networks have shown remarkable performance in image generation, there are some challenges in image realism and convergence speed. The results of some models display the imbalances of quality within a generated image, in which some defective parts appear compared with other regions. Different from general single global optimization methods, we introduce an adaptive global and local bilevel optimization model(GL-GAN). The model achieves the generation of high-resolution images in a complementary and promoting way, where global optimization is to optimize the whole images and local is only to optimize the low-quality areas. With a simple network structure, GL-GAN is allowed to effectively avoid the nature of imbalance by local bilevel optimization, which is accomplished by first locating low-quality areas and then optimizing them. Moreover, by using feature map cues from discriminator output, we propose the adaptive local and global optimization method(Ada-OP) for specific implementation and find that it boosts the convergence speed. Compared with the current GAN methods, our model has shown impressive performance on CelebA, CelebA-HQ and LSUN datasets.
Data-driven Meta-set Based Fine-Grained Visual Classification
Venue: arXiv cs.CV 2020
Year: 2020

Constructing fine-grained image datasets typically requires domain-specific expert knowledge, which is not always available for crowd-sourcing platform annotators. Accordingly, learning directly from web images becomes an alternative method for fine-grained visual recognition. However, label noise in the web training set can severely degrade the model performance. To this end, we propose a data-driven meta-set based approach to deal with noisy web images for fine-grained recognition. Specifically, guided by a small amount of clean meta-set, we train a selection net in a meta-learning manner to distinguish in- and out-of-distribution noisy images. To further boost the robustness of model, we also learn a labeling net to correct the labels of in-distribution noisy data. In this way, our proposed method can alleviate the harmful effects caused by out-of-distribution noise and properly exploit the in-distribution noisy samples for training. Extensive experiments on three commonly used fine-grained datasets demonstrate that our approach is much superior to state-of-the-art noise-robust methods.
Few-shot Classification via Adaptive Attention
Venue: arXiv cs.CV 2020
Year: 2020

Training a neural network model that can quickly adapt to a new task is highly desirable yet challenging for few-shot learning problems. Recent few-shot learning methods mostly concentrate on developing various meta-learning strategies from two aspects, namely optimizing an initial model or learning a distance metric. In this work, we propose a novel few-shot learning method via optimizing and fast adapting the query sample representation based on very few reference samples. To be specific, we devise a simple and efficient meta-reweighting strategy to adapt the sample representations and generate soft attention to refine the representation such that the relevant features from the query and support samples can be extracted for a better few-shot classification. Such an adaptive attention model is also able to explain what the classification model is looking for as the evidence for classification to some extent. As demonstrated experimentally, the proposed model achieves state-of-the-art classification results on various benchmark few-shot classification and fine-grained recognition datasets.
Design of Reconfigurable Multi-Operand Adder for Massively Parallel Processing
Venue: arXiv cs.AR 2020
Year: 2020

The paper presents a systematic study and implementation of a reconfigurable combinatorial multi-operand adder for use in Deep Learning systems. The size of carry changes with the number of operands and hence a reliable algorithm to estimate exact number of carry bits is needed for optimal implementation of a reconfigurable multi-operand adder. A combinatorial multi-operand adder can be faster compared to a sequential implementation using a two operand adder. Use cases for such adders occur in modern processors for deep neural networks. Such processors require massively parallel computing resources on chip. This paper presents a method to estimate the upper bound on the size of carry. A method to compute the exact number of carry bits required for a multi-operand addition operation. A fast combinatorial parallel 4-operand adder module is presented. An algorithm to reconfigure these adder modules to implement larger adders is also described. Further, the paper presents two compact but slower iterative structures that implement multi-operand addition, iterating with one column at a time till the entire word is covered. Such serial/iterative operations are slow but occupy small space while parallel operations are fast but use large silicon area on chip. Interestingly, the area-to-throughput ratio of two architectures can tilt in favor of slower, smaller and large number units instead of the fewer numbers of fast and large compute units. A lemma presented in the paper may be used to identify the condition when such tilt occurs. Potentially, this can save silicon space and increase the throughput of chips for high performance computing. Simulation results of a 16 operand adder and using an set of 4-operand adders for use in neural networks have been presented. Simulation results show that performance gain improves as the number of operations or operands increases.
Nephalai: Towards LPWAN C-RAN with Physical Layer Compression
Venue: arXiv cs.NI 2020
Year: 2020

We propose Nephalai, a Compressive Sensing-based Cloud Radio Access Network (C-RAN), to reduce the uplink bit rate of the physical layer (PHY) between the gateways and the cloud server for multi-channel LPWANs. Recent research shows that single-channel LPWANs suffer from scalability issues. While multiple channels improve these issues, data transmission is expensive. Furthermore, recent research has shown that jointly decoding raw physical layers that are offloaded by LPWAN gateways in the cloud can improve the signal-to-noise ratio (SNR) of week radio signals. However, when it comes to multiple channels, this approach requires high bandwidth of network infrastructure to transport a large amount of PHY samples from gateways to the cloud server, which results in network congestion and high cost due to Internet data usage. In order to reduce the operation's bandwidth, we propose a novel LPWAN packet acquisition mechanism based on Compressive Sensing with a custom design dictionary that exploits the structure of LPWAN packets, reduces the bit rate of samples on each gateway, and demodulates PHY in the cloud with (joint) sparse approximation. Moreover, we propose an adaptive compression method that takes the Spreading Factor (SF) and SNR into account. Our empirical evaluation shows that up to 93.7% PHY samples can be reduced by Nephalai when SF = 9 and SNR is high without degradation in the packet reception rate (PRR). With four gateways, 1.7x PRR can be achieved with 87.5% PHY samples compressed, which can extend the battery lifetime of embedded IoT devices to 1.7.
Approach for Document Detection by Contours and Contrasts
Venue: arXiv cs.CV 2020
Year: 2020

This paper considers arbitrary document detection performed on a mobile device. The classical contour-based approach often fails in cases featuring occlusion, complex background, or blur. The region-based approach, which relies on the contrast between object and background, does not have application limitations, however, its known implementations are highly resource-consuming. We propose a modification of the contour-based method, in which the competing contour location hypotheses are ranked according to the contrast between the areas inside and outside the border. In the experiments, such modification allows for the decrease of alternatives ordering errors by 40% and the decrease of the overall detection errors by 10%. The proposed method provides unmatched state-of-the-art performance on the open MIDV-500 dataset, and it demonstrates results comparable with state-of-the-art performance on the SmartDoc dataset.
Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets
Venue: arXiv cs.CL 2020
Year: 2020

Ideally Open-Domain Question Answering models should exhibit a number of competencies, ranging from simply memorizing questions seen at training time, to answering novel question formulations with answers seen during training, to generalizing to completely novel questions with novel answers. However, single aggregated test set scores do not show the full picture of what capabilities models truly have. In this work, we perform a detailed study of the test sets of three popular open-domain benchmark datasets with respect to these competencies. We find that 60-70% of test-time answers are also present somewhere in the training sets. We also find that 30% of test-set questions have a near-duplicate paraphrase in their corresponding training sets. Using these findings, we evaluate a variety of popular open-domain models to obtain greater insight into what extent they can actually generalize, and what drives their overall performance. We find that all models perform dramatically worse on questions that cannot be memorized from training sets, with a mean absolute performance difference of 63% between repeated and non-repeated data. Finally we show that simple nearest-neighbor models out-perform a BART closed-book QA model, further highlighting the role that training set memorization plays in these benchmarks
Deep Reinforcement Learning for Tactile Robotics: Learning to Type on a Braille Keyboard
Venue: arXiv cs.RO 2020
Year: 2020

Artificial touch would seem well-suited for Reinforcement Learning (RL), since both paradigms rely on interaction with an environment. Here we propose a new environment and set of tasks to encourage development of tactile reinforcement learning: learning to type on a braille keyboard. Four tasks are proposed, progressing in difficulty from arrow to alphabet keys and from discrete to continuous actions. A simulated counterpart is also constructed by sampling tactile data from the physical environment. Using state-of-the-art deep RL algorithms, we show that all of these tasks can be successfully learnt in simulation, and 3 out of 4 tasks can be learned on the real robot. A lack of sample efficiency currently makes the continuous alphabet task impractical on the robot. To the best of our knowledge, this work presents the first demonstration of successfully training deep RL agents in the real world using observations that exclusively consist of tactile images. To aid future research utilising this environment, the code for this project has been released along with designs of the braille keycaps for 3D printing and a guide for recreating the experiments. A brief video summary is also available at https://youtu.be/eNylCA2uE_E.
Machine Learning for Health: Personalized Models for Forecasting of Alzheimer Disease Progression
Venue: arXiv cs.LG 2020
Year: 2020

In this thesis the aim is to work on optimizing the modern machine learning models for personalized forecasting of Alzheimer Disease (AD) Progression from clinical trial data. The data comes from the TADPOLE challenge, which is one of the largest publicly available datasets for AD research (ADNI dataset). The goal of the project is to develop machine learning models that can be used to perform personalized forecasts of the participants cognitive changes (e.g., ADAS-Cog13 scores) over the time period of 6,12, 18 and 24 months in the future and the change in Clinical Status (CS) i.e., whether a person will convert to AD within 2 years or not. This is important for informing current clinical trials and better design of future clinical trials for AD. We will work with personalized Gaussian processes as machine learning models to predict ADAS-Cog13 score and Cox model along with a classifier to predict the conversion in a patient within 2 years.This project is done with the collaboration with researchers from the MIT MediaLab.
Clock Drift Impact on Target Wake Time in IEEE 802.11ax/ah Networks
Venue: arXiv cs.NI 2020
Year: 2020

In the Internet of Things scenarios, it is crucially important to provide low energy consumption of client devices. To address this challenge, new Wi-Fi standards introduce the Target Wake Time (TWT) mechanism. With TWT, devices transmit their data according to a schedule and move to the doze state afterwards. The main problem of this mechanism is the clock drift phenomenon, because of which the devices cease to strictly comply with the schedule. As a result, they can miss the scheduled transmission time, which increases active time and thus power consumption. The paper investigates uplink transmission with two different TWT operation modes. With the first mode, a sensor transmits a packet to the access point (AP) after waking up, using the random channel access. With the second mode, the AP polls stations and they can transmit a packet only after receiving a trigger frame from the AP. For both modes, the paper studies how the average transmission time, the packet loss rate and the average energy consumption depend on the different TWT parameters. It is shown that when configured to guarantee the given packet loss rate, the first mode provides lower transmission time, while the second mode provides lower energy consumption.
Pairwise Relation Learning for Semi-supervised Gland Segmentation
Venue: arXiv cs.CV 2020
Year: 2020

Accurate and automated gland segmentation on histology tissue images is an essential but challenging task in the computer-aided diagnosis of adenocarcinoma. Despite their prevalence, deep learning models always require a myriad number of densely annotated training images, which are difficult to obtain due to extensive labor and associated expert costs related to histology image annotations. In this paper, we propose the pairwise relation-based semi-supervised (PRS^2) model for gland segmentation on histology images. This model consists of a segmentation network (S-Net) and a pairwise relation network (PR-Net). The S-Net is trained on labeled data for segmentation, and PR-Net is trained on both labeled and unlabeled data in an unsupervised way to enhance its image representation ability via exploiting the semantic consistency between each pair of images in the feature space. Since both networks share their encoders, the image representation ability learned by PR-Net can be transferred to S-Net to improve its segmentation performance. We also design the object-level Dice loss to address the issues caused by touching glands and combine it with other two loss functions for S-Net. We evaluated our model against five recent methods on the GlaS dataset and three recent methods on the CRAG dataset. Our results not only demonstrate the effectiveness of the proposed PR-Net and object-level Dice loss, but also indicate that our PRS^2 model achieves the state-of-the-art gland segmentation performance on both benchmarks.
Recommending Influenceable Targets based on Influence Propagation through Activity Behaviors in Online Social Media
Venue: arXiv cs.SI 2020
Year: 2020

Online Social Media (OSM) is a platform through which the users present themselves to the connected world by means of messaging, posting, reacting, tagging, and sharing on different contents with also other social activities. Nowadays, it has a vast impact on various aspects of the industry, business and society along with on users life. In an OSN platform, reaching the target users is one of the primary focus for most of the businesses and other organizations. Identification and recommendation of influenceable targets help to capture the appropriate audience efficiently and effectively. In this paper, an effective model has been discussed in egocentric OSN by incorporating an efficient influence measured Recommendation System in order to generate a list of top most influenceable target users among all connected network members for any specific social network user. Firstly the list of interacted network members has been updated based on all activities. On which the interacted network members with most similar activities have been recommended based on the specific influence category with sentiment type. After that, the top most influenceable network members in the basis of the required amount among those updated list of interacted network members have been identified with proper ranking by analyzing the similarity and frequency of their activity contents with respect to the activity contents of the main user. Through these two continuous stages, an effective list of top influenceable targets of the main user has been distinguished from the egocentric view of any social network.
Discovering and Categorising Language Biases in Reddit
Venue: arXiv cs.CL 2020
Year: 2020

We present a data-driven approach using word embeddings to discover and categorise language biases on the discussion platform Reddit. As spaces for isolated user communities, platforms such as Reddit are increasingly connected to issues of racism, sexism and other forms of discrimination. Hence, there is a need to monitor the language of these groups. One of the most promising AI approaches to trace linguistic biases in large textual datasets involves word embeddings, which transform text into high-dimensional dense vectors and capture semantic relations between words. Yet, previous studies require predefined sets of potential biases to study, e.g., whether gender is more or less associated with particular types of jobs. This makes these approaches unfit to deal with smaller and community-centric datasets such as those on Reddit, which contain smaller vocabularies and slang, as well as biases that may be particular to that community. This paper proposes a data-driven approach to automatically discover language biases encoded in the vocabulary of online discourse communities on Reddit. In our approach, protected attributes are connected to evaluative words found in the data, which are then categorised through a semantic analysis system. We verify the effectiveness of our method by comparing the biases we discover in the Google News dataset with those found in previous literature. We then successfully discover gender bias, religion bias, and ethnic bias in different Reddit communities. We conclude by discussing potential application scenarios and limitations of this data-driven bias discovery method.
Singularly Optimal Randomized Leader Election
Venue: arXiv cs.DC 2020
Year: 2020

This paper concerns designing distributed algorithms that are singularly optimal, i.e., algorithms that are simultaneously time and message optimal, for the fundamental leader election problem in networks. Our main result is a randomized distributed leader election algorithm for asynchronous complete networks that is essentially (up to a polylogarithmic factor) singularly optimal. Our algorithm uses $O(n)$ messages with high probability and runs in $O(\log^2 n)$ time (with high probability) to elect a unique leader. The $O(n)$ message complexity should be contrasted with the $Ω(n \log n)$ lower bounds for the deterministic message complexity of leader election algorithms (regardless of time), proven by Korach, Moran, and Zaks (TCS, 1989) for asynchronous algorithms and by Afek and Gafni (SIAM J. Comput., 1991) for synchronous networks. Hence, our result also separates the message complexities of randomized and deterministic leader election. More importantly, our (randomized) time complexity of $O(\log^2 n)$ for obtaining the optimal $O(n)$ message complexity is significantly smaller than the long-standing $\tildeΘ(n)$ time complexity obtained by Afek and Gafni and by Singh (SIAM J. Comput., 1997) for message optimal (deterministic) election in asynchronous networks. In synchronous complete networks, Afek and Gafni showed an essentially singularly optimal deterministic algorithm with $O(\log n)$ time and $O(n \log n)$ messages. Ramanathan et al. (Distrib. Comput. 2007) used randomization to improve the message complexity, and showed a randomized algorithm with $O(n)$ messages and $O(\log n)$ time (with failure probability $O(1 / \log^{Ω(1)}n)$). Our second result is a tightly singularly optimal randomized algorithm, with $O(1)$ time and $O(n)$ messages, for this setting, whose time bound holds with certainty and message bound holds with high probability.
Bisimulation and bisimilarity for fuzzy description logics under the Gödel semantics
Venue: arXiv cs.LO 2020
Year: 2020

Description logics (DLs) are a suitable formalism for representing knowledge about domains in which objects are described not only by attributes but also by binary relations between objects. Fuzzy extensions of DLs can be used for such domains when data and knowledge about them are vague and imprecise. One of the possible ways to specify classes of objects in such domains is to use concepts in fuzzy DLs. As DLs are variants of modal logics, indiscernibility in DLs is characterized by bisimilarity. The bisimilarity relation of an interpretation is the largest auto-bisimulation of that interpretation. In DLs and their fuzzy extensions, such equivalence relations can be used for concept learning. In this paper, we define and study fuzzy bisimulation and bisimilarity for fuzzy DLs under the Gödel semantics, as well as crisp bisimulation and strong bisimilarity for such logics extended with involutive negation. The considered logics are fuzzy extensions of the DL $\mathcal{ALC}_{reg}$ (a variant of PDL) with additional features among inverse roles, nominals, (qualified or unqualified) number restrictions, the universal role, local reflexivity of a role and involutive negation. We formulate and prove results on invariance of concepts under fuzzy (resp. crisp) bisimulation, conditional invariance of fuzzy TBoxex/ABoxes under bisimilarity (resp. strong bisimilarity), and the Hennessy-Milner property of fuzzy (resp. crisp) bisimulation for fuzzy DLs without (resp. with) involutive negation under the Gödel semantics. Apart from these fundamental results, we also provide results on using fuzzy bisimulation to separate the expressive powers of fuzzy DLs, as well as results on using strong bisimilarity to minimize fuzzy interpretations.
Prolog for Verification, Analysis and Transformation Tools
Venue: arXiv cs.PL 2020
Year: 2020

This article examines the use of the Prolog language for writing verification, analysis and transformation tools. Guided by experience in teaching and the development of verification tools like ProB or specialisation tools like ECCE and LOGEN, the article presents an assessment of various aspects of Prolog and provides guidelines for using them. The article shows the usefulness of a few key Prolog features. In particular, it discusses how to deal with negation at the level of the object programs being verified or analysed.
Data Weighted Training Strategies for Grammatical Error Correction
Venue: arXiv cs.CL 2020
Year: 2020

Recent progress in the task of Grammatical Error Correction (GEC) has been driven by addressing data sparsity, both through new methods for generating large and noisy pretraining data and through the publication of small and higher-quality finetuning data in the BEA-2019 shared task. Building upon recent work in Neural Machine Translation (NMT), we make use of both kinds of data by deriving example-level scores on our large pretraining data based on a smaller, higher-quality dataset. In this work, we perform an empirical study to discover how to best incorporate delta-log-perplexity, a type of example scoring, into a training schedule for GEC. In doing so, we perform experiments that shed light on the function and applicability of delta-log-perplexity. Models trained on scored data achieve state-of-the-art results on common GEC test sets.