OpenRaiser/Intern-Atlas huggingface dataset:
Evolving Neural Architecture Using One Shot Model
Venue: arXiv cs.NE 2020
Year: 2020

Neural Architecture Search (NAS) is emerging as a new research direction which has the potential to replace the hand-crafted neural architectures designed for specific tasks. Previous evolution based architecture search requires high computational resources resulting in high search time. In this work, we propose a novel way of applying a simple genetic algorithm to the NAS problem called EvNAS (Evolving Neural Architecture using One Shot Model) which reduces the search time significantly while still achieving better result than previous evolution based methods. The architectures are represented by using the architecture parameter of the one shot model which results in the weight sharing among the architectures for a given population of architectures and also weight inheritance from one generation to the next generation of architectures. We propose a decoding technique for the architecture parameter which is used to divert majority of the gradient information towards the given architecture and is also used for improving the performance prediction of the given architecture from the one shot model during the search process. Furthermore, we use the accuracy of the partially trained architecture on the validation data as a prediction of its fitness in order to reduce the search time. EvNAS searches for the architecture on the proxy dataset i.e. CIFAR-10 for 4.4 GPU day on a single GPU and achieves top-1 test error of 2.47% with 3.63M parameters which is then transferred to CIFAR-100 and ImageNet achieving top-1 error of 16.37% and top-5 error of 7.4% respectively. All of these results show the potential of evolutionary methods in solving the architecture search problem.
Avoiding Monochromatic Rectangles Using Shift Patterns
Venue: arXiv cs.DM 2020
Year: 2020

Ramsey Theory deals with avoiding certain patterns. When constructing an instance that avoids one pattern, it is observed that other patterns emerge. For example, repetition emerges when avoiding arithmetic progression (Van der Waerden numbers), while reflection emerges when avoiding monochromatic solutions of $a+b=c$ (Schur numbers). We exploit observed patterns when coloring a grid while avoiding monochromatic rectangles. Like many problems in Ramsey Theory, this problem has a rapidly growing search space that makes computer search difficult. Steinbach et al. obtained a solution of an 18 by 18 grid with 4 colors by enforcing a rotation symmetry. However, that symmetry is not suitable for 5 colors. In this article, we will encode this problem into propositional logic and enforce so-called internal symmetries, which preserves satisfiability, to guide SAT-solving. We first observe patterns with 2 and 3 colors, among which the "shift pattern" can be easily generalized and efficiently encoded. Using this pattern, we obtain a new solution of the 18 by 18 grid that is non-isomorphic to the known solution. We further analyze the pattern and obtain necessary conditions to further trim down the search space. We conclude with our attempts on finding a 5-coloring of a 26 by 26 grid, as well as further open problems on the shift pattern.
Distributed Adaptive Control: An ideal Cognitive Architecture candidate for managing a robotic recycling plant
Venue: arXiv cs.MA 2020
Year: 2020

In the past decade, society has experienced notable growth in a variety of technological areas. However, the Fourth Industrial Revolution has not been embraced yet. Industry 4.0 imposes several challenges which include the necessity of new architectural models to tackle the uncertainty that open environments represent to cyber-physical systems (CPS). Waste Electrical and Electronic Equipment (WEEE) recycling plants stand for one of such open environments. Here, CPSs must work harmoniously in a changing environment, interacting with similar and not so similar CPSs, and adaptively collaborating with human workers. In this paper, we support the Distributed Adaptive Control (DAC) theory as a suitable Cognitive Architecture for managing a recycling plant. Specifically, a recursive implementation of DAC (between both single-agent and large-scale levels) is proposed to meet the expected demands of the European Project HR-Recycler. Additionally, with the aim of having a realistic benchmark for future implementations of the recursive DAC, a micro-recycling plant prototype is presented.
ConvMath: A Convolutional Sequence Network for Mathematical Expression Recognition
Venue: arXiv cs.CV 2020
Year: 2020

Despite the recent advances in optical character recognition (OCR), mathematical expressions still face a great challenge to recognize due to their two-dimensional graphical layout. In this paper, we propose a convolutional sequence modeling network, ConvMath, which converts the mathematical expression description in an image into a LaTeX sequence in an end-to-end way. The network combines an image encoder for feature extraction and a convolutional decoder for sequence generation. Compared with other Long Short Term Memory(LSTM) based encoder-decoder models, ConvMath is entirely based on convolution, thus it is easy to perform parallel computation. Besides, the network adopts multi-layer attention mechanism in the decoder, which allows the model to align output symbols with source feature vectors automatically, and alleviates the problem of lacking coverage while training the model. The performance of ConvMath is evaluated on an open dataset named IM2LATEX-100K, including 103556 samples. The experimental results demonstrate that the proposed network achieves state-of-the-art accuracy and much better efficiency than previous methods.
Library of efficient algorithms for phylogenetic analysis
Venue: arXiv cs.DS 2020
Year: 2020

Evolutionary relationships between species are usually inferred through phylogenetic analysis, which provides phylogenetic trees computed from allelic profiles built by sequencing specific regions of the sequences and abstracting them to categorical indexes. With growing exchanges of people and merchandise, epidemics have become increasingly important, and combining information of country-specific datasets can now reveal unknown spreading patterns. The phylogenetic analysis workflow is mainly composed of four consecutive steps, the distance calculation, distance correction, inference algorithm, and local optimization steps. There are many phylogenetic tools out there, however most implement only some of these steps and serve only one single purpose, such as one type of algorithm. Another problem with these is that they are often hard to use and integrate, since each of them has its own API. This project resulted a library that implements the four steps of the workflow, and is highly performant, extensible, reusable, and portable, while providing common APIs and documentation. It also differs from other tools in the sense that, it is able to stop and resume the workflow whenever the user desires, and it was built to be continuously extended and not just serve a single purpose. The time benchmarks conducted on this library show that its implementations of the algorithms conform to their theoretical time complexity. Meanwhile, the memory benchmarks showcase that the implementations of the NJ algorithms follow a linear memory complexity, while the implementations of the MST and GCP algorithms follow a quadratic memory complexity.
A Note on the Significance Adjustment for FA*IR with Two Protected Groups
Venue: arXiv cs.IR 2020
Year: 2020

In this report we provide an improvement of the significance adjustment from the FA*IR algorithm of Zehlike et al., which did not work for very short rankings in combination with a low minimum proportion $p$ for the protected group. We show how the minimum number of protected candidates per ranking position can be calculated exactly and provide a mapping from the continuous space of significance levels ($α$) to a discrete space of tables, which allows us to find $α_c$ using a binary search heuristic.
Coin Flipping in Dynamic Programming is Almost Useless
Venue: arXiv cs.CC 2020
Year: 2020

We consider probabilistic circuits working over the real numbers, and using arbitrary semialgebraic functions of bounded description complexity as gates. In particular, such circuits can use all arithmetic operations +, -, x, /, optimization operations min and max, conditional branching (if-then-else), and many more. We show that probabilistic circuits using any of these operations as gates can be simulated by deterministic circuits with only about a quadratical blowup in size. A not much larger blow up in circuit size is also shown when derandomizing approximating circuits. The algorithmic consequence, motivating the title, is that randomness cannot substantially speed up dynamic programming algorithms.
ANR: Articulated Neural Rendering for Virtual Avatars
Venue: arXiv cs.CV 2020
Year: 2020

The combination of traditional rendering with neural networks in Deferred Neural Rendering (DNR) provides a compelling balance between computational complexity and realism of the resulting images. Using skinned meshes for rendering articulating objects is a natural extension for the DNR framework and would open it up to a plethora of applications. However, in this case the neural shading step must account for deformations that are possibly not captured in the mesh, as well as alignment inaccuracies and dynamics -- which can confound the DNR pipeline. We present Articulated Neural Rendering (ANR), a novel framework based on DNR which explicitly addresses its limitations for virtual human avatars. We show the superiority of ANR not only with respect to DNR but also with methods specialized for avatar creation and animation. In two user studies, we observe a clear preference for our avatar model and we demonstrate state-of-the-art performance on quantitative evaluation metrics. Perceptually, we observe better temporal stability, level of detail and plausibility.
Privacy Preservation for Wireless Sensor Networks in Healthcare: State of the Art, and Open Research Challenges
Venue: arXiv cs.CR 2020
Year: 2020

The advent of miniature biosensors has generated numerous opportunities for deploying wireless sensor networks in healthcare. However, an important barrier is that acceptance by healthcare stakeholders is influenced by the effectiveness of privacy safeguards for personal and intimate information which is collected and transmitted over the air, within and beyond these networks. In particular, these networks are progressing beyond traditional sensors, towards also using multimedia sensors, which raise further privacy concerns. Paradoxically, less research has addressed privacy protection, compared to security. Nevertheless, privacy protection has gradually evolved from being assumed an implicit by-product of security measures, and it is maturing into a research concern in its own right. However, further technical and socio-technical advances are needed. As a contribution towards galvanising further research, the hallmarks of this paper include: (i) a literature survey explicitly anchored on privacy preservation, it is underpinned by untangling privacy goals from security goals, to avoid mixing privacy and security concerns, as is often the case in other papers; (ii) a critical survey of privacy preservation services for wireless sensor networks in healthcare, including threat analysis and assessment methodologies; it also offers classification trees for the multifaceted challenge of privacy protection in healthcare, and for privacy threats, attacks and countermeasures; (iii) a discussion of technical advances complemented by reflection over the implications of regulatory frameworks; (iv) a discussion of open research challenges, leading onto offers of directions for future research towards unlocking the door onto privacy protection which is appropriate for healthcare in the twenty-first century.
Convolutional Neural Network for Elderly Wandering Prediction in Indoor Scenarios
Venue: arXiv cs.CV 2020
Year: 2020

This work proposes a way to detect the wandering activity of Alzheimer's patients from path data collected from non-intrusive indoor sensors around the house. Due to the lack of adequate data, we've manually generated a dataset of 220 paths using our own developed application. Wandering patterns in the literature are normally identified by visual features (such as loops or random movement), thus our dataset was transformed into images and augmented. Convolutional layers were used on the neural network model since they tend to have good results finding patterns, especially on images. The Convolutional Neural Network model was trained with the generated data and achieved an f1 score (relation between precision and recall) of 75%, recall of 60%, and precision of 100% on our 10 sample validation slice
General Domain Adaptation Through Proportional Progressive Pseudo Labeling
Venue: arXiv cs.LG 2020
Year: 2020

Domain adaptation helps transfer the knowledge gained from a labeled source domain to an unlabeled target domain. During the past few years, different domain adaptation techniques have been published. One common flaw of these approaches is that while they might work well on one input type, such as images, their performance drops when applied to others, such as text or time-series. In this paper, we introduce Proportional Progressive Pseudo Labeling (PPPL), a simple, yet effective technique that can be implemented in a few lines of code to build a more general domain adaptation technique that can be applied on several different input types. At the beginning of the training phase, PPPL progressively reduces target domain classification error, by training the model directly with pseudo-labeled target domain samples, while excluding samples with more likely wrong pseudo-labels from the training set and also postponing training on such samples. Experiments on 6 different datasets that include tasks such as anomaly detection, text sentiment analysis and image classification demonstrate that PPPL can beat other baselines and generalize better.
Disentangling semantics in language through VAEs and a certain architectural choice
Venue: arXiv cs.CL 2020
Year: 2020

We present an unsupervised method to obtain disentangled representations of sentences that single out semantic content. Using modified Transformers as building blocks, we train a Variational Autoencoder to translate the sentence to a fixed number of hierarchically structured latent variables. We study the influence of each latent variable in generation on the dependency structure of sentences, and on the predicate structure it yields when passed through an Open Information Extraction model. Our model could separate verbs, subjects, direct objects, and prepositional objects into latent variables we identified. We show that varying the corresponding latent variables results in varying these elements in sentences, and that swapping them between couples of sentences leads to the expected partial semantic swap.
A Generalized A* Algorithm for Finding Globally Optimal Paths in Weighted Colored Graphs
Venue: arXiv cs.RO 2020
Year: 2020

Both geometric and semantic information of the search space is imperative for a good plan. We encode those properties in a weighted colored graph (geometric information in terms of edge weight and semantic information in terms of edge and vertex color), and propose a generalized A* to find the shortest path among the set of paths with minimal inclusion of low-ranked color edges. We prove the completeness and optimality of this Class-Ordered A* (COA*) algorithm with respect to the hereto defined notion of optimality. The utility of COA* is numerically validated in a ternary graph with feasible, infeasible, and unknown vertices and edges for the cases of a 2D mobile robot, a 3D robotic arm, and a 5D robotic arm with limited sensing capabilities. We compare the results of COA* to that of the regular A* algorithm, the latter of which finds the shortest path regardless of uncertainty, and we show that the COA* dominates the A* solution in terms of finding less uncertain paths.
Quantifying the Privacy-Utility Trade-offs in COVID-19 Contact Tracing Apps
Venue: arXiv cs.CY 2020
Year: 2020

How to contain the spread of the COVID-19 virus is a major concern for most countries. As the situation continues to change, various countries are making efforts to reopen their economies by lifting some restrictions and enforcing new measures to prevent the spread. In this work, we review some approaches that have been adopted to contain the COVID-19 virus such as contact tracing, clusters identification, movement restrictions, and status validation. Specifically, we classify available techniques based on some characteristics such as technology, architecture, trade-offs (privacy vs utility), and the phase of adoption. We present a novel approach for evaluating privacy using both qualitative and quantitative measures of privacy-utility assessment of contact tracing applications. In this new method, we classify utility at three (3) distinct levels: no privacy, 100% privacy, and at k where k is set by the system providing the utility or privacy.
A Frequency-Based Learning-To-Rank Approach for Personal Digital Traces
Venue: arXiv cs.IR 2020
Year: 2020

Personal digital traces are constantly produced by connected devices, internet services and interactions. These digital traces are typically small, heterogeneous and stored in various locations in the cloud or on local devices, making it a challenge for users to interact with and search their own data. By adopting a multidimensional data model based on the six natural questions -- what, when, where, who, why and how -- to represent and unify heterogeneous personal digital traces, we can propose a learning-to-rank approach using the state of the art LambdaMART algorithm and frequency-based features that leverage the correlation between content (what), users (who), time (when), location (where) and data source (how) to improve the accuracy of search results. Due to the lack of publicly available personal training data, a combination of known-item query generation techniques and an unsupervised ranking model (field-based BM25) is used to build our own training sets. Experiments performed over a publicly available email collection and a personal digital data trace collection from a real user show that the frequency-based learning approach improves search accuracy when compared with traditional search tools.
SubICap: Towards Subword-informed Image Captioning
Venue: arXiv cs.CL 2020
Year: 2020

Existing Image Captioning (IC) systems model words as atomic units in captions and are unable to exploit the structural information in the words. This makes representation of rare words very difficult and out-of-vocabulary words impossible. Moreover, to avoid computational complexity, existing IC models operate over a modest sized vocabulary of frequent words, such that the identity of rare words is lost. In this work we address this common limitation of IC systems in dealing with rare words in the corpora. We decompose words into smaller constituent units 'subwords' and represent captions as a sequence of subwords instead of words. This helps represent all words in the corpora using a significantly lower subword vocabulary, leading to better parameter learning. Using subword language modeling, our captioning system improves various metric scores, with a training vocabulary size approximately 90% less than the baseline and various state-of-the-art word-level models. Our quantitative and qualitative results and analysis signify the efficacy of our proposed approach.
WEmbSim: A Simple yet Effective Metric for Image Captioning
Venue: arXiv cs.CV 2020
Year: 2020

The area of automatic image caption evaluation is still undergoing intensive research to address the needs of generating captions which can meet adequacy and fluency requirements. Based on our past attempts at developing highly sophisticated learning-based metrics, we have discovered that a simple cosine similarity measure using the Mean of Word Embeddings(MOWE) of captions can actually achieve a surprisingly high performance on unsupervised caption evaluation. This inspires our proposed work on an effective metric WEmbSim, which beats complex measures such as SPICE, CIDEr and WMD at system-level correlation with human judgments. Moreover, it also achieves the best accuracy at matching human consensus scores for caption pairs, against commonly used unsupervised methods. Therefore, we believe that WEmbSim sets a new baseline for any complex metric to be justified.
Q-SR: An Extensible Optimization Framework for Segment Routing
Venue: arXiv cs.NI 2020
Year: 2020

Segment routing (SR) combines the advantages of source routing supported by centralized software-defined networking (SDN) paradigm and hop-by-hop routing applied in distributed IP network infrastructure. However, because of the computation inefficiency, it is nearly impossible to evaluate whether various types of networks will benefit from the SR with multiple segments using conventional approaches. In this paper, we propose a flexible $Q$-SR model as well as its formulation in order to fully explore the potential of SR from an algorithmic perspective. The model leads to a highly extensible framework to design and evaluate algorithms that can be adapted to various network topologies and traffic matrices. For the offline setting, we develop a fully polynomial time approximation scheme (FPTAS) which can finds a $(1+ω)$-approximation solution for any specified $ω>0$ in time that is a polynomial function of the network size. To the best of our knowledge, the proposed FPTAS is the first algorithm that can compute arbitrarily accurate solution. For the online setting, we develop an online primal-dual algorithm that proves $O(1)$-competitive and violates link capacities by a factor of $O(\log n)$, where $n$ is the node number. We also prove performance bounds for the proposed algorithms. We conduct simulations on realistic topologies to validate SR parameters and algorithmic parameters in both offline and online scenarios.
Gender Bias in Multilingual Neural Machine Translation: The Architecture Matters
Venue: arXiv cs.CL 2020
Year: 2020

Multilingual Neural Machine Translation architectures mainly differ in the amount of sharing modules and parameters among languages. In this paper, and from an algorithmic perspective, we explore if the chosen architecture, when trained with the same data, influences the gender bias accuracy. Experiments in four language pairs show that Language-Specific encoders-decoders exhibit less bias than the Shared encoder-decoder architecture. Further interpretability analysis of source embeddings and the attention shows that, in the Language-Specific case, the embeddings encode more gender information, and its attention is more diverted. Both behaviors help in mitigating gender bias.
Spectral Ranking of Causal Influence in Complex Systems
Venue: arXiv cs.IT 2020
Year: 2020

Like natural complex systems such as the Earth's climate or a living cell, semiconductor lithography systems are characterized by nonlinear dynamics across more than a dozen orders of magnitude in space and time. Thousands of sensors measure relevant process variables at appropriate sampling rates, to provide time series as primary sources for system diagnostics. However, high-dimensionality, non-linearity and non-stationarity of data remain a major challenge to effectively diagnose rare or new system issues by merely using model-based approaches. To reduce the causal search space, we validate an algorithm that applies transfer entropy to obtain a weighted directed graph from a system's multivariate time series and graph eigenvector centrality to identify the system's most influential parameters. The results suggest that this approach robustly identifies the true influential sources in a complex system, even when its information transfer network includes redundant edges.