OpenRaiser/Intern-Atlas huggingface dataset:
DISCO Verification: Division of Input Space into COnvex polytopes for neural network verification
Venue: arXiv cs.AI 2021
Year: 2021

The impressive results of modern neural networks partly come from their non linear behaviour. Unfortunately, this property makes it very difficult to apply formal verification tools, even if we restrict ourselves to networks with a piecewise linear structure. However, such networks yields subregions that are linear and thus simpler to analyse independently. In this paper, we propose a method to simplify the verification problem by operating a partitionning into multiple linear subproblems. To evaluate the feasibility of such an approach, we perform an empirical analysis of neural networks to estimate the number of linear regions, and compare them to the bounds currently known. We also present the impact of a technique aiming at reducing the number of linear regions during training.
Multi-output, multi-level, multi-gate design using non-linear programming
Venue: arXiv cs.AR 2021
Year: 2021

Using logic gates is the traditional way of designing logic circuits. However, most of the minimization algorithms concern a limited set of gates (complete sets), like sum of products, exclusive-or sum of products, NAND gates, NOR gates e.t.c.. In this paper, a method is proposed for minimizing multi-output Boolean functions using any kind of two-input gates although it can easily be extended to multi-input gates. The method is based on non-linear mixed integer programming. The experimental results show that the method gives the same or better results compared to other methods available in the literature. However, other methods do not ensure that they produce the minimal solution, while the main advantages of the proposed method are that it does guarantee minimality and it can also handle Boolean functions for incompletely specified functions. The method is general enough and can easily be extended to more complicated design modules than just basic gates.
STRIDE : Scene Text Recognition In-Device
Venue: arXiv cs.CV 2021
Year: 2021

Optical Character Recognition (OCR) systems have been widely used in various applications for extracting semantic information from images. To give the user more control over their privacy, an on-device solution is needed. The current state-of-the-art models are too heavy and complex to be deployed on-device. We develop an efficient lightweight scene text recognition (STR) system, which has only 0.88M parameters and performs real-time text recognition. Attention modules tend to boost the accuracy of STR networks but are generally slow and not optimized for device inference. So, we propose the use of convolution attention modules to the text recognition networks, which aims to provide channel and spatial attention information to the LSTM module by adding very minimal computational cost. It boosts our word accuracy on ICDAR 13 dataset by almost 2\%. We also introduce a novel orientation classifier module, to support the simultaneous recognition of both horizontal and vertical text. The proposed model surpasses on-device metrics of inference time and memory footprint and achieves comparable accuracy when compared to the leading commercial and other open-source OCR engines. We deploy the system on-device with an inference speed of 2.44 ms per word on the Exynos 990 chipset device and achieve an accuracy of 88.4\% on ICDAR-13 dataset.
Dissolving yourself in connection to others: shared experiences of ego attenuation and connectedness during group VR experiences can be comparable to psychedelics
Venue: arXiv cs.HC 2021
Year: 2021

With a growing body of research highlighting the therapeutic potential of experiential phenomenology which diminishes egoic identity and increases one's sense of connectedness, there is significant interest in how to elicit such 'self-transcendent experiences' (STEs) in laboratory contexts. Psychedelic drugs (YDs) have proven particularly effective in this respect, producing subjective phenomenology which reliably elicits intense STEs. With virtual reality (VR) emerging as a powerful tool for constructing new perceptual environments, we describe a VR framework called 'Isness-distributed' (Isness-D) which harnesses the unique affordances of distributed multi-person VR to blur conventional self-other boundaries. Within Isness-D, groups of participants co-habit a shared virtual space, collectively experiencing their bodies as luminous energetic essences with diffuse spatial boundaries. It enables moments of 'energetic coalescence', a new class of embodied phenomenological intersubjective experience where bodies can fluidly merge, enabling participants to have an experience of including multiple others within their self-representation. To evaluate Isness-D, we adopted a citizen science approach, coordinating an international network of Isness-D 'nodes'. We analyzed the results (N = 58) using 4 different self-report scales previously applied to analyze subjective YD phenomenology (the inclusion of community in self scale, ego-dissolution inventory, communitas scale, and the MEQ30 mystical experience questionnaire). Despite the complexities associated with a distributed experiment like this, the Isness-D scores on all 4 scales were statistically indistinguishable from recently published YD studies, demonstrating that distributed VR can be used to design intersubjective STEs where people dissolve their sense of self in the connection to others.
How to Explain Neural Networks: an Approximation Perspective
Venue: arXiv cs.LG 2021
Year: 2021

The lack of interpretability has hindered the large-scale adoption of AI technologies. However, the fundamental idea of interpretability, as well as how to put it into practice, remains unclear. We provide notions of interpretability based on approximation theory in this study. We first implement this approximation interpretation on a specific model (fully connected neural network) and then propose to use MLP as a universal interpreter to explain arbitrary black-box models. Extensive experiments demonstrate the effectiveness of our approach.
An Experimental Analysis of Work-Life Balance Among The Employees using Machine Learning Classifiers
Venue: arXiv cs.CY 2021
Year: 2021

Researchers today have found out the importance of Artificial Intelligence, and Machine Learning in our daily lives, as well as they can be used to improve the quality of our lives as well as the cities and nations alike. An example of this is that it is currently speculated that ML can provide ways to relieve workers as it can predict effective working schedules and patterns which increase the efficiency of the workers. Ultimately this is leading to a Work-Life Balance for the workers. But how is this possible? It is practically possible with the Machine Learning algorithms to predict, calculate the factors affecting the feelings of the worker's work-life balance. In order to actually do this, a sizeable amount of 12,756 people's data has been taken under consideration. Upon analysing the data and calculating under various factors, we have found out the correlation of various factors and WLB(Work-Life Balance in short). There are some factors that have to be taken into serious consideration as they play a major role in WLB. We have trained 80% of our data with Random Forest Classifier, SVM and Naive Bayes algorithms. Upon testing, the algorithms predict the WLB with 71.5% as the best accuracy.
Hard Choices and Hard Limits for Artificial Intelligence
Venue: arXiv cs.CY 2021
Year: 2021

Artificial intelligence (AI) is supposed to help us make better choices. Some of these choices are small, like what route to take to work, or what music to listen to. Others are big, like what treatment to administer for a disease or how long to sentence someone for a crime. If AI can assist with these big decisions, we might think it can also help with hard choices, cases where alternatives are neither better, worse nor equal but on a par. The aim of this paper, however, is to show that this view is mistaken: the fact of parity shows that there are hard limits on AI in decision making and choices that AI cannot, and should not, resolve.
Optimal Group Formulation Using Machine Learning
Venue: arXiv cs.CY 2021
Year: 2021

Group formation itself a perplexing process. Over the decade of time education and others disciple has improved imminently but optimal group formation in educational system is still struggling. Our research focus on to create optimal group in a class of any institute. In this research we use Simulated Annealing (SA) for best group formation based on the previous academic record. We generally create an arbitrary cluster first then optimise using SA. Our model has significant success rate over a large number of datasets. This research will play a pioneer role in group formations in the academic and related researches.
The challenges and realities of retailing in a COVID-19 world: Identifying trending and Vital During Crisis keywords during Covid-19 using Machine Learning (Austria as a case study)
Venue: arXiv cs.CY 2021
Year: 2021

From global pandemics to geopolitical turmoil, leaders in logistics, product allocation, procurement and operations are facing increasing difficulty with safeguarding their organizations against supply chain vulnerabilities. It is recommended to opt for forecasting against trending based benchmark because auditing a future forecast puts more focus on seasonality. The forecasting models provide with end-to-end, real time oversight of the entire supply chain, while utilizing predictive analytics and artificial intelligence to identify potential disruptions before they occur. By combining internal and external data points, coming up with an AI-enabled modelling engine can greatly reduce risk by helping retail companies proactively respond to supply and demand variability. This research paper puts focus on creating an ingenious way to tackle the impact of COVID19 on Supply chain, product allocation, trending and seasonality. Key words: Supply chain, covid-19, forecasting, coronavirus, manufacturing, seasonality, trending, retail.
Efficient and accurate group testing via Belief Propagation: an empirical study
Venue: arXiv cs.AI 2021
Year: 2021

The group testing problem asks for efficient pooling schemes and algorithms that allow to screen moderately large numbers of samples for rare infections. The goal is to accurately identify the infected samples while conducting the least possible number of tests. Exploring the use of techniques centred around the Belief Propagation message passing algorithm, we suggest a new test design that significantly increases the accuracy of the results. The new design comes with Belief Propagation as an efficient inference algorithm. Aiming for results on practical rather than asymptotic problem sizes, we conduct an experimental study.
Topology for Substrate Routing in Semiconductor Package Design
Venue: arXiv cs.CG 2021
Year: 2021

In this work, we propose a new signal routing method for solving routing problems that occur in the design process of semiconductor package substrates. Our work uses a topological transformation of the layers of the package substrate in order to simplify the routing problem into a problem of connecting points on a circle with non-intersecting straight line segments. The circle, which we call the Circular Frame, is a polygonal schema, which is originally used in topology to study the topological structure of 2-manifolds. We show through experiments that our new routing method based on the Circular Frame competes with certain grid-based routing algorithms.
Physics-informed attention-based neural network for solving non-linear partial differential equations
Venue: arXiv cs.LG 2021
Year: 2021

Physics-Informed Neural Networks (PINNs) have enabled significant improvements in modelling physical processes described by partial differential equations (PDEs). PINNs are based on simple architectures, and learn the behavior of complex physical systems by optimizing the network parameters to minimize the residual of the underlying PDE. Current network architectures share some of the limitations of classical numerical discretization schemes when applied to non-linear differential equations in continuum mechanics. A paradigmatic example is the solution of hyperbolic conservation laws that develop highly localized nonlinear shock waves. Learning solutions of PDEs with dominant hyperbolic character is a challenge for current PINN approaches, which rely, like most grid-based numerical schemes, on adding artificial dissipation. Here, we address the fundamental question of which network architectures are best suited to learn the complex behavior of non-linear PDEs. We focus on network architecture rather than on residual regularization. Our new methodology, called Physics-Informed Attention-based Neural Networks, (PIANNs), is a combination of recurrent neural networks and attention mechanisms. The attention mechanism adapts the behavior of the deep neural network to the non-linear features of the solution, and break the current limitations of PINNs. We find that PIANNs effectively capture the shock front in a hyperbolic model problem, and are capable of providing high-quality solutions inside and beyond the training set.
BigEarthNet-MM: A Large Scale Multi-Modal Multi-Label Benchmark Archive for Remote Sensing Image Classification and Retrieval
Venue: arXiv cs.CV 2021
Year: 2021

This paper presents the multi-modal BigEarthNet (BigEarthNet-MM) benchmark archive made up of 590,326 pairs of Sentinel-1 and Sentinel-2 image patches to support the deep learning (DL) studies in multi-modal multi-label remote sensing (RS) image retrieval and classification. Each pair of patches in BigEarthNet-MM is annotated with multi-labels provided by the CORINE Land Cover (CLC) map of 2018 based on its thematically most detailed Level-3 class nomenclature. Our initial research demonstrates that some CLC classes are challenging to be accurately described by only considering (single-date) BigEarthNet-MM images. In this paper, we also introduce an alternative class-nomenclature as an evolution of the original CLC labels to address this problem. This is achieved by interpreting and arranging the CLC Level-3 nomenclature based on the properties of BigEarthNet-MM images in a new nomenclature of 19 classes. In our experiments, we show the potential of BigEarthNet-MM for multi-modal multi-label image retrieval and classification problems by considering several state-of-the-art DL models. We also demonstrate that the DL models trained from scratch on BigEarthNet-MM outperform those pre-trained on ImageNet, especially in relation to some complex classes, including agriculture and other vegetated and natural environments. We make all the data and the DL models publicly available at https://bigearth.net, offering an important resource to support studies on multi-modal image scene classification and retrieval problems in RS.
Ensemble-based Semi-supervised Learning to Improve Noisy Soiling Annotations in Autonomous Driving
Venue: arXiv cs.CV 2021
Year: 2021

Manual annotation of soiling on surround view cameras is a very challenging and expensive task. The unclear boundary for various soiling categories like water drops or mud particles usually results in a large variance in the annotation quality. As a result, the models trained on such poorly annotated data are far from being optimal. In this paper, we focus on handling such noisy annotations via pseudo-label driven ensemble model which allow us to quickly spot problematic annotations and in most cases also sufficiently fixing them. We train a soiling segmentation model on both noisy and refined labels and demonstrate significant improvements using the refined annotations. It also illustrates that it is possible to effectively refine lower cost coarse annotations.
MMGET: A Markov model for generalized evidence theory
Venue: arXiv cs.AI 2021
Year: 2021

In real life, lots of information merges from time to time. To appropriately describe the actual situations, lots of theories have been proposed. Among them, Dempster-Shafer evidence theory is a very useful tool in managing uncertain information. To better adapt to complex situations of open world, a generalized evidence theory is designed. However, everything occurs in sequence and owns some underlying relationships with each other. In order to further embody the details of information and better conforms to situations of real world, a Markov model is introduced into the generalized evidence theory which helps extract complete information volume from evidence provided. Besides, some numerical examples is offered to verify the correctness and rationality of the proposed method.
Behavior-based Neuroevolutionary Training in Reinforcement Learning
Venue: arXiv cs.NE 2021
Year: 2021

In addition to their undisputed success in solving classical optimization problems, neuroevolutionary and population-based algorithms have become an alternative to standard reinforcement learning methods. However, evolutionary methods often lack the sample efficiency of standard value-based methods that leverage gathered state and value experience. If reinforcement learning for real-world problems with significant resource cost is considered, sample efficiency is essential. The enhancement of evolutionary algorithms with experience exploiting methods is thus desired and promises valuable insights. This work presents a hybrid algorithm that combines topology-changing neuroevolutionary optimization with value-based reinforcement learning. We illustrate how the behavior of policies can be used to create distance and loss functions, which benefit from stored experiences and calculated state values. They allow us to model behavior and perform a directed search in the behavior space by gradient-free evolutionary algorithms and surrogate-based optimization. For this purpose, we consolidate different methods to generate and optimize agent policies, creating a diverse population. We exemplify the performance of our algorithm on standard benchmarks and a purpose-built real-world problem. Our results indicate that combining methods can enhance the sample efficiency and learning speed for evolutionary approaches.
Learning to Automatically Catch Potholes in Worldwide Road Scene Images
Venue: arXiv cs.CV 2021
Year: 2021

Among several road hazards that are present in any paved way in the world, potholes are one of the most annoying and also involving higher maintenance costs. There exists an increasing interest on the automated detection of these hazards enabled by technological and research progress. Our research work tackled the challenge of pothole detection from images of real world road scenes. The main novelty resides on the application of the latest progress in AI to learn the visual appearance of potholes. We built a large dataset of images with pothole annotations. They contained road scenes from different cities in the world, taken with different cameras, vehicles and viewpoints under varied environmental conditions. Then, we fine-tuned four different object detection models based on Faster R-CNN and SSD deep neural networks. We achieved high average precision and the pothole detector was tested on the Nvidia DrivePX2 platform with GPGPU capability, which can be embedded on vehicles. Moreover, it was deployed on a real vehicle to notify the detected potholes to a given IoT platform as part of AUTOPILOT H2020 project.
Controlling an Inverted Pendulum with Policy Gradient Methods-A Tutorial
Venue: arXiv cs.LG 2021
Year: 2021

This paper provides the details of implementing two important policy gradient methods to solve the inverted pendulum problem. These are namely the Deep Deterministic Policy Gradient (DDPG) and the Proximal Policy Optimization (PPO) algorithm. The problem is solved by using an actor-critic model where an actor-network is used to learn the policy function and a critic network is to evaluate the actor-network by learning to estimate the Q function. Apart from briefly explaining the mathematics behind these two algorithms, the details of python implementation are provided which helps in demystifying the underlying complexity of the algorithm. In the process, the readers will be introduced to OpenAI/Gym, Tensorflow 2.x and Keras utilities used for implementing the above concepts.
Understanding and Improvement of Adversarial Training for Network Embedding from an Optimization Perspective
Venue: arXiv cs.LG 2021
Year: 2021

Network Embedding aims to learn a function mapping the nodes to Euclidean space contribute to multiple learning analysis tasks on networks. However, the noisy information behind the real-world networks and the overfitting problem both negatively impact the quality of embedding vectors. To tackle these problems, researchers utilize Adversarial Training for Network Embedding (AdvTNE) and achieve state-of-the-art performance. Unlike the mainstream methods introducing perturbations on the network structure or the data feature, AdvTNE directly perturbs the model parameters, which provides a new chance to understand the mechanism behind. In this paper, we explain AdvTNE theoretically from an optimization perspective. Considering the Power-law property of networks and the optimization objective, we analyze the reason for its excellent results. According to the above analysis, we propose a new activation to enhance the performance of AdvTNE. We conduct extensive experiments on four real networks to validate the effectiveness of our method in node classification and link prediction. The results demonstrate that our method is superior to the state-of-the-art baseline methods.
Supporting Context Monotonicity Abstractions in Neural NLI Models
Venue: arXiv cs.CL 2021
Year: 2021

Natural language contexts display logical regularities with respect to substitutions of related concepts: these are captured in a functional order-theoretic property called monotonicity. For a certain class of NLI problems where the resulting entailment label depends only on the context monotonicity and the relation between the substituted concepts, we build on previous techniques that aim to improve the performance of NLI models for these problems, as consistent performance across both upward and downward monotone contexts still seems difficult to attain even for state-of-the-art models. To this end, we reframe the problem of context monotonicity classification to make it compatible with transformer-based pre-trained NLI models and add this task to the training pipeline. Furthermore, we introduce a sound and complete simplified monotonicity logic formalism which describes our treatment of contexts as abstract units. Using the notions in our formalism, we adapt targeted challenge sets to investigate whether an intermediate context monotonicity classification task can aid NLI models' performance on examples exhibiting monotonicity reasoning.
StrobeNet: Category-Level Multiview Reconstruction of Articulated Objects
Venue: arXiv cs.CV 2021
Year: 2021

We present StrobeNet, a method for category-level 3D reconstruction of articulating objects from one or more unposed RGB images. Reconstructing general articulating object categories % has important applications, but is challenging since objects can have wide variation in shape, articulation, appearance and topology. We address this by building on the idea of category-level articulation canonicalization -- mapping observations to a canonical articulation which enables correspondence-free multiview aggregation. Our end-to-end trainable neural network estimates feature-enriched canonical 3D point clouds, articulation joints, and part segmentation from one or more unposed images of an object. These intermediate estimates are used to generate a final implicit 3D reconstruction.Our approach reconstructs objects even when they are observed in different articulations in images with large baselines, and animation of reconstructed shapes. Quantitative and qualitative evaluations on different object categories show that our method is able to achieve high reconstruction accuracy, especially as more views are added.