OpenRaiser/Intern-Atlas huggingface dataset:
An Attention Ensemble Approach for Efficient Text Classification of Indian Languages
Venue: arXiv cs.CL 2021
Year: 2021

The recent surge of complex attention-based deep learning architectures has led to extraordinary results in various downstream NLP tasks in the English language. However, such research for resource-constrained and morphologically rich Indian vernacular languages has been relatively limited. This paper proffers team SPPU\_AKAH's solution for the TechDOfication 2020 subtask-1f: which focuses on the coarse-grained technical domain identification of short text documents in Marathi, a Devanagari script-based Indian language. Availing the large dataset at hand, a hybrid CNN-BiLSTM attention ensemble model is proposed that competently combines the intermediate sentence representations generated by the convolutional neural network and the bidirectional long short-term memory, leading to efficient text classification. Experimental results show that the proposed model outperforms various baseline machine learning and deep learning models in the given task, giving the best validation accuracy of 89.57\% and f1-score of 0.8875. Furthermore, the solution resulted in the best system submission for this subtask, giving a test accuracy of 64.26\% and f1-score of 0.6157, transcending the performances of other teams as well as the baseline system given by the organizers of the shared task.
Towards Accurate and Compact Architectures via Neural Architecture Transformer
Venue: arXiv cs.CV 2021
Year: 2021

Designing effective architectures is one of the key factors behind the success of deep neural networks. Existing deep architectures are either manually designed or automatically searched by some Neural Architecture Search (NAS) methods. However, even a well-designed/searched architecture may still contain many nonsignificant or redundant modules/operations. Thus, it is necessary to optimize the operations inside an architecture to improve the performance without introducing extra computational cost. To this end, we have proposed a Neural Architecture Transformer (NAT) method which casts the optimization problem into a Markov Decision Process (MDP) and seeks to replace the redundant operations with more efficient operations, such as skip or null connection. Note that NAT only considers a small number of possible transitions and thus comes with a limited search/transition space. As a result, such a small search space may hamper the performance of architecture optimization. To address this issue, we propose a Neural Architecture Transformer++ (NAT++) method which further enlarges the set of candidate transitions to improve the performance of architecture optimization. Specifically, we present a two-level transition rule to obtain valid transitions, i.e., allowing operations to have more efficient types (e.g., convolution->separable convolution) or smaller kernel sizes (e.g., 5x5->3x3). Note that different operations may have different valid transitions. We further propose a Binary-Masked Softmax (BMSoftmax) layer to omit the possible invalid transitions. Extensive experiments on several benchmark datasets show that the transformed architecture significantly outperforms both its original counterpart and the architectures optimized by existing methods.
Unavailable Transit Feed Specification: Making it Available with Recurrent Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Studies on public transportation in Europe suggest that European inhabitants use buses in ca. 56% of all public transport travels. One of the critical factors affecting such a percentage and more, in general, the demand for public transport services, with an increasing reluctance to use them, is their quality. End-users can perceive quality from various perspectives, including the availability of information, i.e., the access to details about the transit and the provided services. The approach proposed in this paper, using innovative methodologies resorting on data mining and machine learning techniques, aims to make available the unavailable data about public transport. In particular, by mining GPS traces, we manage to reconstruct the complete transit graph of public transport. The approach has been successfully validated on a real dataset collected from the local bus system of the city of L'Aquila (Italy). The experimental results demonstrate that the proposed approach and implemented framework are both effective and efficient, thus being ready for deployment.
A Python Framework for Fast Modelling and Simulation of Cellular Nonlinear Networks and other Finite-difference Time-domain Systems
Venue: arXiv cs.DC 2021
Year: 2021

This paper introduces and evaluates a freely available cellular nonlinear network simulator optimized for the effective use of GPUs, to achieve fast modelling and simulations. Its relevance is demonstrated for several applications in nonlinear complex dynamical systems, such as slow-growth phenomena as well as for various image processing applications such as edge detection. The simulator is designed as a Jupyter notebook written in Python and functionally tested and optimized to run on the freely available cloud platform Google Collaboratory. Although the simulator, in its actual form, is designed to model the FitzHugh Nagumo Reaction-Diffusion cellular nonlinear network, it can be easily adapted for any other type of finite-difference time-domain model. Four implementation versions are considered, namely using the PyCUDA, NUMBA respectively CUPY libraries (all three supporting GPU computations) as well as a NUMPY-based implementation to be used when GPU is not available. The specificities and performances for each of the four implementations are analyzed concluding that the PyCUDA implementation ensures a very good performance being capable to run up to 14000 Mega cells per seconds (each cell referring to the basic nonlinear dynamic system composing the cellular nonlinear network).
Everything is Relative: Understanding Fairness with Optimal Transport
Venue: arXiv cs.CY 2021
Year: 2021

To study discrimination in automated decision-making systems, scholars have proposed several definitions of fairness, each expressing a different fair ideal. These definitions require practitioners to make complex decisions regarding which notion to employ and are often difficult to use in practice since they make a binary judgement a system is fair or unfair instead of explaining the structure of the detected unfairness. We present an optimal transport-based approach to fairness that offers an interpretable and quantifiable exploration of bias and its structure by comparing a pair of outcomes to one another. In this work, we use the optimal transport map to examine individual, subgroup, and group fairness. Our framework is able to recover well known examples of algorithmic discrimination, detect unfairness when other metrics fail, and explore recourse opportunities.
Self-Supervised Learning via multi-Transformation Classification for Action Recognition
Venue: arXiv cs.CV 2021
Year: 2021

Self-supervised tasks have been utilized to build useful representations that can be used in downstream tasks when the annotation is unavailable. In this paper, we introduce a self-supervised video representation learning method based on the multi-transformation classification to efficiently classify human actions. Self-supervised learning on various transformations not only provides richer contextual information but also enables the visual representation more robust to the transforms. The spatio-temporal representation of the video is learned in a self-supervised manner by classifying seven different transformations i.e. rotation, clip inversion, permutation, split, join transformation, color switch, frame replacement, noise addition. First, seven different video transformations are applied to video clips. Then the 3D convolutional neural networks are utilized to extract features for clips and these features are processed to classify the pseudo-labels. We use the learned models in pretext tasks as the pre-trained models and fine-tune them to recognize human actions in the downstream task. We have conducted the experiments on UCF101 and HMDB51 datasets together with C3D and 3D Resnet-18 as backbone networks. The experimental results have shown that our proposed framework is outperformed other SOTA self-supervised action recognition approaches. The code will be made publicly available.
Efficient Numerical Methods for Secrecy Capacity of Gaussian MIMO Wiretap Channel
Venue: arXiv cs.IT 2021
Year: 2021

This paper presents two different low-complexity methods for obtaining the secrecy capacity of multiple-input multiple-output (MIMO) wiretap channel subject to a sum power constraint (SPC). The challenges in deriving computationally efficient solutions to the secrecy capacity problem are due to the fact that the secrecy rate is a difference of convex functions (DC) of the transmit covariance matrix, for which its convexity is only known for \emph{the degraded case}. In the first method, we capitalize on the accelerated DC algorithm, which requires solving a sequence of convex subproblems. In particular, we show that each subproblem indeed admits a water-filling solution. In the second method, based on the equivalent convex-concave reformulation of the secrecy capacity problem, we develop a so-called partial best response algorithm (PBRA). Each iteration of the PBRA is also done in closed form. Simulation results are provided to demonstrate the superior performance of the proposed methods.
Importance of Environment Design in Reinforcement Learning: A Study of a Robotic Environment
Venue: arXiv cs.LG 2021
Year: 2021

An in-depth understanding of the particular environment is crucial in reinforcement learning (RL). To address this challenge, the decision-making process of a mobile collaborative robotic assistant modeled by the Markov decision process (MDP) framework is studied in this paper. The optimal state-action combinations of the MDP are calculated with the non-linear Bellman optimality equations. This system of equations can be solved with relative ease by the computational power of Wolfram Mathematica, where the obtained optimal action-values point to the optimal policy. Unlike other RL algorithms, this methodology does not approximate the optimal behavior, it gives the exact, explicit solution, which provides a strong foundation for our study. With this, we offer new insights into understanding the action selection mechanisms in RL by presenting various small modifications on the very same schema that lead to different optimal policies.
Decaying Clipping Range in Proximal Policy Optimization
Venue: arXiv cs.LG 2021
Year: 2021

Proximal Policy Optimization (PPO) is among the most widely used algorithms in reinforcement learning, which achieves state-of-the-art performance in many challenging problems. The keys to its success are the reliable policy updates through the clipping mechanism and the multiple epochs of minibatch updates. The aim of this research is to give new simple but effective alternatives to the former. For this, we propose linearly and exponentially decaying clipping range approaches throughout the training. With these, we would like to provide higher exploration at the beginning and stronger restrictions at the end of the learning phase. We investigate their performance in several classical control and locomotive robotic environments. During the analysis, we found that they influence the achieved rewards and are effective alternatives to the constant clipping method in many reinforcement learning tasks.
Trumpets: Injective Flows for Inference and Inverse Problems
Venue: arXiv cs.LG 2021
Year: 2021

We propose injective generative models called Trumpets that generalize invertible normalizing flows. The proposed generators progressively increase dimension from a low-dimensional latent space. We demonstrate that Trumpets can be trained orders of magnitudes faster than standard flows while yielding samples of comparable or better quality. They retain many of the advantages of the standard flows such as training based on maximum likelihood and a fast, exact inverse of the generator. Since Trumpets are injective and have fast inverses, they can be effectively used for downstream Bayesian inference. To wit, we use Trumpet priors for maximum a posteriori estimation in the context of image reconstruction from compressive measurements, outperforming competitive baselines in terms of reconstruction quality and speed. We then propose an efficient method for posterior characterization and uncertainty quantification with Trumpets by taking advantage of the low-dimensional latent space.
Towards the k-server conjecture: A unifying potential, pushing the frontier to the circle
Venue: arXiv cs.DS 2021
Year: 2021

The $k$-server conjecture, first posed by Manasse, McGeoch and Sleator in 1988, states that a $k$-competitive deterministic algorithm for the $k$-server problem exists. It is conjectured that the work function algorithm (WFA) achieves this guarantee, a multi-purpose algorithm with applications to various online problems. This has been shown for several special cases: $k=2$, $(k+1)$-point metrics, $(k+2)$-point metrics, the line metric, weighted star metrics, and $k=3$ in the Manhattan plane. The known proofs of these results are based on potential functions tied to each particular special case, thus requiring six different potential functions for the six cases. We present a single potential function proving $k$-competitiveness of WFA for all these cases. We also use this potential to show $k$-competitiveness of WFA on multiray spaces and for $k=3$ on trees. While the DoubleCoverage algorithm was known to be $k$-competitive for these latter cases, it has been open for WFA. Our potential captures a type of lazy adversary and thus shows that in all settled cases, the worst-case adversary is lazy. Chrobak and Larmore conjectured in 1992 that a potential capturing the lazy adversary would resolve the $k$-server conjecture. To our major surprise, this is not the case, as we show (using connections to the $k$-taxi problem) that our potential fails for three servers on the circle. Thus, our potential highlights laziness of the adversary as a fundamental property that is shared by all settled cases but violated in general. On the one hand, this weakens our confidence in the validity of the $k$-server conjecture. On the other hand, if the $k$-server conjecture holds, then we believe it can be proved by a variant of our potential.
Effect of Cloud Based Learning Management System on The Learning Management System Implementation Process: Faculty and Student Perspectives
Venue: arXiv cs.CY 2021
Year: 2021

The concept of E-learning in Universities has grown rapidly over the years to include not just only a learning management system but also tools initially not designed for learning such as Facebook and advanced learning tools, for example games, simulations and virtualization. As a result, Cloud-based LMS is being touted as the next evolution of the traditional LMS. It is hoped that Cloud based LMS will resolve some of the challenges associated with the traditional LMS implementation process. In a previous study, we reported that lack of involvement of faculty and students in the LMS implementation process results in the limited use of the LMS by faculty and students. The question then is, Will the cloud-based LMS resolve these issues? We conducted a review of literature and presented an overview of the traditional LMS, cloud computing and the cloudbased LMS and we described how the cloud computing LMS resolve issues raised by faculty and students. we find that even though, cloud-based LMS resolve most of the technical issues associated with the traditional LMS, some of the human issues were not resolved. We hope that this study draws attention to non-technical issues associated with the LMS implementation process.
Semi-supervised learning combining backpropagation and STDP: STDP enhances learning by backpropagation with a small amount of labeled data in a spiking neural network
Venue: arXiv cs.NE 2021
Year: 2021

A semi-supervised learning method for spiking neural networks is proposed. The proposed method consists of supervised learning by backpropagation and subsequent unsupervised learning by spike-timing-dependent plasticity (STDP), which is a biologically plausible learning rule. Numerical experiments show that the proposed method improves the accuracy without additional labeling when a small amount of labeled data is used. This feature has not been achieved by existing semi-supervised learning methods of discriminative models. It is possible to implement the proposed learning method for event-driven systems. Hence, it would be highly efficient in real-time problems if it were implemented on neuromorphic hardware. The results suggest that STDP plays an important role other than self-organization when applied after supervised learning, which differs from the previous method of using STDP as pre-training interpreted as self-organization.
A Concept Knowledge-Driven Keywords Retrieval Framework for Sponsored Search
Venue: arXiv cs.IR 2021
Year: 2021

In sponsored search, retrieving synonymous keywords for exact match type is important for accurately targeted advertising. Data-driven deep learning-based method has been proposed to tackle this problem. An apparent disadvantage of this method is its poor generalization performance on entity-level long-tail instances, even though they might share similar concept-level patterns with frequent instances. With the help of a large knowledge base, we find that most commercial synonymous query-keyword pairs can be abstracted into meaningful conceptual patterns through concept tagging. Based on this fact, we propose a novel knowledge-driven conceptual retrieval framework to mitigate this problem, which consists of three parts: data conceptualization, matching via conceptual patterns and concept-augmented discrimination. Both offline and online experiments show that our method is very effective. This framework has been successfully applied to Baidu's sponsored search system, which yields a significant improvement in revenue.
A Projection-Stable Grammatical Model for the Distributed Execution of Administrative Processes with Emphasis on Actors' Views
Venue: arXiv cs.SE 2021
Year: 2021

During the last two decades, the decentralized execution of business processes has been one of the main research topics in Business Process Management. Several models (languages) for processes' specification in order to facilitate their distributed execution, have been proposed. LSAWfP is among the most recent in this area: it helps to specify administrative processes with grammatical models indicating, in addition to their fundamental elements, the permissions (reading, writing and execution) of each actor in relation to each of their tasks. In this paper, we present a model for a completely decentralized and artifact-centric execution of administrative processes specified using LSAWfP. The presented model puts particular emphasis on actors' views: it then allows the confidential execution of certain tasks by ensuring that, each actor potentially has only a partial perception of the processes' global execution states. The model thus solves a very important problem in business process execution, which is often sidelined in existing approaches. To accomplish this, the model rely on three projection algorithms allowing to partially replicate the processes' global execution states at a given moment, to consistently update the obtained partial states and to deduce new coherent global states. The proposal of these three algorithms, the proof of underlying mathematical tools' stability and a proposal of their implementation, are this paper's main contributions.
Mapping Surgeon's Hand/Finger Motion During Conventional Microsurgery to Enhance Intuitive Surgical Robot Teleoperation
Venue: arXiv cs.RO 2021
Year: 2021

Purpose: Recent developments in robotics and artificial intelligence (AI) have led to significant advances in healthcare technologies enhancing robot-assisted minimally invasive surgery (RAMIS) in some surgical specialties. However, current human-robot interfaces lack intuitive teleoperation and cannot mimic surgeon's hand/finger sensing and fine motion. These limitations make tele-operated robotic surgery not suitable for micro-surgery and difficult to learn for established surgeons. We report a pilot study showing an intuitive way of recording and mapping surgeon's gross hand motion and the fine synergic motion during cardiac micro-surgery as a way to enhance future intuitive teleoperation. Methods: We set to develop a prototype system able to train a Deep Neural Net-work (DNN) by mapping wrist, hand and surgical tool real-time data acquisition(RTDA) inputs during mock-up heart micro-surgery procedures. The trained network was used to estimate the tools poses from refined hand joint angles. Results: Based on surgeon's feedback during mock micro-surgery, the developed wearable system with light-weight sensors for motion tracking did not interfere with the surgery and instrument handling. The wearable motion tracking system used 15 finger-thumb-wrist joint angle sensors to generate meaningful data-sets representing inputs of the DNN network with new hand joint angles added as necessary based on comparing the estimated tool poses against measured tool pose. The DNN architecture was optimized for the highest estimation accuracy and the ability to determine the tool pose with the least mean squared error. This novel approach showed that the surgical instrument's pose, an essential requirement for teleoperation, can be accurately estimated from recorded surgeon's hand/finger movements with a mean squared error (MSE) less than 0.3%
Model Checking for Decision Making System of Long Endurance Unmanned Surface Vehicle
Venue: arXiv cs.FL 2021
Year: 2021

This work aims to develop a model checking method to verify the decision making system of Unmanned Surface Vehicle (USV) in a long range surveillance mission. The scenario in this work was captured from a long endurance USV surveillance mission using C-Enduro, an USV manufactured by ASV Ltd. The C-Enduro USV may encounter multiple non-deterministic and concurrent problems including lost communication signals, collision risk and malfunction. The vehicle is designed to utilise multiple energy sources from solar panel, wind turbine and diesel generator. The energy state can be affected by the solar irradiance condition, wind condition, states of the diesel generator, sea current condition and states of the USV. In this research, the states and the interactive relations between environmental uncertainties, sensors, USV energy system, USV and Ground Control Station (GCS) decision making systems are abstracted and modelled successfully using Kripke models. The desirable properties to be verified are expressed using temporal logic statement and finally the safety properties and the long endurance properties are verified using the model checker MCMAS, a model checker for multi-agent systems. The verification results are analyzed and show the feasibility of applying model checking method to retrospect the desirable property of the USV decision making system. This method could assist researcher to identify potential design error of decision making system in advance.