OpenRaiser/Intern-Atlas huggingface dataset:
Subdimensional Expansion for Multi-objective Multi-agent Path Finding
Venue: arXiv cs.RO 2021
Year: 2021

Conventional multi-agent path planners typically determine a path that optimizes a single objective, such as path length. Many applications, however, may require multiple objectives, say time-to-completion and fuel use, to be simultaneously optimized in the planning process. Often, these criteria may not be readily compared and sometimes lie in competition with each other. Simply applying standard multi-objective search algorithms to multi-agent path finding may prove to be inefficient because the size of the space of possible solutions, i.e., the Pareto-optimal set, can grow exponentially with the number of agents (the dimension of the search space). This paper presents an approach that bypasses this so-called curse of dimensionality by leveraging our prior multi-agent work with a framework called subdimensional expansion. One example of subdimensional expansion, when applied to A*, is called M* and M* was limited to a single objective function. We combine principles of dominance and subdimensional expansion to create a new algorithm named multi-objective M* (MOM*), which dynamically couples agents for planning only when those agents have to "interact" with each other. MOM* computes the complete Pareto-optimal set for multiple agents efficiently and naturally trades off sub-optimal approximations of the Pareto-optimal set and computational efficiency. Our approach is able to find the complete Pareto-optimal set for problem instances with hundreds of solutions which the standard multi-objective A* algorithms could not find within a bounded time.
Leveraging IoT and Weather Conditions to Estimate the Riders Waiting for the Bus Transit on Campus
Venue: arXiv cs.LG 2021
Year: 2021

The communication technology revolution in this era has increased the use of smartphones in the world of transportation. In this paper, we propose to leverage IoT device data, capturing passengers' smartphones' Wi-Fi data in conjunction with weather conditions to predict the expected number of passengers waiting at a bus stop at a specific time using deep learning models. Our study collected data from the transit bus system at James Madison University (JMU) in Virginia, USA. This paper studies the correlation between the number of passengers waiting at bus stops and weather conditions. Empirically, an experiment with several bus stops in JMU, was utilized to confirm a high precision level. We compared our Deep Neural Network (DNN) model against two baseline models: Linear Regression (LR) and a Wide Neural Network (WNN). The gap between the baseline models and DNN was 35% and 14% better Mean Squared Error (MSE) scores for predictions in favor of the DNN compared to LR and WNN, respectively.
Bayesian Neural Networks for Virtual Flow Metering: An Empirical Study
Venue: arXiv cs.LG 2021
Year: 2021

Recent works have presented promising results from the application of machine learning (ML) to the modeling of flow rates in oil and gas wells. Encouraging results and advantageous properties of ML models, such as computationally cheap evaluation and ease of calibration to new data, have sparked optimism for the development of data-driven virtual flow meters (VFMs). Data-driven VFMs are developed in the small data regime, where it is important to question the uncertainty and robustness of models. The modeling of uncertainty may help to build trust in models, which is a prerequisite for industrial applications. The contribution of this paper is the introduction of a probabilistic VFM based on Bayesian neural networks. Uncertainty in the model and measurements is described, and the paper shows how to perform approximate Bayesian inference using variational inference. The method is studied by modeling on a large and heterogeneous dataset, consisting of 60 wells across five different oil and gas assets. The predictive performance is analyzed on historical and future test data, where an average error of 4-6% and 8-13% is achieved for the 50% best performing models, respectively. Variational inference appears to provide more robust predictions than the reference approach on future data. Prediction performance and uncertainty calibration is explored in detail and discussed in light of four data challenges. The findings motivate the development of alternative strategies to improve the robustness of data-driven VFMs.
Low-Rate Overuse Flow Tracer (LOFT): An Efficient and Scalable Algorithm for Detecting Overuse Flows
Venue: arXiv cs.NI 2021
Year: 2021

Current probabilistic flow-size monitoring can only detect heavy hitters (e.g., flows utilizing 10 times their permitted bandwidth), but cannot detect smaller overuse (e.g., flows utilizing 50-100% more than their permitted bandwidth). Thus, these systems lack accuracy in the challenging environment of high-throughput packet processing, where fast-memory resources are scarce. Nevertheless, many applications rely on accurate flow-size estimation, e.g. for network monitoring, anomaly detection and Quality of Service. We design, analyze, implement, and evaluate LOFT, a new approach for efficiently detecting overuse flows that achieves dramatically better properties than prior work. LOFT can detect 1.5x overuse flows in one second, whereas prior approaches fail to detect 2x overuse flows within a timeout of 300 seconds. We demonstrate LOFT's suitability for high-speed packet processing with implementations in the DPDK framework and on an FPGA.
AURSAD: Universal Robot Screwdriving Anomaly Detection Dataset
Venue: arXiv cs.LG 2021
Year: 2021

Screwdriving is one of the most popular industrial processes. As such, it is increasingly common to automate that procedure by using various robots. Even though the automation increases the efficiency of the screwdriving process, if the process is not monitored correctly, faults may occur during operation, which can impact the effectiveness and quality of assembly. Machine Learning (ML) has the potential to detect those undesirable events and limit their impact. In order to do so, first a dataset that fully describes the operation of an industrial robot performing automated screwdriving must be available. This report describes a dataset created using a UR3e series robot and OnRobot Screwdriver. We create different scenarios and introduce 4 types of anomalies to the process while all available robot and screwdriver sensors are continuously recorded. The resulting data contains 2042 samples of normal and anomalous robot operation. Brief ML benchmarks using this data are also provided, showcasing the data's suitability and potential for further analysis and experimentation.
Face Recognition using 3D CNNs
Venue: arXiv cs.CV 2021
Year: 2021

The area of face recognition is one of the most widely researched areas in the domain of computer vision and biometric. This is because, the non-intrusive nature of face biometric makes it comparatively more suitable for application in area of surveillance at public places such as airports. The application of primitive methods in face recognition could not give very satisfactory performance. However, with the advent of machine and deep learning methods and their application in face recognition, several major breakthroughs were obtained. The use of 2D Convolution Neural networks(2D CNN) in face recognition crossed the human face recognition accuracy and reached to 99%. Still, robust face recognition in the presence of real world conditions such as variation in resolution, illumination and pose is a major challenge for researchers in face recognition. In this work, we used video as input to the 3D CNN architectures for capturing both spatial and time domain information from the video for face recognition in real world environment. For the purpose of experimentation, we have developed our own video dataset called CVBL video dataset. The use of 3D CNN for face recognition in videos shows promising results with DenseNets performing the best with an accuracy of 97% on CVBL dataset.
Enterprise domain ontology learning from web-based corpus
Venue: arXiv cs.AI 2021
Year: 2021

Enterprise knowledge is a key asset in the competing and fast-changing corporate landscape. The ability to learn, store and distribute implicit and explicit knowledge can be the difference between success and failure. While enterprise knowledge management is a well-defined research domain, current implementations lack orientation towards small and medium enterprise. We propose a semantic search engine for relevant documents in an enterprise, based on automatic generated domain ontologies. In this paper we focus on the component for ontology learning and population.
Orientation Convolutional Networks for Image Recognition
Venue: arXiv cs.CV 2021
Year: 2021

Deep Convolutional Neural Networks (DCNNs) are capable of obtaining powerful image representations, which have attracted great attentions in image recognition. However, they are limited in modeling orientation transformation by the internal mechanism. In this paper, we develop Orientation Convolution Networks (OCNs) for image recognition based on the proposed Landmark Gabor Filters (LGFs) that the robustness of the learned representation against changed of orientation can be enhanced. By modulating the convolutional filter with LGFs, OCNs can be compatible with any existing deep learning networks. LGFs act as a Gabor filter bank achieved by selecting $ p $ $ \left( \ll n\right) $ representative Gabor filters as andmarks and express the original Gabor filters as sparse linear combinations of these landmarks. Specifically, based on a matrix factorization framework, a flexible integration for the local and the global structure of original Gabor filters by sparsity and low-rank constraints is utilized. With the propogation of the low-rank structure, the corresponding sparsity for representation of original Gabor filter bank can be significantly promoted. Experimental results over several benchmarks demonstrate that our method is less sensitive to the orientation and produce higher performance both in accuracy and cost, compared with the existing state-of-art methods. Besides, our OCNs have few parameters to learn and can significantly reduce the complexity of training network.
CTC-based Compression for Direct Speech Translation
Venue: arXiv cs.CL 2021
Year: 2021

Previous studies demonstrated that a dynamic phone-informed compression of the input audio is beneficial for speech translation (ST). However, they required a dedicated model for phone recognition and did not test this solution for direct ST, in which a single model translates the input audio into the target language without intermediate representations. In this work, we propose the first method able to perform a dynamic compression of the input indirect ST models. In particular, we exploit the Connectionist Temporal Classification (CTC) to compress the input sequence according to its phonetic characteristics. Our experiments demonstrate that our solution brings a 1.3-1.5 BLEU improvement over a strong baseline on two language pairs (English-Italian and English-German), contextually reducing the memory footprint by more than 10%.
Size Matters
Venue: arXiv cs.LG 2021
Year: 2021

Fully convolutional neural networks can process input of arbitrary size by applying a combination of downsampling and pooling. However, we find that fully convolutional image classifiers are not agnostic to the input size but rather show significant differences in performance: presenting the same image at different scales can result in different outcomes. A closer look reveals that there is no simple relationship between input size and model performance (no `bigger is better'), but that each each network has a preferred input size, for which it shows best results. We investigate this phenomenon by applying different methods, including spectral analysis of layer activations and probe classifiers, showing that there are characteristic features depending on the network architecture. From this we find that the size of discriminatory features is critically influencing how the inference process is distributed among the layers.
Towards Multi-agent Reinforcement Learning for Wireless Network Protocol Synthesis
Venue: arXiv cs.LG 2021
Year: 2021

This paper proposes a multi-agent reinforcement learning based medium access framework for wireless networks. The access problem is formulated as a Markov Decision Process (MDP), and solved using reinforcement learning with every network node acting as a distributed learning agent. The solution components are developed step by step, starting from a single-node access scenario in which a node agent incrementally learns to control MAC layer packet loads for reining in self-collisions. The strategy is then scaled up for multi-node fully-connected scenarios by using more elaborate reward structures. It also demonstrates preliminary feasibility for more general partially connected topologies. It is shown that by learning to adjust MAC layer transmission probabilities, the protocol is not only able to attain theoretical maximum throughput at an optimal load, but unlike classical approaches, it can also retain that maximum throughput at higher loading conditions. Additionally, the mechanism is agnostic to heterogeneous loading while preserving that feature. It is also shown that access priorities of the protocol across nodes can be parametrically adjusted. Finally, it is also shown that the online learning feature of reinforcement learning is able to make the protocol adapt to time-varying loading conditions.
Statistical privacy-preserving message dissemination for peer-to-peer networks
Venue: arXiv cs.NI 2021
Year: 2021

Concerns for the privacy of communication is widely discussed in research and overall society. For the public financial infrastructure of blockchains, this discussion encompasses the privacy of transaction data and its broadcasting throughout the network. To tackle this problem, we transform a discrete-time protocol for contact networks over infinite trees into a computer network protocol for peer-to-peer networks. Peer-to-peer networks are modeled as organically growing graphs. We show that the distribution of shortest paths in such a network can be modeled using a normal distribution $\mathcal{N}(μ,σ^2).$ We determine statistical estimators for $μ,σ$ via multivariate models. The model behaves logarithmic over the number of nodes n and proportional to an inverse exponential over the number of added edges k. These results facilitate the computation of optimal forwarding probabilities during the dissemination phase for optimal privacy in a limited information environment.
Depth separation beyond radial functions
Venue: arXiv cs.LG 2021
Year: 2021

High-dimensional depth separation results for neural networks show that certain functions can be efficiently approximated by two-hidden-layer networks but not by one-hidden-layer ones in high-dimensions $d$. Existing results of this type mainly focus on functions with an underlying radial or one-dimensional structure, which are usually not encountered in practice. The first contribution of this paper is to extend such results to a more general class of functions, namely functions with piece-wise oscillatory structure, by building on the proof strategy of (Eldan and Shamir, 2016). We complement these results by showing that, if the domain radius and the rate of oscillation of the objective function are constant, then approximation by one-hidden-layer networks holds at a $\mathrm{poly}(d)$ rate for any fixed error threshold. A common theme in the proofs of depth-separation results is the fact that one-hidden-layer networks fail to approximate high-energy functions whose Fourier representation is spread in the domain. On the other hand, existing approximation results of a function by one-hidden-layer neural networks rely on the function having a sparse Fourier representation. The choice of the domain also represents a source of gaps between upper and lower approximation bounds. Focusing on a fixed approximation domain, namely the sphere $\mathbb{S}^{d-1}$ in dimension $d$, we provide a characterisation of both functions which are efficiently approximable by one-hidden-layer networks and of functions which are provably not, in terms of their Fourier expansion.
Supporting Real Demands in Software Engineering with a Four Steps Project-Based Learning Approach
Venue: arXiv cs.SE 2021
Year: 2021

Project-based learning (PBL) is a student-centered and learn-by-doing approach that organizes learning around projects. While entrepreneurship and PBL in SE education are thrilling research topics, there seems to be very little work focusing on the pros and cons of involving external stakeholders to support real demands in software engineering education. Working on real projects also supports students to acquire leadership skills, such as communication, project management, and teamwork. This paper describes a case study integrating students from different Software Engineering programs and involving external stakeholders, underpinned by PBL concepts. We present how this study was designed and implemented in a large institution, in four steps, summarized as follows: (I) requirements gathering and design; (II) information system development and implementation; (III) integration tests and deployment process; (IV) support and maintenance activities. The study had the participation of 59 students from a professional technical course in step one, working in teams, and 10 undergraduate students from a Bachelor's program in Information Systems in the following steps, working in pairs. Overall, the feedback from stakeholders and students exceeded expectations, although it increased the workload of teachers. We were able to distill a new set of lessons learned, and we expect that at least some of them will be useful for anyone implementing a similar course. As a consequence of this study, we plan to institutionally formalize the PBL course improvement process by defining specific outcomes and measurements.
A New Design of Cache-aided Multiuser Private Information Retrieval with Uncoded Prefetching
Venue: arXiv cs.IT 2021
Year: 2021

In the problem of cache-aided multiuser private information retrieval (MuPIR), a set of $K_{\rm u}$ cache-equipped users wish to privately download a set of messages from $N$ distributed databases each holding a library of $K$ messages. The system works in two phases: {\it cache placement (prefetching) phase} in which the users fill up their cache memory, and {\it private delivery phase} in which the users' demands are revealed and they download an answer from each database so that the their desired messages can be recovered while each individual database learns nothing about the identities of the requested messages. The goal is to design the placement and the private delivery phases such that the \emph{load}, which is defined as the total number of downloaded bits normalized by the message size, is minimized given any user memory size. This paper considers the MuPIR problem with two messages, arbitrary number of users and databases where uncoded prefetching is assumed, i.e., the users directly copy some bits from the library as their cached contents. We propose a novel MuPIR scheme inspired by the Maddah-Ali and Niesen (MAN) coded caching scheme. The proposed scheme achieves lower load than any existing schemes, especially the product design (PD), and is shown to be optimal within a factor of $8$ in general and exactly optimal at very high or low memory regime.
Private Linear Transformation: The Joint Privacy Case
Venue: arXiv cs.IT 2021
Year: 2021

We introduce the problem of Private Linear Transformation (PLT). This problem includes a single (or multiple) remote server(s) storing (identical copies of) $K$ messages and a user who wants to compute $L$ linear combinations of a $D$-subset of these messages by downloading the minimum amount of information from the server(s) while protecting the privacy of the entire set of $D$ messages. This problem generalizes the Private Information Retrieval and Private Linear Computation problems. In this work, we focus on the single-server case. For the setting in which the coefficient matrix of the required $L$ linear combinations generates a Maximum Distance Separable (MDS) code, we characterize the capacity -- defined as the supremum of all achievable download rates, for all parameters $K, D, L$. In addition, we present lower and/or upper bounds on the capacity for the settings with non-MDS coefficient matrices and the settings with a prior side information.
Skills-based on technological knowledge in the digital economy activity
Venue: arXiv cs.CY 2021
Year: 2021

This research seeks to measure the impact of people with technological knowledge on regional digital economic activity and the implications of prosperous cities' contagion effect on neighbouring ones. The focus of this study is quantitative, cross-sectional, and its design is correlational-causal. This study covers seven micro-regions of Minas Gerais in Brazil, organized in 89 municipalities, with 69% urban population and 31% rural. The data used consisted of 4,361 observations obtained in the Brazilian government's public repositories, organized into panel data, and analysed using partial least squares, micro-regional spatial regression, and identification patterns with machine learning. The confirmatory analysis of the regression test establishes a significant impact between the CE's technological knowledge and the digital economic activity AED through a predictive value of R2 = .749, \b{eta} = .867, p = .000 (value t = 18,298). With high notoriety among the variables, public and private university institutions (IUPP), professors with doctorates and masters (DCNT), and information technology occupations (CBO). A geographic concentration of companies that demand technology-based skills had effects by slowing down the development of small municipalities, suggesting the development of new government technology initiatives that support new business models based on technological knowledge.
Reliability Analysis of Artificial Intelligence Systems Using Recurrent Events Data from Autonomous Vehicles
Venue: arXiv cs.AI 2021
Year: 2021

Artificial intelligence (AI) systems have become increasingly common and the trend will continue. Examples of AI systems include autonomous vehicles (AV), computer vision, natural language processing, and AI medical experts. To allow for safe and effective deployment of AI systems, the reliability of such systems needs to be assessed. Traditionally, reliability assessment is based on reliability test data and the subsequent statistical modeling and analysis. The availability of reliability data for AI systems, however, is limited because such data are typically sensitive and proprietary. The California Department of Motor Vehicles (DMV) oversees and regulates an AV testing program, in which many AV manufacturers are conducting AV road tests. Manufacturers participating in the program are required to report recurrent disengagement events to California DMV. This information is being made available to the public. In this paper, we use recurrent disengagement events as a representation of the reliability of the AI system in AV, and propose a statistical framework for modeling and analyzing the recurrent events data from AV driving tests. We use traditional parametric models in software reliability and propose a new nonparametric model based on monotonic splines to describe the event process. We develop inference procedures for selecting the best models, quantifying uncertainty, and testing heterogeneity in the event process. We then analyze the recurrent events data from four AV manufacturers, and make inferences on the reliability of the AI systems in AV. We also describe how the proposed analysis can be applied to assess the reliability of other AI systems.