OpenRaiser/Intern-Atlas huggingface dataset:
Design Knowledge Representation with Technology Semantic Network
Venue: arXiv cs.IR 2020
Year: 2020

Engineers often need to discover and learn designs from unfamiliar domains for inspiration or other particular uses. However, the complexity of the technical design descriptions and the unfamiliarity to the domain make it hard for engineers to comprehend the function, behavior, and structure of a design. To help engineers quickly understand a complex technical design description new to them, one approach is to represent it as a network graph of the design-related entities and their relations as an abstract summary of the design. While graph or network visualizations are widely adopted in the engineering design literature, the challenge remains in retrieving the design entities and deriving their relations. In this paper, we propose a network mapping method that is powered by Technology Semantic Network (TechNet). Through a case study, we showcase how TechNet's unique characteristic of being trained on a large technology-related data source advantages itself over common-sense knowledge bases, such as WordNet and ConceptNet, for design knowledge representation.
Discovering Dialog Structure Graph for Open-Domain Dialog Generation
Venue: arXiv cs.AI 2020
Year: 2020

Learning interpretable dialog structure from human-human dialogs yields basic insights into the structure of conversation, and also provides background knowledge to facilitate dialog generation. In this paper, we conduct unsupervised discovery of dialog structure from chitchat corpora, and then leverage it to facilitate dialog generation in downstream systems. To this end, we present a Discrete Variational Auto-Encoder with Graph Neural Network (DVAE-GNN), to discover a unified human-readable dialog structure. The structure is a two-layer directed graph that contains session-level semantics in the upper-layer vertices, utterance-level semantics in the lower-layer vertices, and edges among these semantic vertices. In particular, we integrate GNN into DVAE to fine-tune utterance-level semantics for more effective recognition of session-level semantic vertex. Furthermore, to alleviate the difficulty of discovering a large number of utterance-level semantics, we design a coupling mechanism that binds each utterance-level semantic vertex with a distinct phrase to provide prior semantics. Experimental results on two benchmark corpora confirm that DVAE-GNN can discover meaningful dialog structure, and the use of dialog structure graph as background knowledge can facilitate a graph grounded conversational system to conduct coherent multi-turn dialog generation.
Bibliometric analysis of world scientific production in Chemical Engineering during 2000-2011. Part 1: Analysis of total scientific production
Venue: arXiv cs.DL 2020
Year: 2020

A comprehensive bibliometric analysis of Chemical Engineering area has been carried out through the analysis of the scientific production covered in Web of Science during the period 2000-2011. Three complementary studies have been carried out. Part 1 analyzes total scientific production in the area. An important displacement of the scientific production to the Far East has occurred, mainly by the increase in publications from China (the world most productive country since 2008) but also from countries such as India and Iran. Although the share of publications from Europe, and especially from North America, have been decreased significantly, United States is still the country with the highest number of articles among the 1,000 most cited (31.5%), followed by Germany (8.4%) and China (7.5%). Switzerland, Italy, Netherlands, Singapore and Spain outstands as the countries with the highest number of cites per article and year and average impact factors of their publications. The international collaboration in the area is considerably high, especially within European countries (>40% publications are in international collaboration). The scientific production of the area is concentrated in a few journals (top 10 journals publishing around 30% publications and top 25 journals, around 50%) and publishers (Elsevier, Wiley-Blackwell, Taylor & Francis and the American Chemical Society). The countries with the highest number of institutions among the top 100 most productive are China and United States (12 each), followed by France (9), Japan (7) and United Kingdom (7). The top five institutions were from France, China, India, United States and Russia.
Bibliometric analysis of the world scientific production in Chemical Engineering during 2000-2011. Part 2: Analysis of the 1,000 most cited publications
Venue: arXiv cs.DL 2020
Year: 2020

A comprehensive bibliometric analysis of the scientific production of Chemical Engineering area has been carried out using the Web of Science database for the period 2000-2011 through three complementary studies. Part 2 demonstrated a displacement of the most cited publications to the Far East, especially due to China, however, this displacement is less important to that observed for total scientific production (Part 1). United States is still the country with the highest number of articles among the 1,000 most cited (31.5%), largely above what expected from their number of publications, followed by Germany (8.4%) and China (7.5%). The international collaboration, at least globally, seems not being an important issue for producing highly cited papers. In fact, only two from the top 25 most cited papers were international collaborations (8%). Furthermore, a large share of reviews among the 1,000 most cited papers (65%) has been observed. Although the number of institutions with more publications among the most cited in the area are from United States, the two institutions with the highest cited papers are CNRS (France) and CSIC (Spain). The most cited papers are highly concentrated in a few journals: around half of the most cited papers were published in five journals. Generally, the most cited papers are published in journals with high impact factors, however, there is also a significant number of highly cited papers published in journals with low or not having impact factor.
Bibliometric analysis of the world scientific production in Chemical Engineering during 2000-2011. Part 3: Analysis of research trends and hot topics
Venue: arXiv cs.DL 2020
Year: 2020

A comprehensive bibliometric analysis of the scientific production of Chemical Engineering area has been carried out using the Web of Science database for the period 2000-2011 through three complementary studies. Part 3 has analyzed the distribution of words in article titles, keyword plus and author keywords of both total scientific production and the 1,000 most cited publications. The main areas of Chemical Engineering have been identified; they are mainly related to chemical reaction engineering such as catalysis, reactors, kinetics, and unit operations such as adsorption. Furthermore, a total of ten hotspots in the area have been identified: hydrogen as a new energy vector, wastewater treatments, carbon dioxide (capture and sequestration), photocatalysis, nanoparticles, biodiesel, nanotubes, ionic liquids, advanced oxidation processes, membranes, fuel cells and the use of biomass as raw material (e.g. bioethanol, energy production, etc.). Results obtained suggest thematic areas and research trends can be easier analyzed through the most cited publications, decreasing significantly the time necessary for these analyses. Words in article title, keyword plus and author keywords are complementary, however, author keywords are suggested as the source of most useful data. Compared to other strategies, the author keywords are more valuable for identifying research areas and trends, and the total number of words to be analyzed is lower. Whatever the case, authors recommend a revision of the results obtained by experts in the area to avoid inaccurate results and get the most meaningful information.
Searching a Raw Video Database using Natural Language Queries
Venue: arXiv cs.LG 2020
Year: 2020

The number of videos being produced and consequently stored in databases for video streaming platforms has been increasing exponentially over time. This vast database should be easily index-able to find the requisite clip or video to match the given search specification, preferably in the form of a textual query. This work aims to provide an end-to-end pipeline to search a video database with a voice query from the end user. The pipeline makes use of Recurrent Neural Networks in combination with Convolutional Neural Networks to generate captions of the video clips present in the database.
Robust Data-Driven Error Compensation for a Battery Model
Venue: arXiv cs.LG 2020
Year: 2020

- This work has been submitted to IFAC for possible publication - Models of traction batteries are an essential tool throughout the development of automotive drivetrains. Surprisingly, today's massively collected battery data is not yet used for more accurate and reliable simulations. Primarily, the non-uniform excitation during regular battery operations prevent a consequent utilization of such measurements. Hence, there is a need for methods which enable robust models based on large datasets. For that reason, a data-driven error model is introduced enhancing an existing physically motivated model. A neural network compensates the existing dynamic error and is further limited based on a description of the underlying data. This paper tries to verify the effectiveness and robustness of the general setup and additionally evaluates a one-class support vector machine as the proposed model for the training data distribution. Based on a five datasets it is shown, that gradually limiting the data-driven error compensation outside the boundary leads to a similar improvement and an increased overall robustness.
EfficientNet-Absolute Zero for Continuous Speech Keyword Spotting
Venue: arXiv cs.SD 2020
Year: 2020

Keyword spotting is a process of finding some specific words or phrases in recorded speeches by computers. Deep neural network algorithms, as a powerful engine, can handle this problem if they are trained over an appropriate dataset. To this end, the football keyword dataset (FKD), as a new keyword spotting dataset in Persian, is collected with crowdsourcing. This dataset contains nearly 31000 samples in 18 classes. The continuous speech synthesis method proposed to made FKD usable in the practical application which works with continuous speeches. Besides, we proposed a lightweight architecture called EfficientNet-A0 (absolute zero) by applying the compound scaling method on EfficientNet-B0 for keyword spotting task. Finally, the proposed architecture is evaluated with various models. It is realized that EfficientNet-A0 and Resnet models outperform other models on this dataset.
Relational Deep Reinforcement Learning for Routing in Wireless Networks
Venue: arXiv cs.NI 2020
Year: 2020

While routing in wireless networks has been studied extensively, existing protocols are typically designed for a specific set of network conditions and so cannot accommodate any drastic changes in those conditions. For instance, protocols designed for connected networks cannot be easily applied to disconnected networks. In this paper, we develop a distributed routing strategy based on deep reinforcement learning that generalizes to diverse traffic patterns, congestion levels, network connectivity, and link dynamics. We make the following key innovations in our design: (i) the use of relational features as inputs to the deep neural network approximating the decision space, which enables our algorithm to generalize to diverse network conditions, (ii) the use of packet-centric decisions to transform the routing problem into an episodic task by viewing packets, rather than wireless devices, as reinforcement learning agents, which provides a natural way to propagate and model rewards accurately during learning, and (iii) the use of extended-time actions to model the time spent by a packet waiting in a queue, which reduces the amount of training data needed and allows the learning algorithm to converge more quickly. We evaluate our routing algorithm using a packet-level simulator and show that the policy our algorithm learns during training is able to generalize to larger and more congested networks, different topologies, and diverse link dynamics. Our algorithm outperforms shortest path and backpressure routing with respect to packets delivered and delay per packet.
Basketball Player's Value Evaluation by a Networks-based Variant Parameter Hidden Markov Model
Venue: arXiv cs.SI 2020
Year: 2020

Determining the value of basketball players through analyzing the players' behavior is important for the managers of modern basketball teams. However, conventional methods always utilize isolated statistical data, leading to ineffective and inaccurate evaluations. Existing models based on dynamic network theory offer major improvements to the results of such evaluations, but said models remain imprecise because they focus merely on evaluating the values of individual players rather than considering them within their current teams. To solve this problem, we propose an analysis and evaluation model based on networks and a hidden Markov model. To the best of our knowledge, we are the first to combine a network form representing the players who are playing with the use of a hidden Markov model to mine the network and generate the desired results. Applying our approach to SportVU data collected from the National Basketball Association shows that this analysis and evaluation model can effectively analyze the performance of each player in a game and provides an assistive tool for team managers.
Limitations of Deep Neural Networks: a discussion of G. Marcus' critical appraisal of deep learning
Venue: arXiv cs.AI 2020
Year: 2020

Deep neural networks have triggered a revolution in artificial intelligence, having been applied with great results in medical imaging, semi-autonomous vehicles, ecommerce, genetics research, speech recognition, particle physics, experimental art, economic forecasting, environmental science, industrial manufacturing, and a wide variety of applications in nearly every field. This sudden success, though, may have intoxicated the research community and blinded them to the potential pitfalls of assigning deep learning a higher status than warranted. Also, research directed at alleviating the weaknesses of deep learning may seem less attractive to scientists and engineers, who focus on the low-hanging fruit of finding more and more applications for deep learning models, thus letting short-term benefits hamper long-term scientific progress. Gary Marcus wrote a paper entitled Deep Learning: A Critical Appraisal, and here we discuss Marcus' core ideas, as well as attempt a general assessment of the subject. This study examines some of the limitations of deep neural networks, with the intention of pointing towards potential paths for future research, and of clearing up some metaphysical misconceptions, held by numerous researchers, that may misdirect them.
Promoting Graph Awareness in Linearized Graph-to-Text Generation
Venue: arXiv cs.CL 2020
Year: 2020

Generating text from structured inputs, such as meaning representations or RDF triples, has often involved the use of specialized graph-encoding neural networks. However, recent applications of pretrained transformers to linearizations of graph inputs have yielded state-of-the-art generation results on graph-to-text tasks. Here, we explore the ability of these linearized models to encode local graph structures, in particular their invariance to the graph linearization strategy and their ability to reconstruct corrupted inputs. Our findings motivate solutions to enrich the quality of models' implicit graph encodings via scaffolding. Namely, we use graph-denoising objectives implemented in a multi-task text-to-text framework. We find that these denoising scaffolds lead to substantial improvements in downstream generation in low-resource settings.
The Sample Complexity of Robust Covariance Testing
Venue: arXiv cs.LG 2020
Year: 2020

We study the problem of testing the covariance matrix of a high-dimensional Gaussian in a robust setting, where the input distribution has been corrupted in Huber's contamination model. Specifically, we are given i.i.d. samples from a distribution of the form $Z = (1-ε) X + εB$, where $X$ is a zero-mean and unknown covariance Gaussian $\mathcal{N}(0, Σ)$, $B$ is a fixed but unknown noise distribution, and $ε>0$ is an arbitrarily small constant representing the proportion of contamination. We want to distinguish between the cases that $Σ$ is the identity matrix versus $γ$-far from the identity in Frobenius norm. In the absence of contamination, prior work gave a simple tester for this hypothesis testing task that uses $O(d)$ samples. Moreover, this sample upper bound was shown to be best possible, within constant factors. Our main result is that the sample complexity of covariance testing dramatically increases in the contaminated setting. In particular, we prove a sample complexity lower bound of $Ω(d^2)$ for $ε$ an arbitrarily small constant and $γ= 1/2$. This lower bound is best possible, as $O(d^2)$ samples suffice to even robustly {\em learn} the covariance. The conceptual implication of our result is that, for the natural setting we consider, robust hypothesis testing is at least as hard as robust estimation.
Fairness in Machine Learning
Venue: arXiv cs.LG 2020
Year: 2020

Machine learning based systems are reaching society at large and in many aspects of everyday life. This phenomenon has been accompanied by concerns about the ethical issues that may arise from the adoption of these technologies. ML fairness is a recently established area of machine learning that studies how to ensure that biases in the data and model inaccuracies do not lead to models that treat individuals unfavorably on the basis of characteristics such as e.g. race, gender, disabilities, and sexual or political orientation. In this manuscript, we discuss some of the limitations present in the current reasoning about fairness and in methods that deal with it, and describe some work done by the authors to address them. More specifically, we show how causal Bayesian networks can play an important role to reason about and deal with fairness, especially in complex unfairness scenarios. We describe how optimal transport theory can be used to develop methods that impose constraints on the full shapes of distributions corresponding to different sensitive attributes, overcoming the limitation of most approaches that approximate fairness desiderata by imposing constraints on the lower order moments or other functions of those distributions. We present a unified framework that encompasses methods that can deal with different settings and fairness criteria, and that enjoys strong theoretical guarantees. We introduce an approach to learn fair representations that can generalize to unseen tasks. Finally, we describe a technique that accounts for legal restrictions about the use of sensitive attributes.
Federated Nonconvex Sparse Learning
Venue: arXiv cs.LG 2020
Year: 2020

Nonconvex sparse learning plays an essential role in many areas, such as signal processing and deep network compression. Iterative hard thresholding (IHT) methods are the state-of-the-art for nonconvex sparse learning due to their capability of recovering true support and scalability with large datasets. Theoretical analysis of IHT is currently based on centralized IID data. In realistic large-scale situations, however, data are distributed, hardly IID, and private to local edge computing devices. It is thus necessary to examine the property of IHT in federated settings, which update in parallel on local devices and communicate with a central server only once in a while without sharing local data. In this paper, we propose two IHT methods: Federated Hard Thresholding (Fed-HT) and Federated Iterative Hard Thresholding (FedIter-HT). We prove that both algorithms enjoy a linear convergence rate and have strong guarantees to recover the optimal sparse estimator, similar to traditional IHT methods, but now with decentralized non-IID data. Empirical results demonstrate that the Fed-HT and FedIter-HT outperform their competitor - a distributed IHT, in terms of decreasing the objective values with lower requirements on communication rounds and bandwidth.
Computable Random Variables and Conditioning
Venue: arXiv cs.LO 2020
Year: 2020

The aim of this paper is to present an elementary computable theory of random variables, based on the approach to probability via valuations. The theory is based on a type of lower-measurable sets, which are controlled limits of open sets, and extends existing work in this area by providing a computable theory of conditional random variables. The theory is based within the framework of type-two effectivity, so has an explicit direct link with Turing computation, and is expressed in a system of computable types and operations, so has a clean mathematical description.
Imitation Learning for High Precision Peg-in-Hole Tasks
Venue: arXiv cs.RO 2020
Year: 2020

Industrial robot manipulators are not able to match the precision and speed with which humans are able to execute contact rich tasks even to this day. Therefore, as a means overcome this gap, we demonstrate generative methods for imitating a peg-in-hole insertion task in a 6-DOF robot manipulator. In particular, generative adversarial imitation learning (GAIL) is used to successfully achieve this task with a 10 um, and a 6 um peg-hole clearance on the Yaskawa GP8 industrial robot. Experimental results show that the policy successfully learns within 20 episodes from a handful of human expert demonstrations on the robot (i.e., < 10 tele-operated robot demonstrations). The insertion time improves from > 20 seconds (which also includes failed insertions) to < 15 seconds, thereby validating the effectiveness of this approach.
Stochastic Action Prediction for Imitation Learning
Venue: arXiv cs.LG 2020
Year: 2020

Imitation learning is a data-driven approach to acquiring skills that relies on expert demonstrations to learn a policy that maps observations to actions. When performing demonstrations, experts are not always consistent and might accomplish the same task in slightly different ways. In this paper, we demonstrate inherent stochasticity in demonstrations collected for tasks including line following with a remote-controlled car and manipulation tasks including reaching, pushing, and picking and placing an object. We model stochasticity in the data distribution using autoregressive action generation, generative adversarial nets, and variational prediction and compare the performance of these approaches. We find that accounting for stochasticity in the expert data leads to substantial improvement in the success rate of task completion.
Methodology for design of templates of text communication messages for software marketing
Venue: arXiv cs.HC 2020
Year: 2020

A methodology is proposed for design of templates of text communication messages that are based on best practices of experts in software marketing, ideas of marketing, communication theory, copywriting, media linguistics, semiotics. Description of the subject area is based on conceptual modeling and production systems. For the purposes of testing, the methodology was used as the basis of a software product. Decision support recommender system for design of communication messages for software marketing.