OpenRaiser/Intern-Atlas huggingface dataset:
Towards Full-Fledged Argument Search: A Framework for Extracting and Clustering Arguments from Unstructured Text
Venue: arXiv cs.CL 2021
Year: 2021

Argument search aims at identifying arguments in natural language texts. In the past, this task has been addressed by a combination of keyword search and argument identification on the sentence- or document-level. However, existing frameworks often address only specific components of argument search and do not address the following aspects: (1) argument-query matching: identifying arguments that frame the topic slightly differently than the actual search query; (2) argument identification: identifying arguments that consist of multiple sentences; (3) argument clustering: selecting retrieved arguments by topical aspects. In this paper, we propose a framework for addressing these shortcomings. We suggest (1) to combine the keyword search with precomputed topic clusters for argument-query matching, (2) to apply a novel approach based on sentence-level sequence-labeling for argument identification, and (3) to present aggregated arguments to users based on topic-aware argument clustering. Our experiments on several real-world debate data sets demonstrate that density-based clustering algorithms, such as HDBSCAN, are particularly suitable for argument-query matching. With our sentence-level, BiLSTM-based sequence-labeling approach we achieve a macro F1 score of 0.71. Finally, evaluating our argument clustering method indicates that a fine-grained clustering of arguments by subtopics remains challenging but is worthwhile to be explored.
Adaptive Optimization with Examplewise Gradients
Venue: arXiv cs.LG 2021
Year: 2021

We propose a new, more general approach to the design of stochastic gradient-based optimization methods for machine learning. In this new framework, optimizers assume access to a batch of gradient estimates per iteration, rather than a single estimate. This better reflects the information that is actually available in typical machine learning setups. To demonstrate the usefulness of this generalized approach, we develop Eve, an adaptation of the Adam optimizer which uses examplewise gradients to obtain more accurate second-moment estimates. We provide preliminary experiments, without hyperparameter tuning, which show that the new optimizer slightly outperforms Adam on a small scale benchmark and performs the same or worse on larger scale benchmarks. Further work is needed to refine the algorithm and tune hyperparameters.
A generic physics-informed neural network-based framework for reliability assessment of multi-state systems
Venue: arXiv cs.LG 2021
Year: 2021

In this paper, we leverage the recent advances in physics-informed neural network (PINN) and develop a generic PINN-based framework to assess the reliability of multi-state systems (MSSs). The proposed methodology consists of two major steps. In the first step, we recast the reliability assessment of MSS as a machine learning problem using the framework of PINN. A feedforward neural network with two individual loss groups are constructed to encode the initial condition and state transitions governed by ordinary differential equations (ODEs) in MSS. Next, we tackle the problem of high imbalance in the magnitude of the back-propagated gradients in PINN from a multi-task learning perspective. Particularly, we treat each element in the loss function as an individual task, and adopt a gradient surgery approach named projecting conflicting gradients (PCGrad), where a task's gradient is projected onto the norm plane of any other task that has a conflicting gradient. The gradient projection operation significantly mitigates the detrimental effects caused by the gradient interference when training PINN, thus accelerating the convergence speed of PINN to high-precision solutions to MSS reliability assessment. With the proposed PINN-based framework, we investigate its applications for MSS reliability assessment in several different contexts in terms of time-independent or dependent state transitions and system scales varying from small to medium. The results demonstrate that the proposed PINN-based framework shows generic and remarkable performance in MSS reliability assessment, and the incorporation of PCGrad in PINN leads to substantial improvement in solution quality and convergence speed.
Efficient loading of reduced data ensembles produced at ORNL SNS/HFIR neutron time-of-flight facilities
Venue: arXiv cs.DB 2021
Year: 2021

We present algorithmic improvements to the loading operations of certain reduced data ensembles produced from neutron scattering experiments at Oak Ridge National Laboratory (ORNL) facilities. Ensembles from multiple measurements are required to cover a wide range of the phase space of a sample material of interest. They are stored using the standard NeXus schema on individual HDF5 files. This makes it a scalability challenge, as the number of experiments stored increases in a single ensemble file. The present work follows up on our previous efforts on data management algorithms, to address identified input output (I/O) bottlenecks in Mantid, an open-source data analysis framework used across several neutron science facilities around the world. We reuse an in-memory binary-tree metadata index that resembles data access patterns, to provide a scalable search and extraction mechanism. In addition, several memory operations are refactored and optimized for the current common use cases, ranging most frequently from 10 to 180, and up to 360 separate measurement configurations. Results from this work show consistent speed ups in wall-clock time on the Mantid LoadMD routine, ranging from 19\% to 23\% on average, on ORNL production computing systems. The latter depends on the complexity of the targeted instrument-specific data and the system I/O and compute variability for the shared computational resources available to users of ORNL's Spallation Neutron Source (SNS) and the High Flux Isotope Reactor (HFIR) instruments. Nevertheless, we continue to highlight the need for more research to address reduction challenges as experimental data volumes, user time and processing costs increase.
Concurrent Transmission for Multi-Robot Coordination
Venue: arXiv cs.RO 2021
Year: 2021

An efficient communication mechanism forms the backbone for any multi-robot system to achieve fruitful collaboration and coordination. Limitation in the existing asynchronous transmission based strategies in fast dissemination and aggregation compels the designers to prune down such requirements as much as possible. This also restricts the possible application areas of mobile multi-robot systems. In this work, we introduce concurrent transmission based strategy as an alternative. Despite the commonly found difficulties in concurrent transmission such as microsecond level time synchronization, hardware heterogeneity, etc., we demonstrate how it can be exploited for multi-robot systems. We propose a split architecture where the two major activities - communication and computation are carried out independently and coordinate through periodic interactions. The proposed split architecture is applied on a custom build full networked control system consisting of five two-wheel differential drive mobile robots having heterogeneous architecture. We use the proposed design in a leader-follower setting for coordinated dynamic speed variation as well as the independent formation of various shapes. Experiments show a centimeter-level spatial and millisecond-level temporal accuracy while spending very low radio duty-cycling over multi-hop communication under a wide testing area.
A Barrier Pair Method for Safe Human-Robot Shared Autonomy
Venue: arXiv cs.RO 2021
Year: 2021

Shared autonomy provides a framework where a human and an automated system, such as a robot, jointly control the system's behavior, enabling an effective solution for various applications, including human-robot interaction. However, a challenging problem in shared autonomy is safety because the human input may be unknown and unpredictable, which affects the robot's safety constraints. If the human input is a force applied through physical contact with the robot, it also alters the robot's behavior to maintain safety. We address the safety issue of shared autonomy in real-time applications by proposing a two-layer control framework. In the first layer, we use the history of human input measurements to infer what the human wants the robot to do and define the robot's safety constraints according to that inference. In the second layer, we formulate a rapidly-exploring random tree of barrier pairs, with each barrier pair composed of a barrier function and a controller. Using the controllers in these barrier pairs, the robot is able to maintain its safe operation under the intervention from the human input. This proposed control framework allows the robot to assist the human while preventing them from encountering safety issues. We demonstrate the proposed control framework on a simulation of a two-linkage manipulator robot.
FDA-GAN: Flow-based Dual Attention GAN for Human Pose Transfer
Venue: arXiv cs.CV 2021
Year: 2021

Human pose transfer aims at transferring the appearance of the source person to the target pose. Existing methods utilizing flow-based warping for non-rigid human image generation have achieved great success. However, they fail to preserve the appearance details in synthesized images since the spatial correlation between the source and target is not fully exploited. To this end, we propose the Flow-based Dual Attention GAN (FDA-GAN) to apply occlusion- and deformation-aware feature fusion for higher generation quality. Specifically, deformable local attention and flow similarity attention, constituting the dual attention mechanism, can derive the output features responsible for deformable- and occlusion-aware fusion, respectively. Besides, to maintain the pose and global position consistency in transferring, we design a pose normalization network for learning adaptive normalization from the target pose to the source person. Both qualitative and quantitative results show that our method outperforms state-of-the-art models in public iPER and DeepFashion datasets.
STAR-RISs: A Correlated T&R Phase-Shift Model and Practical Phase-Shift Configuration Strategies
Venue: arXiv cs.IT 2021
Year: 2021

A correlated transmission and reflection (T&R) phase-shift model is proposed for passive lossless simultaneously transmitting and reflecting reconfigurable intelligent surfaces (STAR-RISs). A STAR-RIS-aided two-user downlink communication system is investigated for both orthogonal multiple access (OMA) and non-orthogonal multiple access (NOMA). To evaluate the impact of the correlated T&R phase-shift model on the communication performance, three phase-shift configuration strategies are developed, namely the primary-secondary phase-shift configuration (PS-PSC), the diversity preserving phase-shift configuration (DP-PSC), and the T/R-group phase-shift configuration (TR-PSC) strategies. Furthermore, we derive the outage probabilities for the three proposed phase-shift configuration strategies as well as for those of the random phase-shift configuration and the independent phase-shift model, which constitute performance lower and upper bounds, respectively. Then, the diversity order of each strategy is investigated based on the obtained analytical results. It is shown that the proposed DP-PSC strategy achieves full diversity order simultaneously for users located on both sides of the STAR-RIS. Moreover, power scaling laws are derived for the three proposed strategies and for the random phase-shift configuration. Numerical simulations reveal a performance gain if the users on both sides of the STAR-RIS are served by NOMA instead of OMA. Moreover, it is shown that the proposed DP-PSC strategy yields the same diversity order as achieved by STAR-RISs under the independent phase-shift model and a comparable power scaling law with only 4 dB reduction in received power.
Optimizing for In-memory Deep Learning with Emerging Memory Technology
Venue: arXiv cs.LG 2021
Year: 2021

In-memory deep learning computes neural network models where they are stored, thus avoiding long distance communication between memory and computation units, resulting in considerable savings in energy and time. In-memory deep learning has already demonstrated orders of magnitude higher performance density and energy efficiency. The use of emerging memory technology promises to increase the gains in density, energy, and performance even further. However, emerging memory technology is intrinsically unstable, resulting in random fluctuations of data reads. This can translate to non-negligible accuracy loss, potentially nullifying the gains. In this paper, we propose three optimization techniques that can mathematically overcome the instability problem of emerging memory technology. They can improve the accuracy of the in-memory deep learning model while maximizing its energy efficiency. Experiments show that our solution can fully recover most models' state-of-the-art accuracy, and achieves at least an order of magnitude higher energy efficiency than the state-of-the-art.
A Daily Tourism Demand Prediction Framework Based on Multi-head Attention CNN: The Case of The Foreign Entrant in South Korea
Venue: arXiv cs.LG 2021
Year: 2021

Developing an accurate tourism forecasting model is essential for making desirable policy decisions for tourism management. Early studies on tourism management focus on discovering external factors related to tourism demand. Recent studies utilize deep learning in demand forecasting along with these external factors. They mainly use recursive neural network models such as LSTM and RNN for their frameworks. However, these models are not suitable for use in forecasting tourism demand. This is because tourism demand is strongly affected by changes in various external factors, and recursive neural network models have limitations in handling these multivariate inputs. We propose a multi-head attention CNN model (MHAC) for addressing these limitations. The MHAC uses 1D-convolutional neural network to analyze temporal patterns and the attention mechanism to reflect correlations between input variables. This model makes it possible to extract spatiotemporal characteristics from time-series data of various variables. We apply our forecasting framework to predict inbound tourist changes in South Korea by considering external factors such as politics, disease, season, and attraction of Korean culture. The performance results of extensive experiments show that our method outperforms other deep-learning-based prediction frameworks in South Korea tourism forecasting.
Trusted And Confidential Program Analysis
Venue: arXiv cs.CR 2021
Year: 2021

We develop the concept of Trusted and Confidential Program Analysis (TCPA) which enables program certification to be used where previously there was insufficient trust. Imagine a scenario where a producer may not be trusted to certify its own software (perhaps by a foreign regulator), and the producer is unwilling to release its sources and detailed design to any external body. We present a protocol that can, using trusted computing based on encrypted sources, create certification via which all can trust the delivered object code without revealing the unencrypted sources to any party. Furthermore, we describe a realization of TCPA with trusted execution environments (TEE) that enables general and efficient computation. We have implemented the TCPA protocol in a system called TCWasm for web assembly architectures. In our evaluation with 33 benchmark cases, TCWasm managed to finish the analysis with relatively slight overheads.
Investigation of Training Label Error Impact on RNN-T
Venue: arXiv cs.CL 2021
Year: 2021

In this paper, we propose an approach to quantitatively analyze impacts of different training label errors to RNN-T based ASR models. The result shows deletion errors are more harmful than substitution and insertion label errors in RNN-T training data. We also examined label error impact mitigation approaches on RNN-T and found that, though all the methods mitigate the label-error-caused degradation to some extent, they could not remove the performance gap between the models trained with and without the presence of label errors. Based on the analysis results, we suggest to design data pipelines for RNN-T with higher priority on reducing deletion label errors. We also find that ensuring high-quality training labels remains important, despite of the existence of the label error mitigation approaches.
A Comprehensive Study on Various Statistical Techniques for Prediction of Movie Success
Venue: arXiv cs.LG 2021
Year: 2021

The film industry is one of the most popular entertainment industries and one of the biggest markets for business. Among the contributing factors to this would be the success of a movie in terms of its popularity as well as its box office performance. Hence, we create a comprehensive comparison between the various machine learning models to predict the rate of success of a movie. The effectiveness of these models along with their statistical significance is studied to conclude which of these models is the best predictor. Some insights regarding factors that affect the success of the movies are also found. The models studied include some Regression models, Machine Learning models, a Time Series model and a Neural Network with the Neural Network being the best performing model with an accuracy of about 86%. Additionally, as part of the testing data for the movies released in 2020 are analysed.
How to use Persistent Memory in your Database
Venue: arXiv cs.DB 2021
Year: 2021

Persistent or Non Volatile Memory (PMEM or NVM) has recently become commercially available under several configurations with different purposes and goals. Despite the attention to the topic, we are not aware of a comprehensive empirical analysis of existing relational database engines under different PMEM configurations. Such a study is important to understand the performance implications of the various hardware configurations and how different DB engines can benefit from them. To this end, we analyze three different engines (PostgreSQL, MySQL, and SQLServer) under common workloads (TPC-C and TPC-H) with all possible PMEM configurations supported by Intel's Optane NVM devices (PMEM as persistent memory in AppDirect mode and PMEM as volatile memory in Memory mode). Our results paint a complex picture and are not always intuitive due to the many factors involved. Based on our findings, we provide insights on how the different engines behave with PMEM and which configurations and queries perform best. Our results show that using PMEM as persistent storage usually speeds up query execution, but with some caveats as the I/O path is not fully optimized. Additionally, using PMEM in Memory mode does not offer any performance advantage despite the larger volatile memory capacity. Through the extensive coverage of engines and parameters, we provide an important starting point for exploiting PMEM in databases and tuning relational engines to take advantage of this new technology.
A benchmark with decomposed distribution shifts for 360 monocular depth estimation
Venue: arXiv cs.CV 2021
Year: 2021

In this work we contribute a distribution shift benchmark for a computer vision task; monocular depth estimation. Our differentiation is the decomposition of the wider distribution shift of uncontrolled testing on in-the-wild data, to three distinct distribution shifts. Specifically, we generate data via synthesis and analyze them to produce covariate (color input), prior (depth output) and concept (their relationship) distribution shifts. We also synthesize combinations and show how each one is indeed a different challenge to address, as stacking them produces increased performance drops and cannot be addressed horizontally using standard approaches.
Frequent-Pattern Based Broadcast Scheduling for Conflict Avoidance in Multi-Channel Data Dissemination Systems
Venue: arXiv cs.NI 2021
Year: 2021

With the popularity of mobile devices, using the traditional client-server model to handle a large number of requests is very challenging. Wireless data broadcasting can be used to provide services to many users at the same time, so reducing the average access time has become a popular research topic. For example, some location-based services (LBS) consider using multiple channels to disseminate information to reduce access time. However, data conflicts may occur when multiple channels are used, where multiple data items associated with the request are broadcast at about the same time. In this article, we consider the channel switching time and identify the data conflict issue in an on-demand multi-channel dissemination system. We model the considered problem as a Data Broadcast with Conflict Avoidance (DBCA) problem and prove it is NP-complete. We hence propose the frequent-pattern based broadcast scheduling (FPBS), which provides a new variant of the frequent pattern tree, FP*-tree, to schedule the requested data. Using FPBS, the system can avoid data conflicts when assigning data items to time slots in the channels. In the simulation, we discussed two modes of FPBS: online and offline. The results show that, compared with the existing heuristic methods, FPBS can shorten the average access time by 30%.
'Entanglement' -- A new dynamic metric to measure team flow
Venue: arXiv cs.SI 2021
Year: 2021

We introduce "entanglement", a novel metric to measure how synchronized communication between team members is. This measure calculates the Euclidean distance among team members' social network metrics timeseries. We validate the metric with four case studies. The first case study uses entanglement of 11 medical innovation teams to predict team performance and learning behavior. The second case looks at the e-mail communication of 113 senior executives of an international services firm, predicting employee turnover through lack of entanglement of an employee. The third case analyzes the individual employee performance of 81 managers. The fourth case study predicts performance of 13 customer-dedicated teams at a big international company by comparing entanglement in the e-mail interactions with satisfaction of their customers measured through Net Promoter Score (NPS). While we can only speculate about what is causing the entanglement effect, we find that it is a new and versatile indicator for the analysis of employees' communication, analyzing the hitherto underused temporal dimension of online social networks which could be used as a powerful predictor of employee and team performance, employee turnover, and customer satisfaction.
DPRK-BERT: The Supreme Language Model
Venue: arXiv cs.CL 2021
Year: 2021

Deep language models have achieved remarkable success in the NLP domain. The standard way to train a deep language model is to employ unsupervised learning from scratch on a large unlabeled corpus. However, such large corpora are only available for widely-adopted and high-resource languages and domains. This study presents the first deep language model, DPRK-BERT, for the DPRK language. We achieve this by compiling the first unlabeled corpus for the DPRK language and fine-tuning a preexisting the ROK language model. We compare the proposed model with existing approaches and show significant improvements on two DPRK datasets. We also present a cross-lingual version of this model which yields better generalization across the two Korean languages. Finally, we provide various NLP tools related to the DPRK language that would foster future research.