pwc-archive/papers-with-abstracts huggingface dataset:
ASBERT: Siamese and Triplet network embedding for open question answering
Authors: Olabanji Shonibare
Date: 2021-04-17 00:00:00
Tasks: Answer Selection, Network Embedding, Open-Ended Question Answering, Question Answering, Sentence, Triplet

Answer selection (AS) is an essential subtask in the field of natural language processing with an objective to identify the most likely answer to a given question from a corpus containing candidate answer sentences. A common approach to address the AS problem is to generate an embedding for each candidate sentence and query. Then, select the sentence whose vector representation is closest to the query's. A key drawback is the low quality of the embeddings, hitherto, based on its performance on AS benchmark datasets. In this work, we present ASBERT, a framework built on the BERT architecture that employs Siamese and Triplet neural networks to learn an encoding function that maps a text to a fixed-size vector in an embedded space. The notion of distance between two points in this space connotes similarity in meaning between two texts. Experimental results on the WikiQA and TrecQA datasets demonstrate that our proposed approach outperforms many state-of-the-art baseline methods.
Co-BERT: A Context-Aware BERT Retrieval Model Incorporating Local and Query-specific Context
Authors: Xiaoyang Chen, Kai Hui, Ben He, Xianpei Han, Le Sun, Zheng Ye
Date: 2021-04-17 00:00:00
Tasks: Learning-To-Rank, Re-Ranking, Retrieval

BERT-based text ranking models have dramatically advanced the state-of-the-art in ad-hoc retrieval, wherein most models tend to consider individual query-document pairs independently. In the mean time, the importance and usefulness to consider the cross-documents interactions and the query-specific characteristics in a ranking model have been repeatedly confirmed, mostly in the context of learning to rank. The BERT-based ranking model, however, has not been able to fully incorporate these two types of ranking context, thereby ignoring the inter-document relationships from the ranking and the differences among queries. To mitigate this gap, in this work, an end-to-end transformer-based ranking model, named Co-BERT, has been proposed to exploit several BERT architectures to calibrate the query-document representations using pseudo relevance feedback before modeling the relevance of a group of documents jointly. Extensive experiments on two standard test collections confirm the effectiveness of the proposed model in improving the performance of text re-ranking over strong fine-tuned BERT-Base baselines. We plan to make our implementation open source to enable further comparisons.
Using Machine Learning at Scale in HPC Simulations with SmartSim: An Application to Ocean Climate Modeling
Authors: Sam Partee, Matthew Ellis, Alessandro Rigazzi, Scott Bachman, Gustavo Marques, Andrew Shao, Benjamin Robbins
Date: 2021-04-13 00:00:00
Tasks: BIG-bench Machine Learning

We demonstrate the first climate-scale, numerical ocean simulations improved through distributed, online inference of Deep Neural Networks (DNN) using SmartSim. SmartSim is a library dedicated to enabling online analysis and Machine Learning (ML) for traditional HPC simulations. In this paper, we detail the SmartSim architecture and provide benchmarks including online inference with a shared ML model on heterogeneous HPC systems. We demonstrate the capability of SmartSim by using it to run a 12-member ensemble of global-scale, high-resolution ocean simulations, each spanning 19 compute nodes, all communicating with the same ML architecture at each simulation timestep. In total, 970 billion inferences are collectively served by running the ensemble for a total of 120 simulated years. Finally, we show our solution is stable over the full duration of the model integrations, and that the inclusion of machine learning has minimal impact on the simulation runtimes.
Grid Monitoring for Efficient Flexibility Provision in Distribution Grids
Authors: Ankur Majumdar, Sotirios Dimitrakopoulos, Omid Alizadeh-Mousavi
Date: 2021-04-18 00:00:00

The increased flexibility requirement needs a flexibility market at the distribution grid level operated by the distribution system operators (DSOs) to resolve challenges to ensure secure operation and, to integrate new renewable productions or loads in the grid. Therefore, the network visibility and monitoring are paramount to distribution network operation. This paper presents a methodology demonstrating the value distribution grid monitoring can bring for the realisation of a local flexibility market. This paper further illustrates the reduction of costs of operation of a DSO with the help of grid monitoring and local flexibility market while maintaining a secure and reliable operation. The methodology has been applied on a real 35 node MV network of a Swiss DSO with several GridEye measurement devices. The performance in terms of losses and voltage and flow violation costs is compared with sub-optimal operation without monitoring.
Foreign exchange markets: price response and spread impact
Authors: Juan Camilo Henao Londono, Thomas Guhr
Date: 2021-04-15 00:00:00

We carry out a detailed large-scale data analysis of price response functions in the spot foreign exchange market for different years and different time scales. Such response functions provide quantitative information on the deviation from Markovian behavior. The price response functions show an increase to a maximum followed by a slow decrease as the time lag grows, in trade time scale and in physical time scale, for all analyzed years. Furthermore, we use a price increment point (pip) bid-ask spread definition to group different foreign exchange pairs and analyze the impact of the spread in the price response functions. We find that large pip spreads have a stronger impact on the response. This is similar to what has been found in stock markets.
EdgeCRNN: an edgecomputing oriented model of acoustic feature enhancement for keyword spotting
Authors: Yungen Wei, Zheng Gong, Shunzhi Yang, Kai Ye, Yamin Wen
Date: 2021-03-14 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Edge-computing, Keyword Spotting, speech-recognition, Speech Recognition

Keyword Spotting (KWS) is a significant branch of Automatic Speech Recognition (ASR) and has been widely used in edge computing devices. The goal of KWS is to provide high accuracy with a low False Alarm Rate (FAR), while reducing the costs of memory, computation, and latency. However, limited resources are challenging for KWS applications on edge computing devices. Lightweight models and structures for deep learning have achieved good results in the KWS branch while maintaining efficient performances. In this paper, we present a new Convolutional Recurrent Neural Network (CRNN) architecture named EdgeCRNN for edge computing devices. EdgeCRNN, which is based on depthwise separable convolution and residual structure, uses a feature enhanced method. On the Google Speech Commands Dataset, the experimental results depict that EdgeCRNN can test 11.1 audio data per second on Raspberry Pi 3B+, which is 2.2 times than that of Tpool2. Compared with Tpool2, the accuracy of EdgeCRNN reaches 98.05% whilst its performance is also competitive.
FPGA Implementations of Layered MinSum LDPC Decoders Using RCQ Message Passing
Authors: Caleb Terrill, Linfang Wang, Sean Chen, Chester Hulse, Calvin Kuo, Richard Wesel, Dariush Divsalar
Date: 2021-04-19 00:00:00
Tasks: Decoder, Quantization

Non-uniform message quantization techniques such as reconstruction-computation-quantization (RCQ) improve error-correction performance and decrease hardware complexity of low-density parity-check (LDPC) decoders that use a flooding schedule. Layered MinSum RCQ (L-msRCQ) enables message quantization to be utilized for layered decoders and irregular LDPC codes. We investigate field-programmable gate array (FPGA) implementations of L-msRCQ decoders. Three design methods for message quantization are presented, which we name the Lookup, Broadcast, and Dribble methods. The decoding performance and hardware complexity of these schemes are compared to a layered offset MinSum (OMS) decoder. Simulation results on a (16384, 8192) protograph-based raptor-like (PBRL) LDPC code show that a 4-bit L-msRCQ decoder using the Broadcast method can achieve a 0.03 dB improvement in error-correction performance while using 12% fewer registers than the OMS decoder. A Broadcast-based 3-bit L-msRCQ decoder uses 15% fewer lookup tables, 18% fewer registers, and 13% fewer routed nets than the OMS decoder, but results in a 0.09 dB loss in performance.
Demystifying BERT: Implications for Accelerator Design
Authors: Suchita Pati, Shaizeen Aga, Nuwan Jayasena, Matthew D. Sinclair
Date: 2021-04-14 00:00:00
Tasks: Transfer Learning

Transfer learning in natural language processing (NLP), as realized using models like BERT (Bi-directional Encoder Representation from Transformer), has significantly improved language representation with models that can tackle challenging language problems. Consequently, these applications are driving the requirements of future systems. Thus, we focus on BERT, one of the most popular NLP transfer learning algorithms, to identify how its algorithmic behavior can guide future accelerator design. To this end, we carefully profile BERT training and identify key algorithmic behaviors which are worthy of attention in accelerator design. We observe that while computations which manifest as matrix multiplication dominate BERT's overall runtime, as in many convolutional neural networks, memory-intensive computations also feature prominently. We characterize these computations, which have received little attention so far. Further, we also identify heterogeneity in compute-intensive BERT computations and discuss software and possible hardware mechanisms to further optimize these computations. Finally, we discuss implications of these behaviors as networks get larger and use distributed training environments, and how techniques such as micro-batching and mixed-precision training scale. Overall, our analysis identifies holistic solutions to optimize systems for BERT-like models.
SuperMeshing: A New Deep Learning Architecture for Increasing the Mesh Density of Metal Forming Stress Field with Attention Mechanism and Perceptual Features
Authors: Qingfeng Xu, Zhenguo Nie, Handing Xu, Haosu Zhou, Xinjun Liu
Date: 2021-03-12 00:00:00

In stress field analysis, the finite element analysis is a crucial approach, in which the mesh-density has a significant impact on the results. High mesh density usually contributes authentic to simulation results but costs more computing resources, leading to curtailing efficiency during the design process. To eliminate this drawback, we propose a new data-driven mesh-density boost model named SuperMeshingNet that strengthens the advantages of finite element analysis (FEA) with low mesh-density as inputs to the deep learning model, which consisting of Res-UNet architecture, to acquire high-density stress field instantaneously, shortening computing time and cost automatically. Moreover, the attention mechanism and the perceptual features are utilized, enhancing the performance of SuperMeshingNet. Compared to the baseline that applied the linear interpolation method, SuperMeshingNet achieves a prominent reduction in the mean squared error (MSE) and mean absolute error (MAE) on test data, which contains prior unseen cases. Based on the data set of metal forming, the comparable experiments are proceeded to demonstrate the high quality and superior precision of the reconstructed results generated by our model. The well-trained model can successfully show more excellent performance than the baseline and other methods on the multiple scaled mesh-density, including $2\times$, $4\times$, and $8\times$. With the refined result owning broaden scaling of mesh density and high precision, the FEA process can be accelerated with seldom cost on computation resources. We publicly share our work with full detail of implementation at https://github.com/zhenguonie/2021_SuperMeshing_2D_Metal_Forming
Comprehensive systematic review into combinations of artificial intelligence, human factors, and automation
Authors: Reza Khani-Shekarab, Alireza khani-shekarab
Date: 2021-04-09 00:00:00
Tasks: Articles

Artificial intelligence (AI)-based models used to improve different fields including healthcare, and finance. One of the field that receive advantages of AI is automation. However, it is important to consider human factors in application of AI in automation. This paper reports on a systematic review of the published studies used to investigate the application of AI in PM. This comprehensive systematic review used ScienceDirect to identify relevant articles. Of the 422 articles found, 40 met the inclusion and exclusion criteria and were used in the review. Selected articles were classified based on categories of human factors and areas of application. The results indicated that application of AI in automation with respect to human factors could be divided into three areas of physical ergonomics, cognitive ergonomic and organizational ergonomics. The main areas of application in physical and cognitive ergonomics are including transportation, User experience, and human-machine interactions.
Visual analytics of set data for knowledge discovery and member selection support
Authors: Ryuji Watanabe, Hideaki Ishibashi, Tetsuo Furukawa
Date: 2021-04-04 00:00:00

Visual analytics (VA) is a visually assisted exploratory analysis approach in which knowledge discovery is executed interactively between the user and system in a human-centered manner. The purpose of this study is to develop a method for the VA of set data aimed at supporting knowledge discovery and member selection. A typical target application is a visual support system for team analysis and member selection, by which users can analyze past teams and examine candidate lineups for new teams. Because there are several difficulties, such as the combinatorial explosion problem, developing a VA system of set data is challenging. In this study, we first define the requirements that the target system should satisfy and clarify the accompanying challenges. Then we propose a method for the VA of set data, which satisfies the requirements. The key idea is to model the generation process of sets and their outputs using a manifold network model. The proposed method visualizes the relevant factors as a set of topographic maps on which various information is visualized. Furthermore, using the topographic maps as a bidirectional interface, users can indicate their targets of interest in the system on these maps. We demonstrate the proposed method by applying it to basketball teams, and compare with a benchmark system for outcome prediction and lineup reconstruction tasks. Because the method can be adapted to individual application cases by extending the network structure, it can be a general method by which practical systems can be built.
My Experience in Physical Layer Communications
Authors: Xiang-Gen Xia
Date: 2021-04-19 00:00:00

I feel that I have been very lucky since I have experienced the most dynamic 30 years on electronics in the past. I think that the most visible change in our daily life over the past 30 years is communications. From computer modems, to internet, and to smart phones, people now feel much less lonely or bored since they are always connected. In this article, I would like to share with you on my own experience working on communications in the past decades.
Aiding Long-Term Investment Decisions with XGBoost Machine Learning Model
Authors: Ekaterina Zolotareva
Date: 2021-04-19 00:00:00
Tasks: BIG-bench Machine Learning, Feature Engineering, Time Series, Time Series Analysis

The ability to identify stock market trends has obvious advantages for investors. Buying stock on an upward trend (as well as selling it in case of downward movement) results in profit. Accordingly, the start and end-points of the trend are the optimal points for entering and leaving the market. The research concentrates on recognizing stock market long-term upward and downward trends. The key results are obtained with the use of gradient boosting algorithms, XGBoost in particular. The raw data is represented by time series with basic stock market quotes with periods labelled by experts as Trend or Flat. The features are then obtained via various data transformations, aiming to catch implicit factors resulting in a change of stock direction. Modelling is done in two stages: stage one aims to detect endpoints of tendencies (i.e. sliding windows), stage two recognizes the tendency itself inside the window. The research addresses such issues as imbalanced datasets and contradicting labels, as well as the need for specific quality metrics to keep up with practical applicability. The model can be used to design an investment strategy though further research in feature engineering and fine calibration is required.This paper is the full text of the research, presented at the 20th International Conference on Artificial Intelligence and Soft Computing Web System (ICAISC 2021)
Energy Efficiency Optimization in Radar-Communication Spectrum Sharing
Authors: Emanuele Grossi, Marco Lops, Luca Venturino
Date: 2021-04-19 00:00:00

Energy efficiency, possibly coupled with cognition-based and spectrum-sharing architectures, is a key enabling technology for green communications in 5G-and-beyond standards. In this context, the present paper considers a multiple-input multiple-output communication system cooperatively coexisting with a surveillance radar: the objective function is the communication system energy efficiency, while radar operation is safeguarded by constraining the minimum received signal-to-disturbance ratio for a set of range-azimuth cells of the controlled scene, and no time synchronization between them is assumed. The degrees of freedom are the transmit powers of both systems, the space-time communication codebook and the linear filters at the radar receiver. The resulting optimization problem is non-convex, due to both the objective function and the presence of signal-dependent interference (clutter): we develop a block-coordinate-ascent approximate solution, and offer a thorough performance assessment, so as to elicit the merits of the proposed approach, along with the interplay among the achievable energy efficiency, the density of scatterers in the environment, and the size of the set of protected radar cells.
Fusing information streams in end-to-end audio-visual speech recognition
Authors: Wentao Yu, Steffen Zeiler, Dorothea Kolossa
Date: 2021-04-19 00:00:00
Tasks: Audio-Visual Speech Recognition, Lip Reading, speech-recognition, Speech Recognition, Visual Speech Recognition

End-to-end acoustic speech recognition has quickly gained widespread popularity and shows promising results in many studies. Specifically the joint transformer/CTC model provides very good performance in many tasks. However, under noisy and distorted conditions, the performance still degrades notably. While audio-visual speech recognition can significantly improve the recognition rate of end-to-end models in such poor conditions, it is not obvious how to best utilize any available information on acoustic and visual signal quality and reliability in these models. We thus consider the question of how to optimally inform the transformer/CTC model of any time-variant reliability of the acoustic and visual information streams. We propose a new fusion strategy, incorporating reliability information in a decision fusion net that considers the temporal effects of the attention mechanism. This approach yields significant improvements compared to a state-of-the-art baseline model on the Lip Reading Sentences 2 and 3 (LRS2 and LRS3) corpus. On average, the new system achieves a relative word error rate reduction of 43% compared to the audio-only setup and 31% compared to the audiovisual end-to-end baseline.
Self-supervised Representation Learning With Path Integral Clustering For Speaker Diarization
Authors: Prachi Singh, Sriram Ganapathy
Date: 2021-04-19 00:00:00
Tasks: Clustering, Representation Learning, Self-Supervised Learning, speaker-diarization, Speaker Diarization

Automatic speaker diarization techniques typically involve a two-stage processing approach where audio segments of fixed duration are converted to vector representations in the first stage. This is followed by an unsupervised clustering of the representations in the second stage. In most of the prior approaches, these two stages are performed in an isolated manner with independent optimization steps. In this paper, we propose a representation learning and clustering algorithm that can be iteratively performed for improved speaker diarization. The representation learning is based on principles of self-supervised learning while the clustering algorithm is a graph structural method based on path integral clustering (PIC). The representation learning step uses the cluster targets from PIC and the clustering step is performed on embeddings learned from the self-supervised deep model. This iterative approach is referred to as self-supervised clustering (SSC). The diarization experiments are performed on CALLHOME and AMI meeting datasets. In these experiments, we show that the SSC algorithm improves significantly over the baseline system (relative improvements of 13% and 59% on CALLHOME and AMI datasets respectively in terms of diarization error rate (DER)). In addition, the DER results reported in this work improve over several other recent approaches for speaker diarization.
Automatic Stroke Classification of Tabla Accompaniment in Hindustani Vocal Concert Audio
Authors: Rohit M. A., Preeti Rao
Date: 2021-04-19 00:00:00
Tasks: Data Augmentation, General Classification, Stroke Classification

The tabla is a unique percussion instrument due to the combined harmonic and percussive nature of its timbre, and the contrasting harmonic frequency ranges of its two drums. This allows a tabla player to uniquely emphasize parts of the rhythmic cycle (theka) in order to mark the salient positions. An analysis of the loudness dynamics and timing deviations at various cycle positions is an important part of musicological studies on the expressivity in tabla accompaniment. To achieve this at a corpus-level, and not restrict it to the few recordings that manual annotation can afford, it is helpful to have access to an automatic tabla transcription system. Although a few systems have been built by training models on labeled tabla strokes, the achieved accuracy does not necessarily carry over to unseen instruments. In this article, we report our work towards building an instrument-independent stroke classification system for accompaniment tabla based on the more easily available tabla solo audio tracks. We present acoustic features that capture the distinctive characteristics of tabla strokes and build an automatic system to predict the label as one of a reduced, but musicologically motivated, target set of four stroke categories. To address the lack of sufficient labeled training data, we turn to common data augmentation methods and find the use of pitch-shifting based augmentation to be most promising. We then analyse the important features and highlight the problem of their instrument-dependence while motivating the use of more task-specific data augmentation strategies to improve the diversity of training data.