pwc-archive/papers-with-abstracts huggingface dataset:
MATCH: Metadata-Aware Text Classification in A Large Hierarchy
Authors: Yu Zhang, Zhihong Shen, Yuxiao Dong, Kuansan Wang, Jiawei Han
Date: 2021-02-15 00:00:00
Tasks: Classification, General Classification, Multi Label Text Classification, Multi-Label Text Classification, text-classification, Text Classification

Multi-label text classification refers to the problem of assigning each given document its most relevant labels from the label set. Commonly, the metadata of the given documents and the hierarchy of the labels are available in real-world applications. However, most existing studies focus on only modeling the text information, with a few attempts to utilize either metadata or hierarchy signals, but not both of them. In this paper, we bridge the gap by formalizing the problem of metadata-aware text classification in a large label hierarchy (e.g., with tens of thousands of labels). To address this problem, we present the MATCH solution -- an end-to-end framework that leverages both metadata and hierarchy information. To incorporate metadata, we pre-train the embeddings of text and metadata in the same space and also leverage the fully-connected attentions to capture the interrelations between them. To leverage the label hierarchy, we propose different ways to regularize the parameters and output probability of each child label by its parents. Extensive experiments on two massive text datasets with large-scale label hierarchies demonstrate the effectiveness of MATCH over state-of-the-art deep learning baselines.
ScrofaZero: Mastering Trick-taking Poker Game Gongzhu by Deep Reinforcement Learning
Authors: Naichen Shi, Ruichen Li, Sun Youran
Date: 2021-02-15 00:00:00
Tasks: Bayesian Inference, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning (RL)

People have made remarkable progress in game AIs, especially in domain of perfect information game. However, trick-taking poker game, as a popular form of imperfect information game, has been regarded as a challenge for a long time. Since trick-taking game requires high level of not only reasoning, but also inference to excel, it can be a new milestone for imperfect information game AI. We study Gongzhu, a trick-taking game analogous to, but slightly simpler than contract bridge. Nonetheless, the strategies of Gongzhu are complex enough for both human and computer players. We train a strong Gongzhu AI ScrofaZero from \textit{tabula rasa} by deep reinforcement learning, while few previous efforts on solving trick-taking poker game utilize the representation power of neural networks. Also, we introduce new techniques for imperfect information game including stratified sampling, importance weighting, integral over equivalent class, Bayesian inference, etc. Our AI can achieve human expert level performance. The methodologies in building our program can be easily transferred into a wide range of trick-taking games.
Learning low-rank latent mesoscale structures in networks
Authors: Hanbaek Lyu, Yacoub H. Kureh, Joshua Vendrow, Mason A. Porter
Date: 2021-02-13 00:00:00
Tasks: Denoising, Dictionary Learning

It is common to use networks to encode the architecture of interactions between entities in complex systems in the physical, biological, social, and information sciences. To study the large-scale behavior of complex systems, it is useful to examine mesoscale structures in networks as building blocks that influence such behavior. We present a new approach for describing low-rank mesoscale structures in networks, and we illustrate our approach using several synthetic network models and empirical friendship, collaboration, and protein--protein interaction (PPI) networks. We find that these networks possess a relatively small number of `latent motifs' that together can successfully approximate most subgraphs of a network at a fixed mesoscale. We use an algorithm for `network dictionary learning' (NDL), which combines a network-sampling method and nonnegative matrix factorization, to learn the latent motifs of a given network. The ability to encode a network using a set of latent motifs has a wide variety of applications to network-analysis tasks, such as comparison, denoising, and edge inference. Additionally, using a new network denoising and reconstruction (NDR) algorithm, we demonstrate how to denoise a corrupted network by using only the latent motifs that one learns directly from the corrupted network.
科学博士youtube频道
Authors: 科学博士
Date: 2021-02-01 00:00:00

科学博士youtube频道
宗旨：科学博士，让我们拥抱科学的大脑！
关键词：科学博士，科学知识，科普，思维，智慧，科学实验，趣味科学，北京大学
频道链接：https://www.youtube.com/channel/UCrck45T23NMzEBn_y8QykRA
A Latent Topic Modeling approach for Subject Summarization of Research on the Military Art and Science in South Korea
Authors: S Bae, X Ku, C Park, J Kim
Date: 2020-06-01 00:00:00
Tasks: Articles, General Classification, LEMMA, Management, Sentence, Sentence segmentation

Within the military art and science research articles, many of studies have focused on the empirical examination or theoretically review interests related to military phenomena (e.g., military power, security relations). Furthermore, because the technological and social change in recent years have spurred convergence and generalization research in Korean military art and science, multidisciplinary research related to science and technology and various disciplines is emphasized as well as social science that studies military issues. In this sense, to maintain sustainable development of current research, it is necessary to explore specific topics on academic studies. To accomplish this purpose, we conducted a sequence of three analytic stages First, we selected a web database in KCI OAI-PMH (Korea Citation Index Open Archives Initiative Protocol for Metadata Harvesting) for literature search. And then we performed subsequently snowballing sampling via author affiliation and related-keywords (i.e., military, defense, weapon etc.) of reference list based on initial DB search to find comprehensive articles from relevant studies. It comprised data sets including English Abstract from a total of 4,193 studies (314 journals) during 2002-2019. Second, using these data sets, we extracted token, lemma, and morphological features of potentially useful NOUN by employing Universal Dependencies (UD) pipeline for joint sentence segmentation, word segmentation. Third, based on the topic modeling using topicmodels, OpTop, and topicdoc package with Latent Dirichlet Allocation (LDA) algorithms on this corpus, we presented new subject classification including ten topics (i.e., Defense Reform, security alliance, defense industry, defense R&D, combat simulation, dynamic analysis of weapon systems, reliability evaluation of weapon system, target detection, characteristics analysis of materials or performance). As a result, the current study explored latent topic cluster (subject classification) was divided into 1) weapon system acquisition and management (military force building and intangible force maintenance) and 2) defense R&D (military force operation, military force development, tangible force maintenance) based on the topic network analysis. These findings are preliminary, but it enhances our understanding of the existing sub-subject areas by extending subject classification for military art and science in the Knowledge Classification Scheme of National Research Foundations of Korea.
New methods for metastimuli: architecture, embeddings, and neural network optimization
Authors: Rico A. R. Picone, Dane Webb, Finbarr Obierefu, Jotham Lentz
Date: 2021-02-14 00:00:00
Tasks: Management, Position

Six significant new methodological developments of the previously-presented "metastimuli architecture" for human learning through machine learning of spatially correlated structural position within a user's personal information management system (PIMS), providing the basis for haptic metastimuli, are presented. These include architectural innovation, recurrent (RNN) artificial neural network (ANN) application, a variety of atom embedding techniques (including a novel technique we call "nabla" embedding inspired by linguistics), ANN hyper-parameter (one that affects the network but is not trained, e.g. the learning rate) optimization, and meta-parameter (one that determines the system performance but is not trained and not a hyper-parameter, e.g. the atom embedding technique) optimization for exploring the large design space. A technique for using the system for automatic atom categorization in a user's PIMS is outlined. ANN training and hyper- and meta-parameter optimization results are presented and discussed in service of methodological recommendations.
Exclusion of Extreme Jurors and Minority Representation: The Effect of Jury Selection Procedures
Authors: Andrea Moro, Martin Van der Linden
Date: 2021-02-14 00:00:00

We compare two jury selection procedures meant to safeguard against the inclusion of biased jurors that are perceived as causing minorities to be under-represented. The Strike and Replace procedure presents potential jurors one-by-one to the parties, while the Struck procedure presents all potential jurors before the parties exercise their challenges. Struck more effectively excludes extreme jurors but leads to a worse representation of minorities. The advantage of Struck in terms of excluding extremes is sizable in a wide range of cases. In contrast, Strike and Replace better represents minorities only if the minority and majority are polarized. Results are robust to assuming the parties statistically discriminate against jurors based on group identity.
ChipNet: Budget-Aware Pruning with Heaviside Continuous Approximations
Authors: Rishabh Tiwari, Udbhav Bamba, Arnav Chavan, Deepak K. Gupta
Date: 2021-02-14 00:00:00
Proceeding: ICLR 2021 1

Structured pruning methods are among the effective strategies for extracting small resource-efficient convolutional neural networks from their dense counterparts with minimal loss in accuracy. However, most existing methods still suffer from one or more limitations, that include 1) the need for training the dense model from scratch with pruning-related parameters embedded in the architecture, 2) requiring model-specific hyperparameter settings, 3) inability to include budget-related constraint in the training process, and 4) instability under scenarios of extreme pruning. In this paper, we present ChipNet, a deterministic pruning strategy that employs continuous Heaviside function and a novel crispness loss to identify a highly sparse network out of an existing dense network. Our choice of continuous Heaviside function is inspired by the field of design optimization, where the material distribution task is posed as a continuous optimization problem, but only discrete values (0 or 1) are practically feasible and expected as final outcomes. Our approach's flexible design facilitates its use with different choices of budget constraints while maintaining stability for very low target budgets. Experimental results show that ChipNet outperforms state-of-the-art structured pruning methods by remarkable margins of up to 16.1% in terms of accuracy. Further, we show that the masks obtained with ChipNet are transferable across datasets. For certain cases, it was observed that masks transferred from a model trained on feature-rich teacher dataset provide better performance on the student dataset than those obtained by directly pruning on the student data itself.
A Minimax Game for Instance based Selective Transfer Learning
Authors: Bo wang, Minghui Qiu, Xisen Wang, Yaliang Li, Yu Gong, Xiaoyi Zeng, Jung Huang, Bo Zheng, Deng Cai, Jingren Zhou
Date: 2019-07-01 00:00:00
Tasks: Retrieval, Text Retrieval, Transfer Learning

Deep neural network based transfer learning has been widely used to leverage information from the domain with rich data to help domain with insufficient data. When the source data distribution is different from the target data, transferring knowledge between these domains may lead to negative transfer. To mitigate this problem, a typical way is to select useful source domain data for transferring. However, limited studies focus on selecting high-quality source data to help neural network based transfer learning. To bridge this gap, we propose a general Minimax Game based model for selective Transfer Learning (MGTL). More specifically, we build a selector, a discriminator and a TL module in the proposed method. The discriminator aims to maximize the differences between selected source data and target data, while the selector acts as an attacker to selected source data that are close to the target to minimize the differences. The TL module trains on the selected data and provides rewards to guide the selector. Those three modules play a minimax game to help select useful source data for transferring. Our method is also shown to speed up the training process of the learning task in the target domain than traditional TL methods. To the best of our knowledge, this is the first to build a minimax game based model for selective transfer learning. To examine the generality of our method, we evaluate it on two different tasks: item recommendation and text retrieval. Extensive experiments over both public and real-world datasets demonstrate that our model outperforms the competing methods by a large margin. Meanwhile, the quantitative evaluation shows our model can select data which are close to target data. Our model is also deployed in a real-world system and significant improvement over the baselines is observed.
End-to-End Egospheric Spatial Memory
Authors: Daniel Lenton, Stephen James, Ronald Clark, Andrew J. Davison
Date: 2021-02-15 00:00:00
Tasks: General Reinforcement Learning, Imitation Learning, Robot Navigation, Semantic Segmentation, Spatial Reasoning, Visual Navigation

Spatial memory, or the ability to remember and recall specific locations and objects, is central to autonomous agents' ability to carry out tasks in real environments. However, most existing artificial memory modules are not very adept at storing spatial information. We propose a parameter-free module, Egospheric Spatial Memory (ESM), which encodes the memory in an ego-sphere around the agent, enabling expressive 3D representations. ESM can be trained end-to-end via either imitation or reinforcement learning, and improves both training efficiency and final performance against other memory baselines on both drone and manipulator visuomotor control tasks. The explicit egocentric geometry also enables us to seamlessly combine the learned controller with other non-learned modalities, such as local obstacle avoidance. We further show applications to semantic segmentation on the ScanNet dataset, where ESM naturally combines image-level and map-level inference modalities. Through our broad set of experiments, we show that ESM provides a general computation graph for embodied spatial reasoning, and the module forms a bridge between real-time mapping systems and differentiable memory architectures. Implementation at: https://github.com/ivy-dl/memory.
Membership Inference Attacks are Easier on Difficult Problems
Authors: Avital Shafran, Shmuel Peleg, Yedid Hoshen
Date: 2021-02-15 00:00:00
Proceeding: ICCV 2021 10
Tasks: Image Segmentation, Medical Image Segmentation, Memorization, Segmentation, Semantic Segmentation, Translation

Membership inference attacks (MIA) try to detect if data samples were used to train a neural network model, e.g. to detect copyright abuses. We show that models with higher dimensional input and output are more vulnerable to MIA, and address in more detail models for image translation and semantic segmentation, including medical image segmentation. We show that reconstruction-errors can lead to very effective MIA attacks as they are indicative of memorization. Unfortunately, reconstruction error alone is less effective at discriminating between non-predictable images used in training and easy to predict images that were never seen before. To overcome this, we propose using a novel predictability error that can be computed for each sample, and its computation does not require a training set. Our membership error, obtained by subtracting the predictability error from the reconstruction error, is shown to achieve high MIA accuracy on an extensive number of benchmarks.
How Convolutional Neural Networks Deal with Aliasing
Authors: Antônio H. Ribeiro, Thomas B. Schön
Date: 2021-02-15 00:00:00

The convolutional neural network (CNN) remains an essential tool in solving computer vision problems. Standard convolutional architectures consist of stacked layers of operations that progressively downscale the image. Aliasing is a well-known side-effect of downsampling that may take place: it causes high-frequency components of the original signal to become indistinguishable from its low-frequency components. While downsampling takes place in the max-pooling layers or in the strided-convolutions in these models, there is no explicit mechanism that prevents aliasing from taking place in these layers. Due to the impressive performance of these models, it is natural to suspect that they, somehow, implicitly deal with this distortion. The question we aim to answer in this paper is simply: "how and to what extent do CNNs counteract aliasing?" We explore the question by means of two examples: In the first, we assess the CNNs capability of distinguishing oscillations at the input, showing that the redundancies in the intermediate channels play an important role in succeeding at the task; In the second, we show that an image classifier CNN while, in principle, capable of implementing anti-aliasing filters, does not prevent aliasing from taking place in the intermediate layers.
Learning Intra-Batch Connections for Deep Metric Learning
Authors: Jenny Seidenschwarz, Ismail Elezi, Laura Leal-Taixé
Date: 2021-02-15 00:00:00
Tasks: Clustering, Image Retrieval, Metric Learning, Retrieval

The goal of metric learning is to learn a function that maps samples to a lower-dimensional space where similar samples lie closer than dissimilar ones. Particularly, deep metric learning utilizes neural networks to learn such a mapping. Most approaches rely on losses that only take the relations between pairs or triplets of samples into account, which either belong to the same class or two different classes. However, these methods do not explore the embedding space in its entirety. To this end, we propose an approach based on message passing networks that takes all the relations in a mini-batch into account. We refine embedding vectors by exchanging messages among all samples in a given batch allowing the training process to be aware of its overall structure. Since not all samples are equally important to predict a decision boundary, we use an attention mechanism during message passing to allow samples to weigh the importance of each neighbor accordingly. We achieve state-of-the-art results on clustering and image retrieval on the CUB-200-2011, Cars196, Stanford Online Products, and In-Shop Clothes datasets. To facilitate further research, we make available the code and the models at https://github.com/dvl-tum/intra_batch_connections.
Deep Co-Attention Network for Multi-View Subspace Learning
Authors: Lecheng Zheng, Yu Cheng, Hongxia Yang, Nan Cao, Jingrui He
Date: 2021-02-15 00:00:00
Tasks: Diagnostic

Many real-world applications involve data from multiple modalities and thus exhibit the view heterogeneity. For example, user modeling on social media might leverage both the topology of the underlying social network and the content of the users' posts; in the medical domain, multiple views could be X-ray images taken at different poses. To date, various techniques have been proposed to achieve promising results, such as canonical correlation analysis based methods, etc. In the meanwhile, it is critical for decision-makers to be able to understand the prediction results from these methods. For example, given the diagnostic result that a model provided based on the X-ray images of a patient at different poses, the doctor needs to know why the model made such a prediction. However, state-of-the-art techniques usually suffer from the inability to utilize the complementary information of each view and to explain the predictions in an interpretable manner. To address these issues, in this paper, we propose a deep co-attention network for multi-view subspace learning, which aims to extract both the common information and the complementary information in an adversarial setting and provide robust interpretations behind the prediction to the end-users via the co-attention mechanism. In particular, it uses a novel cross reconstruction loss and leverages the label information to guide the construction of the latent representation by incorporating the classifier into our model. This improves the quality of latent representation and accelerates the convergence speed. Finally, we develop an efficient iterative algorithm to find the optimal encoders and discriminator, which are evaluated extensively on synthetic and real-world data sets. We also conduct a case study to demonstrate how the proposed method robustly interprets the predictions on an image data set.
A Data Quality-Driven View of MLOps
Authors: Cedric Renggli, Luka Rimanic, Nezihe Merve Gürel, Bojan Karlaš, Wentao Wu, Ce Zhang
Date: 2021-02-15 00:00:00
Tasks: BIG-bench Machine Learning

Developing machine learning models can be seen as a process similar to the one established for traditional software development. A key difference between the two lies in the strong dependency between the quality of a machine learning model and the quality of the data used to train or perform evaluations. In this work, we demonstrate how different aspects of data quality propagate through various stages of machine learning development. By performing a joint analysis of the impact of well-known data quality dimensions and the downstream machine learning process, we show that different components of a typical MLOps pipeline can be efficiently designed, providing both a technical and theoretical perspective.
High contrast Ultrafast 3D Ultrasound Imaging using Row Column specific Frame Multiply and Sum
Authors: Joseph Hansen-Shearer, Marcelo Lerendegui, Matthieu Toulemonde, Meng-Xing Tang
Date: 2021-02-15 00:00:00

Row-column arrays have shown to be able to generate 3-D ultrafast ultrasound images with an order of magnitude less independent electronic channels than classic 2D matrix arrays. Unfortunately row-column array images suffer from major imaging artefacts due to the high side lobes. This paper proposes a row-column specific beamforming technique that exploits the incoherent nature of certain row column array artefacts. The geometric mean of the data from each row and column pair is taken prior to summation in beamforming, thus drastically reducing incoherent imaging artefacts compared to traditional coherent compounding. The effectiveness of this technique was demonstrated in silico, and the results show an average fivefold reduction in side-lobe levels. Significantly improved contrast was demonstrated with Tissue-to-noise ratio increasing from $\sim$10dB to $\sim$30dB and Tissue Contrast Ratio increasing from $\sim$21dB to $\sim$42dB when using the proposed new method compared to Delay and Sum. These new techniques allowed for high quality 3D imaging whilst maintaining high frame rate potential.