pwc-archive/papers-with-abstracts huggingface dataset:
ENGNN: A General Edge-Update Empowered GNN Architecture for Radio Resource Management in Wireless Networks
Authors: Yunqi Wang, Yang Li, Qingjiang Shi, Yik-Chung Wu
Date: 2022-12-14 00:00:00
Tasks: Management

In order to achieve high data rate and ubiquitous connectivity in future wireless networks, a key task is to efficiently manage the radio resource by judicious beamforming and power allocation. Unfortunately, the iterative nature of the commonly applied optimization-based algorithms cannot meet the low latency requirements due to the high computational complexity. For real-time implementations, deep learning-based approaches, especially the graph neural networks (GNNs), have been demonstrated with good scalability and generalization performance due to the permutation equivariance (PE) property. However, the current architectures are only equipped with the node-update mechanism, which prohibits the applications to a more general setup, where the unknown variables are also defined on the graph edges. To fill this gap, we propose an edge-update mechanism, which enables GNNs to handle both node and edge variables and prove its PE property with respect to both transmitters and receivers. Simulation results on typical radio resource management problems demonstrate that the proposed method achieves higher sum rate but with much shorter computation time than state-of-the-art methods and generalizes well on different numbers of base stations and users, different noise variances, interference levels, and transmit power budgets.
BASQ: Branch-wise Activation-clipping Search Quantization for Sub-4-bit Neural Networks
Authors: Han-Byul Kim, Eunhyeok Park, Sungjoo Yoo
Date: 2022-10-23 00:00:00
Proceeding: ECCV(European Conference on Computer Vision) 2022 10
Tasks: Neural Architecture Search, Quantization

In this paper, we propose Branch-wise Activation-clipping Search Quantization (BASQ), which is a novel quantization method for low-bit activation. BASQ optimizes clip value in continuous search space while simultaneously searching L2 decay weight factor for updating clip value in discrete search space. We also propose a novel block structure for low precision that works properly on both MobileNet and ResNet structures with branch-wise searching. We evaluate the proposed methods by quantizing both weights and activations to 4-bit or lower. Contrary to the existing methods which are effective only for redundant networks, e.g., ResNet-18, or highly optimized networks, e.g., MobileNet-v2, our proposed method offers constant competitiveness on both types of networks across low precisions from 2 to 4-bits. Specifically, our 2-bit MobileNetv2 offers top-1 accuracy of 64.71% on ImageNet, outperforming the existing method by a large margin (2.8%), and our 4-bit MobileNet-v2 gives 71.98% which is comparable to the full-precision accuracy 71.88% while our uniform quantization method offers comparable accuracy of 2-bit ResNet-18 to the state-of-the-art non-uniform quantization method.
Dataset Factorization for Condensation
Authors: Songhua Liu, Kai Wang, Xingyi Yang, Jingwen Ye, Xinchao Wang
Date: 2022-11-01 00:00:00
Proceeding: NIPS 2022 11
Tasks: Dataset Distillation, Diversity, Hallucination, Informativeness

In this paper, we study dataset distillation (DD), from a novel perspective and introduce a \emph{dataset factorization} approach, termed \emph{HaBa}, which is a plug-and-play strategy portable to any existing DD baseline. Unlike conventional DD approaches that aim to produce distilled and representative samples, \emph{HaBa} explores decomposing a dataset into two components: data \emph{Ha}llucination networks and \emph{Ba}ses, where the latter is fed into the former to reconstruct image samples. The flexible combinations between bases and hallucination networks, therefore, equip the distilled data with exponential informativeness gain, which largely increase the representation capability of distilled datasets. To furthermore increase the data efficiency of compression results, we further introduce a pair of adversarial contrastive \xw{constraints} on the resultant hallucination networks and bases, which increase the diversity of generated images and inject more discriminant information into the factorization. Extensive comparisons and experiments demonstrate that our method can yield significant improvement on downstream classification tasks compared with previous state of the arts, while reducing the total number of compressed parameters by up to 65\%. Moreover, distilled datasets by our approach also achieve \textasciitilde10\% higher accuracy than baseline methods in cross-architecture generalization.
Brain Inspired Cortical Coding Method for Fast Clustering and Codebook Generation
Authors: Meric Yucel, Serdar Bagis, Ahmet Sertbas, Mehmet Sarikaya, Burak Berk Ustundag
Date: 2022-11-17 00:00:00
Proceeding: Entropy 2022 11
Tasks: Anomaly Detection, Clustering, Computational Efficiency, Quantization

A major archetype of artificial intelligence is developing algorithms facilitating temporal efficiency and accuracy while boosting the generalization performance. Even with the latest developments in machine learning, a key limitation has been the inefficient feature extraction from the initial data, which is essential in performance optimization. Here, we introduce a feature extraction method inspired by energy–entropy relations of sensory cortical networks in the brain. Dubbed the brain-inspired cortex, the algorithm provides convergence to orthogonal features from streaming signals with superior computational efficiency while processing data in a compressed form. We demonstrate the performance of the new algorithm using artificially created complex data by comparing it with the commonly used traditional clustering algorithms, such as Birch, GMM, and K-means. While the data processing time is significantly reduced—seconds versus hours—encoding distortions remain essentially the same in the new algorithm, providing a basis for better generalization. Although we show herein the superior performance of the cortical coding model in clustering and vector quantization, it also provides potent implementation opportunities for machine learning fundamental components, such as reasoning, anomaly detection and classification in large scope applications, e.g., finance, cybersecurity, and healthcare.
Lightweight Image Inpainting by Stripe Window Transformer with Joint Attention to CNN
Authors: Tsung-Jung Liu, Bo-Wei Chen, Kuan-Hsien Liu
Date: 2023-01-02 00:00:00
Tasks: Image Inpainting

Image inpainting is an important task in computer vision. As admirable methods are presented, the inpainted image is getting closer to reality. However, the result is still not good enough in the reconstructed texture and structure based on human vision. Although recent advances in computer hardware have enabled the development of larger and more complex models, there is still a need for lightweight models that can be used by individuals and small-sized institutions. Therefore, we propose a lightweight model that combines a specialized transformer with a traditional convolutional neural network (CNN). Furthermore, we have noticed most researchers only consider three primary colors (RGB) in inpainted images, but we think this is not enough. So we propose a new loss function to intensify color details. Extensive experiments on commonly seen datasets (Places2 and CelebA) validate the efficacy of our proposed model compared with other state-of-the-art methods. Index Terms: HSV color space, image inpainting, joint attention, stripe window, transformer
Phase Turbulence Prediction Method for Line-of-Sight Multiple-Input–Multiple-Output Links Caused by Atmospheric Environment
Authors: Rui Zhang, Shoubao Zhang, Zhenwei Zhao, Zhensen Wu, Changsheng Lu, and Leke Lin
Date: 2022-01-16 00:00:00
Proceeding: IEEE ANTENNAS AND WIRELESS PROPAGATION LETTERS 2022 1
Tasks: Prediction

The multiple-input–multiple-output (MIMO) phase of the received signal is the most important parameter in the line-ofsight MIMO (LOS MIMO) system. The MIMO phase turbulence often occurs in practical applications, and the LOS MIMO system cannot operate normally in severe cases. However, there is no practical MIMO phase prediction method presently. A tropospheric radio-duct during phase turbulence was observed using a mesoscale numerical weather prediction (NWP) model. In this study, a novel MIMO phase prediction method was proposed. In the method, the tropospheric refractivity profile was predicted using a mesoscale NWP model, and then MIMO phase was predicted by the parabolic equation model. The comparison of the predicted MIMO phases with the measured results shows good agreement. MIMO phase turbulence prediction method can provide support for LOS MIMO system performance prediction.
MAUD: An Expert-Annotated Legal NLP Dataset for Merger Agreement Understanding
Authors: Steven H. Wang, Antoine Scardigli, Leonard Tang, Wei Chen, Dimitry Levkin, Anya Chen, Spencer Ball, Thomas Woodside, Oliver Zhang, Dan Hendrycks
Date: 2023-01-02 00:00:00
Tasks: Reading Comprehension

Reading comprehension of legal text can be a particularly challenging task due to the length and complexity of legal clauses and a shortage of expert-annotated datasets. To address this challenge, we introduce the Merger Agreement Understanding Dataset (MAUD), an expert-annotated reading comprehension dataset based on the American Bar Association's 2021 Public Target Deal Points Study, with over 39,000 examples and over 47,000 total annotations. Our fine-tuned Transformer baselines show promising results, with models performing well above random on most questions. However, on a large subset of questions, there is still room for significant improvement. As the only expert-annotated merger agreement dataset, MAUD is valuable as a benchmark for both the legal profession and the NLP community.
Optimizing Agent Collaboration through Heuristic Multi-Agent Planning
Authors: Nitsan Soffair
Date: 2023-01-03 00:00:00

The SOTA algorithms for addressing QDec-POMDP issues, QDec-FP and QDec-FPS, are unable to effectively tackle problems that involve different types of sensing agents. We propose a new algorithm that addresses this issue by requiring agents to adopt the same plan if one agent is unable to take a sensing action but the other can. Our algorithm performs significantly better than both QDec-FP and QDec-FPS in these types of situations.
An Empirical Investigation into the Use of Image Captioning for Automated Software Documentation
Authors: Kevin Moran, Ali Yachnes, George Purnell, Junayed Mahmud, Michele Tufano, Carlos Bernal-Cárdenas, Denys Poshyvanyk, Zach H'Doubler
Date: 2023-01-03 00:00:00
Tasks: Image Captioning, Machine Translation

Existing automated techniques for software documentation typically attempt to reason between two main sources of information: code and natural language. However, this reasoning process is often complicated by the lexical gap between more abstract natural language and more structured programming languages. One potential bridge for this gap is the Graphical User Interface (GUI), as GUIs inherently encode salient information about underlying program functionality into rich, pixel-based data representations. This paper offers one of the first comprehensive empirical investigations into the connection between GUIs and functional, natural language descriptions of software. First, we collect, analyze, and open source a large dataset of functional GUI descriptions consisting of 45,998 descriptions for 10,204 screenshots from popular Android applications. The descriptions were obtained from human labelers and underwent several quality control mechanisms. To gain insight into the representational potential of GUIs, we investigate the ability of four Neural Image Captioning models to predict natural language descriptions of varying granularity when provided a screenshot as input. We evaluate these models quantitatively, using common machine translation metrics, and qualitatively through a large-scale user study. Finally, we offer learned lessons and a discussion of the potential shown by multimodal models to enhance future techniques for automated software documentation.
Generative appearance replay for continual unsupervised domain adaptation
Authors: Boqi Chen, Kevin Thandiackal, Pushpak Pati, Orcun Goksel
Date: 2023-01-03 00:00:00
Tasks: Continual Learning, Domain Adaptation, Unsupervised Domain Adaptation

Deep learning models can achieve high accuracy when trained on large amounts of labeled data. However, real-world scenarios often involve several challenges: Training data may become available in installments, may originate from multiple different domains, and may not contain labels for training. Certain settings, for instance medical applications, often involve further restrictions that prohibit retention of previously seen data due to privacy regulations. In this work, to address such challenges, we study unsupervised segmentation in continual learning scenarios that involve domain shift. To that end, we introduce GarDA (Generative Appearance Replay for continual Domain Adaptation), a generative-replay based approach that can adapt a segmentation model sequentially to new domains with unlabeled data. In contrast to single-step unsupervised domain adaptation (UDA), continual adaptation to a sequence of domains enables leveraging and consolidation of information from multiple domains. Unlike previous approaches in incremental UDA, our method does not require access to previously seen data, making it applicable in many practical scenarios. We evaluate GarDA on two datasets with different organs and modalities, where it substantially outperforms existing techniques.
A Survey On Few-shot Knowledge Graph Completion with Structural and Commonsense Knowledge
Authors: Haodi Ma, Daisy Zhe Wang
Date: 2023-01-03 00:00:00
Tasks: Few-Shot Learning, Graph Representation Learning, Knowledge Graph Completion, Knowledge Graphs, Representation Learning

Knowledge graphs (KG) have served as the key component of various natural language processing applications. Commonsense knowledge graphs (CKG) are a special type of KG, where entities and relations are composed of free-form text. However, previous works in KG completion and CKG completion suffer from long-tail relations and newly-added relations which do not have many know triples for training. In light of this, few-shot KG completion (FKGC), which requires the strengths of graph representation learning and few-shot learning, has been proposed to challenge the problem of limited annotated data. In this paper, we comprehensively survey previous attempts on such tasks in the form of a series of methods and applications. Specifically, we first introduce FKGC challenges, commonly used KGs, and CKGs. Then we systematically categorize and summarize existing works in terms of the type of KGs and the methods. Finally, we present applications of FKGC models on prediction tasks in different areas and share our thoughts on future research directions of FKGC.
Causal Discovery for Gene Regulatory Network Prediction
Authors: Jacob Rast
Date: 2023-01-03 00:00:00
Tasks: Causal Discovery, Prediction

Biological systems and processes are networks of complex nonlinear regulatory interactions between nucleic acids, proteins, and metabolites. A natural way in which to represent these interaction networks is through the use of a graph. In this formulation, each node represents a nucleic acid, protein, or metabolite and edges represent intermolecular interactions (inhibition, regulation, promotion, coexpression, etc.). In this work, a novel algorithm for the discovery of latent graph structures given experimental data is presented.
xDeepInt: a hybrid architecture for modeling the vector-wise and bit-wise feature interactions
Authors: YaChen Yan, Liubo Li
Date: 2023-01-03 00:00:00
Tasks: Click-Through Rate Prediction, Feature Engineering, feature selection

Learning feature interactions is the key to success for the large-scale CTR prediction and recommendation. In practice, handcrafted feature engineering usually requires exhaustive searching. In order to reduce the high cost of human efforts in feature engineering, researchers propose several deep neural networks (DNN)-based approaches to learn the feature interactions in an end-to-end fashion. However, existing methods either do not learn both vector-wise interactions and bit-wise interactions simultaneously, or fail to combine them in a controllable manner. In this paper, we propose a new model, xDeepInt, based on a novel network architecture called polynomial interaction network (PIN) which learns higher-order vector-wise interactions recursively. By integrating subspace-crossing mechanism, we enable xDeepInt to balance the mixture of vector-wise and bit-wise feature interactions at a bounded order. Based on the network architecture, we customize a combined optimization strategy to conduct feature selection and interaction selection. We implement the proposed model and evaluate the model performance on three real-world datasets. Our experiment results demonstrate the efficacy and effectiveness of xDeepInt over state-of-the-art models. We open-source the TensorFlow implementation of xDeepInt: https://github.com/yanyachen/xDeepInt.
PIE-QG: Paraphrased Information Extraction for Unsupervised Question Generation from Small Corpora
Authors: Dinesh Nagumothu, Bahadorreza Ofoghi, Guangyan Huang, Peter W. Eklund
Date: 2023-01-03 00:00:00
Tasks: Language Modeling, Language Modelling, Open Information Extraction, Question Answering, Question Generation, Question-Generation

Supervised Question Answering systems (QA systems) rely on domain-specific human-labeled data for training. Unsupervised QA systems generate their own question-answer training pairs, typically using secondary knowledge sources to achieve this outcome. Our approach (called PIE-QG) uses Open Information Extraction (OpenIE) to generate synthetic training questions from paraphrased passages and uses the question-answer pairs as training data for a language model for a state-of-the-art QA system based on BERT. Triples in the form of <subject, predicate, object> are extracted from each passage, and questions are formed with subjects (or objects) and predicates while objects (or subjects) are considered as answers. Experimenting on five extractive QA datasets demonstrates that our technique achieves on-par performance with existing state-of-the-art QA systems with the benefit of being trained on an order of magnitude fewer documents and without any recourse to external reference data sources.
New Information Technologies, Simulation and Automation
Authors: Vitalii Velychko, Svitlana Voinova, Valery Granyak, Liliia Ivanova, Sergii Kotlyk, Alona Kudriashova, Tetiana Kunup, Kyrylo Malakhov, Iryna Pikh, Nataliia Punchenko, Vsevolod Senkivskyy, Olexandra Sergeeva, Oksana Sokolova, Sergiy Fedosov, Oleksandr Khoshaba, Olexandra Tsyra, Yuri Chaplinskyy, Olexander Gurskiy, Kostiantyn Zavertailo, Diana Kotlyk
Date: 2023-01-03 00:00:00
Tasks: Management

The monograph summarizes and analyzes the current state of development of computer and mathematical simulation and modeling, the automation of management processes, the use of information technologies in education, the design of information systems and software complexes, the development of computer telecommunication networks and technologies most areas that are united by the term Industry 4.0
Semi-Structured Object Sequence Encoders
Authors: Rudra Murthy V, Riyaz Bhat, Chulaka Gunasekara, Siva Sankalp Patel, Hui Wan, Tejas Indulal Dhamecha, Danish Contractor, Marina Danilevsky
Date: 2023-01-03 00:00:00
Tasks: Object

In this paper we explore the task of modeling semi-structured object sequences; in particular, we focus our attention on the problem of developing a structure-aware input representation for such sequences. Examples of such data include user activity on websites, machine logs, and many others. This type of data is often represented as a sequence of sets of key-value pairs over time and can present modeling challenges due to an ever-increasing sequence length. We propose a two-part approach, which first considers each key independently and encodes a representation of its values over time; we then self-attend over these value-aware key representations to accomplish a downstream task. This allows us to operate on longer object sequences than existing methods. We introduce a novel shared-attention-head architecture between the two modules and present an innovative training schedule that interleaves the training of both modules with shared weights for some attention heads. Our experiments on multiple prediction tasks using real-world data demonstrate that our approach outperforms a unified network with hierarchical encoding, as well as other methods including a record-centric representation and a flattened representation of the sequence.
A New Perspective to Boost Vision Transformer for Medical Image Classification
Authors: Yuexiang Li, Yawen Huang, Nanjun He, Kai Ma, Yefeng Zheng
Date: 2023-01-03 00:00:00
Tasks: Diabetic Retinopathy Grading, image-classification, Image Classification, Lesion Classification, Medical Image Classification, Representation Learning, Self-Supervised Learning, Skin Lesion Classification

Transformer has achieved impressive successes for various computer vision tasks. However, most of existing studies require to pretrain the Transformer backbone on a large-scale labeled dataset (e.g., ImageNet) for achieving satisfactory performance, which is usually unavailable for medical images. Additionally, due to the gap between medical and natural images, the improvement generated by the ImageNet pretrained weights significantly degrades while transferring the weights to medical image processing tasks. In this paper, we propose Bootstrap Own Latent of Transformer (BOLT), a self-supervised learning approach specifically for medical image classification with the Transformer backbone. Our BOLT consists of two networks, namely online and target branches, for self-supervised representation learning. Concretely, the online network is trained to predict the target network representation of the same patch embedding tokens with a different perturbation. To maximally excavate the impact of Transformer from limited medical data, we propose an auxiliary difficulty ranking task. The Transformer is enforced to identify which branch (i.e., online/target) is processing the more difficult perturbed tokens. Overall, the Transformer endeavours itself to distill the transformation-invariant features from the perturbed tokens to simultaneously achieve difficulty measurement and maintain the consistency of self-supervised representations. The proposed BOLT is evaluated on three medical image processing tasks, i.e., skin lesion classification, knee fatigue fracture grading and diabetic retinopathy grading. The experimental results validate the superiority of our BOLT for medical image classification, compared to ImageNet pretrained weights and state-of-the-art self-supervised learning approaches.
AI in HCI Design and User Experience
Authors: Wei Xu
Date: 2023-01-03 00:00:00

In this chapter, we review and discuss the transformation of AI technology in HCI/UX work and assess how AI technology will change how we do the work. We first discuss how AI can be used to enhance the result of user research and design evaluation. We then discuss how AI technology can be used to enhance HCI/UX design. Finally, we discuss how AI-enabled capabilities can improve UX when users interact with computing systems, applications, and services.