pwc-archive/papers-with-abstracts huggingface dataset:
A Study of Neural Collapse Phenomenon: Grassmannian Frame, Symmetry and Generalization
Authors: Peifeng Gao, Qianqian Xu, Peisong Wen, Huiyang Shao, Zhiyong Yang, Qingming Huang
Date: 2023-04-18 00:00:00

In this paper, we extend original Neural Collapse Phenomenon by proving Generalized Neural Collapse hypothesis. We obtain Grassmannian Frame structure from the optimization and generalization of classification. This structure maximally separates features of every two classes on a sphere and does not require a larger feature dimension than the number of classes. Out of curiosity about the symmetry of Grassmannian Frame, we conduct experiments to explore if models with different Grassmannian Frames have different performance. As a result, we discover the Symmetric Generalization phenomenon. We provide a theorem to explain Symmetric Generalization of permutation. However, the question of why different directions of features can lead to such different generalization is still open for future investigation.
Revisiting k-NN for Fine-tuning Pre-trained Language Models
Authors: Lei LI, Jing Chen, Bozhong Tian, Ningyu Zhang
Date: 2023-04-18 00:00:00
Tasks: Diversity

Pre-trained Language Models (PLMs), as parametric-based eager learners, have become the de-facto choice for current paradigms of Natural Language Processing (NLP). In contrast, k-Nearest-Neighbor (kNN) classifiers, as the lazy learning paradigm, tend to mitigate over-fitting and isolated noise. In this paper, we revisit kNN classifiers for augmenting the PLMs-based classifiers. From the methodological level, we propose to adopt kNN with textual representations of PLMs in two steps: (1) Utilize kNN as prior knowledge to calibrate the training process. (2) Linearly interpolate the probability distribution predicted by kNN with that of the PLMs' classifier. At the heart of our approach is the implementation of kNN-calibrated training, which treats predicted results as indicators for easy versus hard examples during the training process. From the perspective of the diversity of application scenarios, we conduct extensive experiments on fine-tuning, prompt-tuning paradigms and zero-shot, few-shot and fully-supervised settings, respectively, across eight diverse end-tasks. We hope our exploration will encourage the community to revisit the power of classical methods for efficient NLP. Code and datasets are available in https://github.com/zjunlp/Revisit-KNN.
CodeKGC: Code Language Model for Generative Knowledge Graph Construction
Authors: Zhen Bi, Jing Chen, Yinuo Jiang, Feiyu Xiong, Wei Guo, Huajun Chen, Ningyu Zhang
Date: 2023-04-18 00:00:00
Tasks: Code Completion, graph construction, Language Modeling, Language Modelling

Current generative knowledge graph construction approaches usually fail to capture structural knowledge by simply flattening natural language into serialized texts or a specification language. However, large generative language model trained on structured data such as code has demonstrated impressive capability in understanding natural language for structural prediction and reasoning tasks. Intuitively, we address the task of generative knowledge graph construction with code language model: given a code-format natural language input, the target is to generate triples which can be represented as code completion tasks. Specifically, we develop schema-aware prompts that effectively utilize the semantic structure within the knowledge graph. As code inherently possesses structure, such as class and function definitions, it serves as a useful model for prior semantic structural knowledge. Furthermore, we employ a rationale-enhanced generation method to boost the performance. Rationales provide intermediate steps, thereby improving knowledge extraction abilities. Experimental results indicate that the proposed approach can obtain better performance on benchmark datasets compared with baselines. Code and datasets are available in https://github.com/zjunlp/DeepKE/tree/main/example/llm.
You Only Need Two Detectors to Achieve Multi-Modal 3D Multi-Object Tracking
Authors: Xiyang Wang, Chunyun Fu, JiaWei He, Mingguang Huang, Ting Meng, Siyu Zhang, Hangning Zhou, Ziyao Xu, Chi Zhang
Date: 2023-04-18 00:00:00
Tasks: 3D Multi-Object Tracking, Multi-Object Tracking, Object, object-detection, Object Detection, Object Tracking, regression

In the classical tracking-by-detection (TBD) paradigm, detection and tracking are separately and sequentially conducted, and data association must be properly performed to achieve satisfactory tracking performance. In this paper, a new end-to-end multi-object tracking framework is proposed, which integrates object detection and multi-object tracking into a single model. The proposed tracking framework eliminates the complex data association process in the classical TBD paradigm, and requires no additional training. Secondly, the regression confidence of historical trajectories is investigated, and the possible states of a trajectory (weak object or strong object) in the current frame are predicted. Then, a confidence fusion module is designed to guide non-maximum suppression for trajectories and detections to achieve ordered and robust tracking. Thirdly, by integrating historical trajectory features, the regression performance of the detector is enhanced, which better reflects the occlusion and disappearance patterns of objects in real world. Lastly, extensive experiments are conducted on the commonly used KITTI and Waymo datasets. The results show that the proposed framework can achieve robust tracking by using only a 2D detector and a 3D detector, and it is proven more accurate than many of the state-of-the-art TBD-based multi-modal tracking methods. The source codes of the proposed method are available at https://github.com/wangxiyang2022/YONTD-MOT.
Experimental Impact Analysis of Cyberattacks in Power Systems using Digital Real-Time Testbeds
Authors: Kalinath Katuri, Ioannis Zografopoulos, Ha Thi Nguyen, Charalambos Konstantinou
Date: 2023-04-15 00:00:00

Smart grid advancements and the increased integration of digital devices have transformed the existing power grid into a cyber-physical energy system. This reshaping of the current power system can make it vulnerable to cyberattacks, which could cause irreversible damage to the energy infrastructure resulting in the loss of power, equipment damage, etc. Constant threats emphasize the importance of cybersecurity investigations. At the same time, developing cyber-physical system (CPS) simulation testbeds is crucial for vulnerability assessment and the implementation and validation of security solutions. In this paper, two separate real-time CPS testbeds are developed based on the availability of local research facilities for impact analysis of denial-of-service (DoS) attacks on microgrids. The two configurations are implemented using two different digital real-time simulator systems, one using the real-time digital simulator (RTDS) with a hardware-in-the-loop (HIL) setup and the other one using OPAL-RT with ExataCPS to emulate the cyber-layer infrastructure. Both testbeds demonstrate the impact of DoS attacks on microgrid control and protection operation.
Can sparsity improve the privacy of neural networks?
Authors: Antoine Gonon, Léon Zheng, Clément Lalanne, Quoc-Tung Le, Guillaume Lauga, Can Pouliquen
Date: 2023-04-11 00:00:00

Sparse neural networks are mainly motivated by ressource efficiency since they use fewer parameters than their dense counterparts but still reach comparable accuracies. This article empirically investigates whether sparsity could also improve the privacy of the data used to train the networks. The experiments show positive correlations between the sparsity of the model, its privacy, and its classification error. Simply comparing the privacy of two models with different sparsity levels can yield misleading conclusions on the role of sparsity, because of the additional correlation with the classification error. From this perspective, some caveats are raised about previous works that investigate sparsity and privacy.
GradMDM: Adversarial Attack on Dynamic Networks
Authors: Jianhong Pan, Lin Geng Foo, Qichen Zheng, Zhipeng Fan, Hossein Rahmani, Qiuhong Ke, Jun Liu
Date: 2023-04-01 00:00:00
Tasks: Adversarial Attack, Dynamic neural networks

Dynamic neural networks can greatly reduce computation redundancy without compromising accuracy by adapting their structures based on the input. In this paper, we explore the robustness of dynamic neural networks against energy-oriented attacks targeted at reducing their efficiency. Specifically, we attack dynamic models with our novel algorithm GradMDM. GradMDM is a technique that adjusts the direction and the magnitude of the gradients to effectively find a small perturbation for each input, that will activate more computational units of dynamic models during inference. We evaluate GradMDM on multiple datasets and dynamic models, where it outperforms previous energy-oriented attack techniques, significantly increasing computation complexity while reducing the perceptibility of the perturbations.
Attentive Q-Matrix Learning for Knowledge Tracing
Authors: Zhongfeng Jia, Wei Su, Jiamin Liu, Wenli Yue
Date: 2023-04-06 00:00:00
Tasks: Knowledge Tracing

As the rapid development of Intelligent Tutoring Systems (ITS) in the past decade, tracing the students' knowledge state has become more and more important in order to provide individualized learning guidance. This is the main idea of Knowledge Tracing (KT), which models students' mastery of knowledge concepts (KCs, skills needed to solve a question) based on their past interactions on platforms. Plenty of KT models have been proposed and have shown remarkable performance recently. However, the majority of these models use concepts to index questions, which means the predefined skill tags for each question are required in advance to indicate the KCs needed to answer that question correctly. This makes it pretty hard to apply on large-scale online education platforms where questions are often not well-organized by skill tags. In this paper, we propose Q-matrix-based Attentive Knowledge Tracing (QAKT), an end-to-end style model that is able to apply the attentive method to scenes where no predefined skill tags are available without sacrificing its performance. With a novel hybrid embedding method based on the q-matrix and Rasch model, QAKT is capable of modeling problems hierarchically and learning the q-matrix efficiently based on students' sequences. Meanwhile, the architecture of QAKT ensures that it is friendly to questions associated with multiple skills and has outstanding interpretability. After conducting experiments on a variety of open datasets, we empirically validated that our model shows similar or even better performance than state-of-the-art KT methods. Results of further experiments suggest that the q-matrix learned by QAKT is highly model-agnostic and more information-sufficient than the one labeled by human experts, which could help with the data mining tasks in existing ITSs.
Towards Mitigating ChatGPT's Negative Impact on Education: Optimizing Question Design through Bloom's Taxonomy
Authors: Saber Elsayed
Date: 2023-03-31 00:00:00

The popularity of generative text AI tools in answering questions has led to concerns regarding their potential negative impact on students' academic performance and the challenges that educators face in evaluating student learning. To address these concerns, this paper introduces an evolutionary approach that aims to identify the best set of Bloom's taxonomy keywords to generate questions that these tools have low confidence in answering. The effectiveness of this approach is evaluated through a case study that uses questions from a Data Structures and Representation course being taught at the University of New South Wales in Canberra, Australia. The results demonstrate that the optimization algorithm is able to find keywords from different cognitive levels to create questions that ChatGPT has low confidence in answering. This study is a step forward to offer valuable insights for educators seeking to create more effective questions that promote critical thinking among students.
How Do Digital Advertising Auctions Impact Product Prices?
Authors: Dirk Bergemann, Alessandro Bonatti, Nicholas Wu
Date: 2023-04-17 00:00:00

We present a model of digital advertising with three key features: (i) advertisers can reach consumers on and off a platform, (ii) additional data enhances the value of advertiser-consumer matches, and (iii) bidding follows auction-like mechanisms. We contrast data-augmented auctions, which leverage the platform's data advantage to improve match quality, and managed campaign mechanisms that automate match formation and price-setting. The platform-optimal mechanism is a managed campaign that conditions on-platform prices for sponsored products on the off-platform prices set by all advertisers. This mechanism yields the efficient on-platform allocation but inefficient off-platform allocations due to high product prices; it attains the vertical integration profit for the platform and advertisers; and it increases off-platform product prices and decreases consumer surplus, relative to data-augmented auctions.
Joint Beam Scheduling and Power Allocation for SWIPT in Mixed Near- and Far-Field Channels
Authors: Yunpu Zhang, Changsheng You, Weijie Yuan, Fan Liu, Rui Zhang
Date: 2023-04-17 00:00:00
Tasks: Scheduling

Extremely large-scale array (XL-array) has emerged as a promising technology to enhance the spectrum efficiency and spatial resolution in future wireless networks, leading to a fundamental paradigm shift from conventional far-field communications towards the new near-field communications. Different from the existing works that mostly considered simultaneous wireless information and power transfer (SWIPT) in the far field, we consider in this paper a new and practical scenario, called mixed near- and far-field SWIPT, in which energy harvesting (EH) and information decoding (ID) receivers are located in the near- and far-field regions of the XL-array base station (BS), respectively. Specifically, we formulate an optimization problem to maximize the weighted sum-power harvested at all EH receivers by jointly designing the BS beam scheduling and power allocation, under the constraints on the ID sum-rate and BS transmit power. To solve this nonconvex optimization problem, an efficient algorithm is proposed to obtain a suboptimal solution by leveraging the binary variable elimination and successive convex approximation methods. Numerical results demonstrate that our proposed joint design achieves substantial performance gain over other benchmark schemes.
Barankin-Type Bound for Constrained Parameter Estimation
Authors: Eyal Nitzan, Tirza Routtenberg, Joseph Tabrikian
Date: 2023-04-17 00:00:00
Tasks: Direction of Arrival Estimation, parameter estimation, Vocal Bursts Type Prediction

In constrained parameter estimation, the classical constrained Cramer-Rao bound (CCRB) and the recent Lehmann-unbiased CCRB (LU-CCRB) are lower bounds on the performance of mean-unbiased and Lehmann-unbiased estimators, respectively. Both the CCRB and the LU-CCRB require differentiability of the likelihood function, which can be a restrictive assumption. Additionally, these bounds are local bounds that are inappropriate for predicting the threshold phenomena of the constrained maximum likelihood (CML) estimator. The constrained Barankin-type bound (CBTB) is a nonlocal mean-squared-error (MSE) lower bound for constrained parameter estimation that does not require differentiability of the likelihood function. However, this bound requires a restrictive mean-unbiasedness condition in the constrained set. In this work, we propose the Lehmann-unbiased CBTB (LU-CBTB) on the weighted MSE (WMSE). This bound does not require differentiability of the likelihood function and assumes uniform Lehmann-unbiasedness, which is less restrictive than the CBTB uniform mean-unbiasedness. We show that the LU-CBTB is tighter than or equal to the LU-CCRB and coincides with the CBTB for linear constraints. For nonlinear constraints the LU-CBTB and the CBTB are different and the LU-CBTB can be a lower bound on the WMSE of constrained estimators in cases, where the CBTB is not. In the simulations, we consider direction-of-arrival estimation of an unknown constant modulus discrete signal. In this case, the likelihood function is not differentiable and constrained Cramer-Rao-type bounds do not exist, while CBTBs exist. It is shown that the LU-CBTB better predicts the CML estimator performance than the CBTB, since the CML estimator is Lehmann-unbiased but not mean-unbiased.
Learning-Based One-Bit Maximum Likelihood Detection for Massive MIMO Systems: Dithering-Aided Adaptive Approach
Authors: Yunseong Cho, Jinseok Choi, Brian L. Evans
Date: 2023-04-16 00:00:00

In this paper, we propose a learning-based detection framework for uplink massive multiple-input and multiple-output (MIMO) systems with one-bit analog-to-digital converters. The learning-based detection only requires counting the occurrences of the quantized outputs of -1 and +1 for estimating a likelihood probability at each antenna. Accordingly, the key advantage of this approach is to perform maximum likelihood detection without explicit channel estimation which has been one of the primary challenges of one-bit quantized systems. However, due to the quasi-deterministic reception in the high signal-to-noise ratio (SNR) regime, one-bit observations in the high SNR regime are biased to either +1 or -1, and thus, the learning requires excessive training to estimate the small likelihood probabilities. To address this drawback, we propose a dither-and-learning technique to estimate likelihood functions from dithered signals. First, we add a dithering signal to artificially decrease the SNR and then infer the likelihood function from the quantized dithered signals by using an SNR estimate derived from a deep neural network-based estimator which is trained offline. We extend our technique by developing an adaptive dither-and-learning method that updates the dithering power according to the patterns observed in the quantized dithered signals. The proposed framework is also applied to channel-coded MIMO systems by computing a bit-wise and user-wise log-likelihood ratio from the refined likelihood probabilities. Simulation results validate the performance of the proposed methods in both uncoded and coded systems.
Characterization of the weak Pareto boundary of resource allocation problems in wireless networks -- Implications to cell-less systems
Authors: Renato Luis Garrido Cavalcante, Lorenzo Miretti, Slawomir Stanczak
Date: 2023-04-14 00:00:00

We establish necessary and sufficient conditions for a network configuration to provide utilities that are both fair and efficient in a well-defined sense. To cover as many applications as possible with a unified framework, we consider utilities defined in an axiomatic way, and the constraints imposed on the feasible network configurations are expressed with a single inequality involving a monotone norm. In this setting, we prove that a necessary and sufficient condition to obtain network configurations that are efficient in the weak Pareto sense is to select configurations attaining equality in the monotone norm constraint. Furthermore, for a given configuration satisfying this equality, we characterize a criterion for which the configuration can be considered fair for the active links. We illustrate potential implications of the theoretical findings by presenting, for the first time, a simple parametrization based on power vectors of achievable rate regions in modern cell-less systems subject to practical impairments.
Bayesian Formal Synthesis of Unknown Systems via Robust Simulation Relations
Authors: Oliver Schön, Birgit van Huijgevoort, Sofie Haesaert, Sadegh Soudjani
Date: 2023-04-14 00:00:00

This paper addresses the problem of data-driven computation of controllers that are correct by design for safety-critical systems and can provably satisfy (complex) functional requirements. With a focus on continuous-space stochastic systems with parametric uncertainty, we propose a two-stage approach that decomposes the problem into a learning stage and a robust formal controller synthesis stage. The first stage utilizes available Bayesian regression results to compute robust credible sets for the true parameters of the system. For the second stage, we introduce methods for systems subject to both stochastic and parametric uncertainties. We provide simulation relations for enabling correct-by-design control refinement that are founded on coupling uncertainties of stochastic systems via sub-probability measures. The presented relations are essential for constructing abstract models that are related to not only one model but to a set of parameterized models. The results are demonstrated on three case studies, including a nonlinear and a high-dimensional system.
The future of hearing aid technology
Authors: Volker Hohmann
Date: 2023-04-13 00:00:00
Tasks: Speech Enhancement

Background. Hearing aid technology has proven successful in the rehabilitation of hearing loss, but its performance is still limited in difficult everyday conditions characterized by noise and reverberation. Objectives. Introduction to the current state of hearing aid technology and presentation of the current state of research and future development. Methods. Current literature is analyzed and several specific new developments are presented. Results. Both objective and subjective data from empirical studies show the limitation of current technology. Examples of current research show the potential of machine-learning based algorithms and multi-modal signal processing for improving speech processing and perception, of using virtual reality for improving hearing device fitting and of mobile health technology for improving hearing-health services. Conclusions. Hearing device technology will remain a key factor in the rehabilitation of hearing impairment. New technology such as machine learning, and multi-modal signal processing, virtual reality and mobile health technology will improve speech enhancement, individual fitting and communication training.
Introduction of an intriguing approach for eletric current transformer on-site examining repairing
Authors: Yuxuan Chen, Jing Sun, Boqi Meng
Date: 2023-04-17 00:00:00

The working principle of a electric current transformer is based on electromagnetic induction, mainly composed of a closed iron core and windings. Its primary winding has relatively few turns and is connected in series with the current circuit to be measured. However, due to the frequent occurrence of full current passing through the current transformer during use, its secondary winding has relatively more turns. During use, errors may occur in the current transformer. Therefore, technical personnel can adopt the heterodyne measurement method to shield interference signals and ensure the accuracy of measurements during on-site examining and repairing of current transformers. This article mainly introduces the characteristics of transformers and their on-site examining and repairing process and errors, and describes the specific application of heterodyne measurement in on-site examining and repairing of current transformers.