OpenRaiser/Intern-Atlas huggingface dataset:
Analyzing Journal Category Assignment Using a Paper-level Classification System: Multidisciplinary Sciences Journals
Venue: arXiv cs.DL 2023
Year: 2023

In the field of scientometrics, the subject classification system of academic journals holds great importance. Accurate identification and classification of "multidisciplinary" journals are crucial in revealing the scientific structure and evaluating journals. Based on data from the Web of Science database from 2016 to 2020, we calculated the disciplinary diversity of journals using the paper-level subject classification system, then conducted a systematic analysis of JCR multidisciplinary journals. Studies showed that most multidisciplinary journals have high disciplinary diversity, while non-multidisciplinary journals tend to have relatively lower diversity. Some multidisciplinary journals with low disciplinary diversities may misclassify disciplines. In addition, there are inconsistencies in the diversity of journal disciplines at different granularities. Our study also visually analyzed the four types of diversity distribution tendencies of multidisciplinary journals. Moreover, ten potential multidisciplinary journals were found in non-multidisciplinary categories.
Cuttlefish: Low-Rank Model Training without All the Tuning
Venue: arXiv cs.LG 2023
Year: 2023

Recent research has shown that training low-rank neural networks can effectively reduce the total number of trainable parameters without sacrificing predictive accuracy, resulting in end-to-end speedups. However, low-rank model training necessitates adjusting several additional factorization hyperparameters, such as the rank of the factorization at each layer. In this paper, we tackle this challenge by introducing Cuttlefish, an automated low-rank training approach that eliminates the need for tuning factorization hyperparameters. Cuttlefish leverages the observation that after a few epochs of full-rank training, the stable rank (i.e., an approximation of the true rank) of each layer stabilizes at a constant value. Cuttlefish switches from full-rank to low-rank training once the stable ranks of all layers have converged, setting the dimension of each factorization to its corresponding stable rank. Our results show that Cuttlefish generates models up to 5.6 times smaller than full-rank models, and attains up to a 1.2 times faster end-to-end training process while preserving comparable accuracy. Moreover, Cuttlefish outperforms state-of-the-art low-rank model training methods and other prominent baselines. The source code for our implementation can be found at: https://github.com/hwang595/Cuttlefish.
Analyzing Hong Kong's Legal Judgments from a Computational Linguistics point-of-view
Venue: arXiv cs.CL 2023
Year: 2023

Analysis and extraction of useful information from legal judgments using computational linguistics was one of the earliest problems posed in the domain of information retrieval. Presently, several commercial vendors exist who automate such tasks. However, a crucial bottleneck arises in the form of exorbitant pricing and lack of resources available in analysis of judgements mete out by Hong Kong's Legal System. This paper attempts to bridge this gap by providing several statistical, machine learning, deep learning and zero-shot learning based methods to effectively analyze legal judgments from Hong Kong's Court System. The methods proposed consists of: (1) Citation Network Graph Generation, (2) PageRank Algorithm, (3) Keyword Analysis and Summarization, (4) Sentiment Polarity, and (5) Paragrah Classification, in order to be able to extract key insights from individual as well a group of judgments together. This would make the overall analysis of judgments in Hong Kong less tedious and more automated in order to extract insights quickly using fast inferencing. We also provide an analysis of our results by benchmarking our results using Large Language Models making robust use of the HuggingFace ecosystem.
Text Reading Order in Uncontrolled Conditions by Sparse Graph Segmentation
Venue: arXiv cs.CV 2023
Year: 2023

Text reading order is a crucial aspect in the output of an OCR engine, with a large impact on downstream tasks. Its difficulty lies in the large variation of domain specific layout structures, and is further exacerbated by real-world image degradations such as perspective distortions. We propose a lightweight, scalable and generalizable approach to identify text reading order with a multi-modal, multi-task graph convolutional network (GCN) running on a sparse layout based graph. Predictions from the model provide hints of bidimensional relations among text lines and layout region structures, upon which a post-processing cluster-and-sort algorithm generates an ordered sequence of all the text lines. The model is language-agnostic and runs effectively across multi-language datasets that contain various types of images taken in uncontrolled conditions, and it is small enough to be deployed on virtually any platform including mobile devices.
Chain Rules for Renyi Information Combining
Venue: arXiv cs.IT 2023
Year: 2023

Bounds on information combining are a fundamental tool in coding theory, in particular when analyzing polar codes and belief propagation. They usually bound the evolution of random variables with respect to their Shannon entropy. In recent work this approach was generalized to Renyi $α$-entropies. However, due to the lack of a traditional chain rule for Renyi entropies the picture remained incomplete. In this work we establish the missing link by providing Renyi chain rules connecting different definitions of Renyi entropies by Hayashi and Arimoto. This allows us to provide new information combining bounds for the Arimoto Renyi entropy. In the second part, we generalize the chain rule to the quantum setting and show how they allow us to generalize results and conjectures previously only given for the von Neumann entropy. In the special case of $α=2$ we give the first optimal information combining bounds with quantum side information.
Design and Detection of Unitary Constellations in Non-Coherent SIMO Systems for Short Packet Communications
Venue: arXiv cs.IT 2023
Year: 2023

This paper proposes a novel design of multi-symbol unitary constellation for non-coherent single-input multiple-output (SIMO) communications over block Rayleigh fading channels. To facilitate the design and the detection of large unitary constellations at reduced complexity, the proposed constellations are constructed as the Cartesian product of independent amplitude and phase-shift-keying (PSK) vectors, and hence, can be iteratively detected. The amplitude vector is detected by exhaustive search, whose complexity is sufficiently low in short packet transmission scenarios. To detect the PSK vector, we use the posterior probability as a reliability criterion in the sorted decision-feedback differential detection (sort-DFDD), which results in near-optimal error performance for PSK symbols with equal modulation orders. This detector is called posteriori-based-reliability-sort-DFDD (PR-sort-DFDD) and has polynomial complexity. We also propose an improved detector called improved-PR-sort-DFDD to detect a more generalized PSK structure, i.e., PSK symbols with unequal modulation orders. This detector also approaches the optimal error performance with polynomial complexity. Simulation results show the merits of our proposed multi-symbol unitary constellation when compared to competing low-complexity unitary constellations.
Neighboring Words Affect Human Interpretation of Saliency Explanations
Venue: arXiv cs.CL 2023
Year: 2023

Word-level saliency explanations ("heat maps over words") are often used to communicate feature-attribution in text-based models. Recent studies found that superficial factors such as word length can distort human interpretation of the communicated saliency scores. We conduct a user study to investigate how the marking of a word's neighboring words affect the explainee's perception of the word's importance in the context of a saliency explanation. We find that neighboring words have significant effects on the word's importance rating. Concretely, we identify that the influence changes based on neighboring direction (left vs. right) and a-priori linguistic and computational measures of phrases and collocations (vs. unrelated neighboring words). Our results question whether text-based saliency explanations should be continued to be communicated at word level, and inform future research on alternative saliency explanation methods.
Uncertainty Aware Deep Learning Model for Secure and Trustworthy Channel Estimation in 5G Networks
Venue: arXiv cs.CR 2023
Year: 2023

With the rise of intelligent applications, such as self-driving cars and augmented reality, the security and reliability of wireless communication systems have become increasingly crucial. One of the most critical components of ensuring a high-quality experience is channel estimation, which is fundamental for efficient transmission and interference management in wireless networks. However, using deep neural networks (DNNs) in channel estimation raises security and trust concerns due to their complexity and the need for more transparency in decision-making. This paper proposes a Monte Carlo Dropout (MCDO)-based approach for secure and trustworthy channel estimation in 5G networks. Our approach combines the advantages of traditional and deep learning techniques by incorporating conventional pilot-based channel estimation as a prior in the deep learning model. Additionally, we use MCDO to obtain uncertainty-aware predictions, enhancing the model's security and trustworthiness. Our experiments demonstrate that our proposed approach outperforms traditional and deep learning-based approaches regarding security, trustworthiness, and performance in 5G scenarios.
Towards End-to-End Semi-Supervised Table Detection with Deformable Transformer
Venue: arXiv cs.CV 2023
Year: 2023

Table detection is the task of classifying and localizing table objects within document images. With the recent development in deep learning methods, we observe remarkable success in table detection. However, a significant amount of labeled data is required to train these models effectively. Many semi-supervised approaches are introduced to mitigate the need for a substantial amount of label data. These approaches use CNN-based detectors that rely on anchor proposals and post-processing stages such as NMS. To tackle these limitations, this paper presents a novel end-to-end semi-supervised table detection method that employs the deformable transformer for detecting table objects. We evaluate our semi-supervised method on PubLayNet, DocBank, ICADR-19 and TableBank datasets, and it achieves superior performance compared to previous methods. It outperforms the fully supervised method (Deformable transformer) by +3.4 points on 10\% labels of TableBank-both dataset and the previous CNN-based semi-supervised approach (Soft Teacher) by +1.8 points on 10\% labels of PubLayNet dataset. We hope this work opens new possibilities towards semi-supervised and unsupervised table detection methods.
On the Parameterized Complexity of the Perfect Phylogeny Problem
Venue: arXiv cs.CC 2023
Year: 2023

This paper categorizes the parameterized complexity of the algorithmic problems Perfect Phylogeny and Triangulating Colored Graphs when parameterized by the number of genes and colors, respectively. We show that they are complete for the parameterized complexity class XALP using a reduction from Tree-chained Multicolor Independent Set and a proof of membership. We introduce the problem Triangulating Multicolored Graphs as a stepping stone and prove XALP-completeness for this problem as well. We also show that, assuming the Exponential Time Hypothesis, there exists no algorithm that solves any of these problems in time $f(k) n^{o(k)}$, where $n$ is the input size, $k$ the parameter, and $f$ any computable function.
MTLSegFormer: Multi-task Learning with Transformers for Semantic Segmentation in Precision Agriculture
Venue: arXiv cs.CV 2023
Year: 2023

Multi-task learning has proven to be effective in improving the performance of correlated tasks. Most of the existing methods use a backbone to extract initial features with independent branches for each task, and the exchange of information between the branches usually occurs through the concatenation or sum of the feature maps of the branches. However, this type of information exchange does not directly consider the local characteristics of the image nor the level of importance or correlation between the tasks. In this paper, we propose a semantic segmentation method, MTLSegFormer, which combines multi-task learning and attention mechanisms. After the backbone feature extraction, two feature maps are learned for each task. The first map is proposed to learn features related to its task, while the second map is obtained by applying learned visual attention to locally re-weigh the feature maps of the other tasks. In this way, weights are assigned to local regions of the image of other tasks that have greater importance for the specific task. Finally, the two maps are combined and used to solve a task. We tested the performance in two challenging problems with correlated tasks and observed a significant improvement in accuracy, mainly in tasks with high dependence on the others.
Strategic flip-flopping in political competition
Venue: arXiv cs.GT 2023
Year: 2023

We study candidates' positioning when adjustments are possible in response to new information about voters' preferences. Re-positioning allows candidates to get closer to the median voter but is costly both financially and electorally. We examine the occurrence and the direction of the adjustments depending on the ex-ante positions and the new information. In the unique subgame perfect equilibrium, candidates anticipate the possibility to adjust in response to future information and diverge ex-ante in order to secure a cost-less victory when the new information is favorable.
Maximum Causal Entropy Inverse Constrained Reinforcement Learning
Venue: arXiv cs.LG 2023
Year: 2023

When deploying artificial agents in real-world environments where they interact with humans, it is crucial that their behavior is aligned with the values, social norms or other requirements of that environment. However, many environments have implicit constraints that are difficult to specify and transfer to a learning agent. To address this challenge, we propose a novel method that utilizes the principle of maximum causal entropy to learn constraints and an optimal policy that adheres to these constraints, using demonstrations of agents that abide by the constraints. We prove convergence in a tabular setting and provide an approximation which scales to complex environments. We evaluate the effectiveness of the learned policy by assessing the reward received and the number of constraint violations, and we evaluate the learned cost function based on its transferability to other agents. Our method has been shown to outperform state-of-the-art approaches across a variety of tasks and environments, and it is able to handle problems with stochastic dynamics and a continuous state-action space.
Input Layer Binarization with Bit-Plane Encoding
Venue: arXiv cs.LG 2023
Year: 2023

Binary Neural Networks (BNNs) use 1-bit weights and activations to efficiently execute deep convolutional neural networks on edge devices. Nevertheless, the binarization of the first layer is conventionally excluded, as it leads to a large accuracy loss. The few works addressing the first layer binarization, typically increase the number of input channels to enhance data representation; such data expansion raises the amount of operations needed and it is feasible only on systems with enough computational resources. In this work, we present a new method to binarize the first layer using directly the 8-bit representation of input data; we exploit the standard bit-planes encoding to extract features bit-wise (using depth-wise convolutions); after a re-weighting stage, features are fused again. The resulting model is fully binarized and our first layer binarization approach is model independent. The concept is evaluated on three classification datasets (CIFAR10, SVHN and CIFAR100) for different model architectures (VGG and ResNet) and, the proposed technique outperforms state of the art methods both in accuracy and BMACs reduction.
Flow Correlator: A Flow Table Cache Management Strategy
Venue: arXiv cs.NI 2023
Year: 2023

Switching, routing, and security functions are the backbone of packet processing networks. Fast and efficient processing of packets requires maintaining the state of a large number of transient network connections. In particular, modern stateful firewalls, security monitoring devices, and software-defined networking (SDN) programmable dataplanes require maintaining stateful flow tables. These flow tables often grow much larger than can be expected to fit within on-chip memory, requiring a managed caching layer to maintain performance. This paper focuses on improving the efficiency of caching, an important architectural component of the packet processing data planes. We present a novel predictive approach to network flow table cache management. Our approach leverages a Hashed Perceptron binary classifier as well as an iterative approach to feature selection and ranking to improve the reliability and performance of the data plane caches. We validate the efficiency of the proposed techniques through extensive experimentation using real-world data sets. Our numerical results demonstrate that our techniques improve the reliability and performance of flow-centric packet processing architectures.
On the Closed-form Weight Enumeration of Polar Codes: 1.5$d$-weight Codewords
Venue: arXiv cs.IT 2023
Year: 2023

The weight distribution of error correction codes is a critical determinant of their error-correcting performance, making enumeration of utmost importance. In the case of polar codes, the minimum weight $\wm$ (which is equal to minimum distance $d$) is the only weight for which an explicit enumerator formula is currently available. Having closed-form weight enumerators for polar codewords with weights greater than the minimum weight not only simplifies the enumeration process but also provides valuable insights towards constructing better polar-like codes. In this paper, we contribute towards understanding the algebraic structure underlying higher weights by analyzing Minkowski sums of orbits. Our approach builds upon the lower triangular affine (LTA) group of decreasing monomial codes. Specifically, we propose a closed-form expression for the enumeration of codewords with weight $1.5\wm$. Our simulations demonstrate the potential for extending this method to higher weights.
NatCS: Eliciting Natural Customer Support Dialogues
Venue: arXiv cs.CL 2023
Year: 2023

Despite growing interest in applications based on natural customer support conversations, there exist remarkably few publicly available datasets that reflect the expected characteristics of conversations in these settings. Existing task-oriented dialogue datasets, which were collected to benchmark dialogue systems mainly in written human-to-bot settings, are not representative of real customer support conversations and do not provide realistic benchmarks for systems that are applied to natural data. To address this gap, we introduce NatCS, a multi-domain collection of spoken customer service conversations. We describe our process for collecting synthetic conversations between customers and agents based on natural language phenomena observed in real conversations. Compared to previous dialogue datasets, the conversations collected with our approach are more representative of real human-to-human conversations along multiple metrics. Finally, we demonstrate potential uses of NatCS, including dialogue act classification and intent induction from conversations as potential applications, showing that dialogue act annotations in NatCS provide more effective training data for modeling real conversations compared to existing synthetic written datasets. We publicly release NatCS to facilitate research in natural dialog systems
OctFormer: Octree-based Transformers for 3D Point Clouds
Venue: arXiv cs.CV 2023
Year: 2023

We propose octree-based transformers, named OctFormer, for 3D point cloud learning. OctFormer can not only serve as a general and effective backbone for 3D point cloud segmentation and object detection but also have linear complexity and is scalable for large-scale point clouds. The key challenge in applying transformers to point clouds is reducing the quadratic, thus overwhelming, computation complexity of attentions. To combat this issue, several works divide point clouds into non-overlapping windows and constrain attentions in each local window. However, the point number in each window varies greatly, impeding the efficient execution on GPU. Observing that attentions are robust to the shapes of local windows, we propose a novel octree attention, which leverages sorted shuffled keys of octrees to partition point clouds into local windows containing a fixed number of points while permitting shapes of windows to change freely. And we also introduce dilated octree attention to expand the receptive field further. Our octree attention can be implemented in 10 lines of code with open-sourced libraries and runs 17 times faster than other point cloud attentions when the point number exceeds 200k. Built upon the octree attention, OctFormer can be easily scaled up and achieves state-of-the-art performances on a series of 3D segmentation and detection benchmarks, surpassing previous sparse-voxel-based CNNs and point cloud transformers in terms of both efficiency and effectiveness. Notably, on the challenging ScanNet200 dataset, OctFormer outperforms sparse-voxel-based CNNs by 7.3 in mIoU. Our code and trained models are available at https://wang-ps.github.io/octformer.
New Adversarial Image Detection Based on Sentiment Analysis
Venue: arXiv cs.CR 2023
Year: 2023

Deep Neural Networks (DNNs) are vulnerable to adversarial examples, while adversarial attack models, e.g., DeepFool, are on the rise and outrunning adversarial example detection techniques. This paper presents a new adversarial example detector that outperforms state-of-the-art detectors in identifying the latest adversarial attacks on image datasets. Specifically, we propose to use sentiment analysis for adversarial example detection, qualified by the progressively manifesting impact of an adversarial perturbation on the hidden-layer feature maps of a DNN under attack. Accordingly, we design a modularized embedding layer with the minimum learnable parameters to embed the hidden-layer feature maps into word vectors and assemble sentences ready for sentiment analysis. Extensive experiments demonstrate that the new detector consistently surpasses the state-of-the-art detection algorithms in detecting the latest attacks launched against ResNet and Inception neutral networks on the CIFAR-10, CIFAR-100 and SVHN datasets. The detector only has about 2 million parameters, and takes shorter than 4.6 milliseconds to detect an adversarial example generated by the latest attack models using a Tesla K80 GPU card.