OpenRaiser/Intern-Atlas huggingface dataset:
PiML Toolbox for Interpretable Machine Learning Model Development and Diagnostics
Venue: arXiv cs.LG 2023
Year: 2023

PiML (read $π$-ML, /`pai`em`el/) is an integrated and open-access Python toolbox for interpretable machine learning model development and model diagnostics. It is designed with machine learning workflows in both low-code and high-code modes, including data pipeline, model training and tuning, model interpretation and explanation, and model diagnostics and comparison. The toolbox supports a growing list of interpretable models (e.g. GAM, GAMI-Net, XGB1/XGB2) with inherent local and/or global interpretability. It also supports model-agnostic explainability tools (e.g. PFI, PDP, LIME, SHAP) and a powerful suite of model-agnostic diagnostics (e.g. weakness, reliability, robustness, resilience, fairness). Integration of PiML models and tests to existing MLOps platforms for quality assurance are enabled by flexible high-code APIs. Furthermore, PiML toolbox comes with a comprehensive user guide and hands-on examples, including the applications for model development and validation in banking. The project is available at https://github.com/SelfExplainML/PiML-Toolbox.
6G for Connected Sky: A Vision for Integrating Terrestrial and Non-Terrestrial Networks
Venue: arXiv cs.NI 2023
Year: 2023

In this paper, we present the vision of our project 6G for Connected Sky (6G-SKY) to integrate terrestrial networks (TNs) and non-terrestrial networks (NTNs) and outline the current research activities in 6G research projects in comparison with our project. From the perspectives of industry and academia, we identify key use case segments connecting both aerial and ground users with our 6G-SKY multi-layer network architecture. We explain functional views of our holistic 6G-SKY architecture addressing the heterogeneity of aerial and space platforms. Architecture elements and communication links are identified. We discuss 6G-SKY network design and management functionalities by considering a set of inherent challenges posed by the multi-layer 3-dimensional networks, which we termed as combined airspace and NTN (combined ASN). Finally, we investigate additional research challenges for 6G-SKY project targets.
RSC-VAE: Recoding Semantic Consistency Based VAE for One-Class Novelty Detection
Venue: arXiv cs.CV 2023
Year: 2023

In recent years, there is an increasing interests in reconstruction based generative models for image One-Class Novelty Detection, most of which only focus on image-level information. While in this paper, we further exploit the latent space of Variational Auto-encoder (VAE), a typical reconstruction based model, and we innovatively divide it into three regions: Normal/Anomalous/Unknown-semantic-region. Based on this hypothesis, we propose a new VAE architecture, Recoding Semantic Consistency Based VAE (RSC-VAE), combining VAE with recoding mechanism and constraining the semantic consistency of two encodings. We come up with three training modes of RSC-VAE: 1. One-Class Training Mode, alleviating False Positive problem of normal samples; 2. Distributionally-Shifted Training Mode, alleviating False Negative problem of anomalous samples; 3. Extremely-Imbalanced Training Mode, introducing a small number of anomalous samples for training to enhance the second mode. The experimental results on multiple datasets demonstrate that our mechanism achieves state-of-the-art performance in various baselines including VAE.
HIORE: Leveraging High-order Interactions for Unified Entity Relation Extraction
Venue: arXiv cs.CL 2023
Year: 2023

Entity relation extraction consists of two sub-tasks: entity recognition and relation extraction. Existing methods either tackle these two tasks separately or unify them with word-by-word interactions. In this paper, we propose HIORE, a new method for unified entity relation extraction. The key insight is to leverage the high-order interactions, i.e., the complex association among word pairs, which contains richer information than the first-order word-by-word interactions. For this purpose, we first devise a W-shape DNN (WNet) to capture coarse-level high-order connections. Then, we build a heuristic high-order graph and further calibrate the representations with a graph neural network (GNN). Experiments on three benchmarks (ACE04, ACE05, SciERC) show that HIORE achieves the state-of-the-art performance on relation extraction and an improvement of 1.1~1.8 F1 points over the prior best unified model.
On the perceived relevance of critical internal quality attributes when evolving software features
Venue: arXiv cs.SE 2023
Year: 2023

Several refactorings performed while evolving software features aim to improve internal quality attributes like cohesion and complexity. Indeed, internal attributes can become critical if their measurements assume anomalous values. Yet, current knowledge is scarce on how developers perceive the relevance of critical internal attributes while evolving features. This qualitative study investigates the developers' perception of the relevance of critical internal attributes when evolving features. We target six class-level critical attributes: low cohesion, high complexity, high coupling, large hierarchy depth, large hierarchy breadth, and large size. We performed two industrial case studies based on online focus group sessions. Developers discussed how much (and why) critical attributes are relevant when adding or enhancing features. We assessed the relevance of critical attributes individually and relatively, the reasons behind the relevance of each critical attribute, and the interrelations of critical attributes. Low cohesion and high complexity were perceived as very relevant because they often make evolving features hard while tracking failures and adding features. The other critical attributes were perceived as less relevant when reusing code or adopting design patterns. An example of perceived interrelation is high complexity leading to high coupling.
Contrastive Enhanced Slide Filter Mixer for Sequential Recommendation
Venue: arXiv cs.IR 2023
Year: 2023

Sequential recommendation (SR) aims to model user preferences by capturing behavior patterns from their item historical interaction data. Most existing methods model user preference in the time domain, omitting the fact that users' behaviors are also influenced by various frequency patterns that are difficult to separate in the entangled chronological items. However, few attempts have been made to train SR in the frequency domain, and it is still unclear how to use the frequency components to learn an appropriate representation for the user. To solve this problem, we shift the viewpoint to the frequency domain and propose a novel Contrastive Enhanced \textbf{SLI}de Filter \textbf{M}ixEr for Sequential \textbf{Rec}ommendation, named \textbf{SLIME4Rec}. Specifically, we design a frequency ramp structure to allow the learnable filter slide on the frequency spectrums across different layers to capture different frequency patterns. Moreover, a Dynamic Frequency Selection (DFS) and a Static Frequency Split (SFS) module are proposed to replace the self-attention module for effectively extracting frequency information in two ways. DFS is used to select helpful frequency components dynamically, and SFS is combined with the dynamic frequency selection module to provide a more fine-grained frequency division. Finally, contrastive learning is utilized to improve the quality of user embedding learned from the frequency domain. Extensive experiments conducted on five widely used benchmark datasets demonstrate our proposed model performs significantly better than the state-of-the-art approaches. Our code is available at https://github.com/sudaada/SLIME4Rec.
FACTIFY-5WQA: 5W Aspect-based Fact Verification through Question Answering
Venue: arXiv cs.CL 2023
Year: 2023

Automatic fact verification has received significant attention recently. Contemporary automatic fact-checking systems focus on estimating truthfulness using numerical scores which are not human-interpretable. A human fact-checker generally follows several logical steps to verify a verisimilitude claim and conclude whether its truthful or a mere masquerade. Popular fact-checking websites follow a common structure for fact categorization such as half true, half false, false, pants on fire, etc. Therefore, it is necessary to have an aspect-based (delineating which part(s) are true and which are false) explainable system that can assist human fact-checkers in asking relevant questions related to a fact, which can then be validated separately to reach a final verdict. In this paper, we propose a 5W framework (who, what, when, where, and why) for question-answer-based fact explainability. To that end, we present a semi-automatically generated dataset called FACTIFY-5WQA, which consists of 391, 041 facts along with relevant 5W QAs - underscoring our major contribution to this paper. A semantic role labeling system has been utilized to locate 5Ws, which generates QA pairs for claims using a masked language model. Finally, we report a baseline QA system to automatically locate those answers from evidence documents, which can serve as a baseline for future research in the field. Lastly, we propose a robust fact verification system that takes paraphrased claims and automatically validates them. The dataset and the baseline model are available at https: //github.com/ankuranii/acl-5W-QA
Interpreting Training Aspects of Deep-Learned Error-Correcting Codes
Venue: arXiv cs.IT 2023
Year: 2023

As new deep-learned error-correcting codes continue to be introduced, it is important to develop tools to interpret the designed codes and understand the training process. Prior work focusing on the deep-learned TurboAE has both interpreted the learned encoders post-hoc by mapping these onto nearby ``interpretable'' encoders, and experimentally evaluated the performance of these interpretable encoders with various decoders. Here we look at developing tools for interpreting the training process for deep-learned error-correcting codes, focusing on: 1) using the Goldreich-Levin algorithm to quickly interpret the learned encoder; 2) using Fourier coefficients as a tool for understanding the training dynamics and the loss landscape; 3) reformulating the training loss, the binary cross entropy, by relating it to encoder and decoder parameters, and the bit error rate (BER); 4) using these insights to formulate and study a new training procedure. All tools are demonstrated on TurboAE, but are applicable to other deep-learned forward error correcting codes (without feedback).
Documenting Bioinformatics Software Via Reverse Engineering
Venue: arXiv cs.SE 2023
Year: 2023

Documentation is one of the most neglected activities in Software Engineering, although it is an important method of assuring quality and understanding. Bioinformatics software is generally written by researchers from fields other than Computer Science who usually do not provide documentation. Documenting bioinformatics software may ease its adoption in multidisciplinary teams and expand its impact on the community. In this paper, we highlight how one can document software that is already finished, using reverse engineering and thinking of the end-user.
A Generalized Framework for Predictive Clustering and Optimization
Venue: arXiv cs.LG 2023
Year: 2023

Clustering is a powerful and extensively used data science tool. While clustering is generally thought of as an unsupervised learning technique, there are also supervised variations such as Spath's clusterwise regression that attempt to find clusters of data that yield low regression error on a supervised target. We believe that clusterwise regression is just a single vertex of a largely unexplored design space of supervised clustering models. In this article, we define a generalized optimization framework for predictive clustering that admits different cluster definitions (arbitrary point assignment, closest center, and bounding box) and both regression and classification objectives. We then present a joint optimization strategy that exploits mixed-integer linear programming (MILP) for global optimization in this generalized framework. To alleviate scalability concerns for large datasets, we also provide highly scalable greedy algorithms inspired by the Majorization-Minimization (MM) framework. Finally, we demonstrate the ability of our models to uncover different interpretable discrete cluster structures in data by experimenting with four real-world datasets.
TaLU: A Hybrid Activation Function Combining Tanh and Rectified Linear Unit to Enhance Neural Networks
Venue: arXiv cs.CV 2023
Year: 2023

The application of the deep learning model in classification plays an important role in the accurate detection of the target objects. However, the accuracy is affected by the activation function in the hidden and output layer. In this paper, an activation function called TaLU, which is a combination of Tanh and Rectified Linear Units (ReLU), is used to improve the prediction. ReLU activation function is used by many deep learning researchers for its computational efficiency, ease of implementation, intuitive nature, etc. However, it suffers from a dying gradient problem. For instance, when the input is negative, its output is always zero because its gradient is zero. A number of researchers used different approaches to solve this issue. Some of the most notable are LeakyReLU, Softplus, Softsign, ELU, ThresholdedReLU, etc. This research developed TaLU, a modified activation function combining Tanh and ReLU, which mitigates the dying gradient problem of ReLU. The deep learning model with the proposed activation function was tested on MNIST and CIFAR-10, and it outperforms ReLU and some other studied activation functions in terms of accuracy(upto 6% in most cases, when used with Batch Normalization and a reasonable learning rate).
Unlocking Practical Applications in Legal Domain: Evaluation of GPT for Zero-Shot Semantic Annotation of Legal Texts
Venue: arXiv cs.CL 2023
Year: 2023

We evaluated the capability of a state-of-the-art generative pre-trained transformer (GPT) model to perform semantic annotation of short text snippets (one to few sentences) coming from legal documents of various types. Discussions of potential uses (e.g., document drafting, summarization) of this emerging technology in legal domain have intensified, but to date there has not been a rigorous analysis of these large language models' (LLM) capacity in sentence-level semantic annotation of legal texts in zero-shot learning settings. Yet, this particular type of use could unlock many practical applications (e.g., in contract review) and research opportunities (e.g., in empirical legal studies). We fill the gap with this study. We examined if and how successfully the model can semantically annotate small batches of short text snippets (10-50) based exclusively on concise definitions of the semantic types. We found that the GPT model performs surprisingly well in zero-shot settings on diverse types of documents (F1=.73 on a task involving court opinions, .86 for contracts, and .54 for statutes and regulations). These findings can be leveraged by legal scholars and practicing lawyers alike to guide their decisions in integrating LLMs in wide range of workflows involving semantic annotation of legal texts.
Goal-oriented inference of environment from redundant observations
Venue: arXiv cs.LG 2023
Year: 2023

The agent learns to organize decision behavior to achieve a behavioral goal, such as reward maximization, and reinforcement learning is often used for this optimization. Learning an optimal behavioral strategy is difficult under the uncertainty that events necessary for learning are only partially observable, called as Partially Observable Markov Decision Process (POMDP). However, the real-world environment also gives many events irrelevant to reward delivery and an optimal behavioral strategy. The conventional methods in POMDP, which attempt to infer transition rules among the entire observations, including irrelevant states, are ineffective in such an environment. Supposing Redundantly Observable Markov Decision Process (ROMDP), here we propose a method for goal-oriented reinforcement learning to efficiently learn state transition rules among reward-related "core states'' from redundant observations. Starting with a small number of initial core states, our model gradually adds new core states to the transition diagram until it achieves an optimal behavioral strategy consistent with the Bellman equation. We demonstrate that the resultant inference model outperforms the conventional method for POMDP. We emphasize that our model only containing the core states has high explainability. Furthermore, the proposed method suits online learning as it suppresses memory consumption and improves learning speed.
Examining the Factors of Place Sameness: A Classroom Re-creation Task in a Virtual Environment
Venue: arXiv cs.HC 2023
Year: 2023

Virtual re-creations of real-world places are attractive and becoming more popular. Attractiveness of re-created place is commonly determined by the degree of visual similarity to the original places. Even if the re-creation has poor similarity to the original, there are cases in which people recognize and allow the re-creation to have similarity as the same entity. However, the mechanisms and factors behind this contradiction have not yet been studied. This is the first study focusing on the activity and meaningofpeoplewhentheyfeelsamenesstoare-createdplace. Thispaper investigates the concept of place sameness, which is defined as the degree of similarity between an original and a virtual re-creation that represents the original place. Further, the factors of place sameness are explored using a classroom re-creation task with virtual reality (VR) technology. The participants were instructed to recreate two classrooms within virtual environmentsusingvirtualfixtures. Asresults, thedisplaydevicesinRoom B, the student desks in Room A at the memory index, display devices in RoomA,andstudentdesksandothersinRoomBattheaveragedistancehad moderate or large correlations with the place sameness index. The results suggest that the reproducibility of the number of objects related to activities in a place, and the inaccuracy of the positions of objects are factors of place sameness. Our own interpretation of the uncanny valley effect of a place was also partially observed. The main contributions of this study are the proposal of the concept of place sameness as a new perspective for virtual re-creation research and the finding of promising factors for that.
AnomalyBERT: Self-Supervised Transformer for Time Series Anomaly Detection using Data Degradation Scheme
Venue: arXiv cs.LG 2023
Year: 2023

Mechanical defects in real situations affect observation values and cause abnormalities in multivariate time series, such as sensor values or network data. To perceive abnormalities in such data, it is crucial to understand the temporal context and interrelation between variables simultaneously. The anomaly detection task for time series, especially for unlabeled data, has been a challenging problem, and we address it by applying a suitable data degradation scheme to self-supervised model training. We define four types of synthetic outliers and propose the degradation scheme in which a portion of input data is replaced with one of the synthetic outliers. Inspired by the self-attention mechanism, we design a Transformer-based architecture to recognize the temporal context and detect unnatural sequences with high efficiency. Our model converts multivariate data points into temporal representations with relative position bias and yields anomaly scores from these representations. Our method, AnomalyBERT, shows a great capability of detecting anomalies contained in complex time series and surpasses previous state-of-the-art methods on five real-world benchmarks. Our code is available at https://github.com/Jhryu30/AnomalyBERT.
Adaptive Learning Path Navigation Based on Knowledge Tracing and Reinforcement Learning
Venue: arXiv cs.AI 2023
Year: 2023

This paper introduces the Adaptive Learning Path Navigation (ALPN) system, a novel approach for enhancing E-learning platforms by providing highly adaptive learning paths for students. The ALPN system integrates the Attentive Knowledge Tracing (AKT) model, which assesses students' knowledge states, with the proposed Entropy-enhanced Proximal Policy Optimization (EPPO) algorithm. This new algorithm optimizes the recommendation of learning materials. By harmonizing these models, the ALPN system tailors the learning path to students' needs, significantly increasing learning effectiveness. Experimental results demonstrate that the ALPN system outperforms previous research by 8.2% in maximizing learning outcomes and provides a 10.5% higher diversity in generating learning paths. The proposed system marks a significant advancement in adaptive E-learning, potentially transforming the educational landscape in the digital era.
MO-DEHB: Evolutionary-based Hyperband for Multi-Objective Optimization
Venue: arXiv cs.LG 2023
Year: 2023

Hyperparameter optimization (HPO) is a powerful technique for automating the tuning of machine learning (ML) models. However, in many real-world applications, accuracy is only one of multiple performance criteria that must be considered. Optimizing these objectives simultaneously on a complex and diverse search space remains a challenging task. In this paper, we propose MO-DEHB, an effective and flexible multi-objective (MO) optimizer that extends the recent evolutionary Hyperband method DEHB. We validate the performance of MO-DEHB using a comprehensive suite of 15 benchmarks consisting of diverse and challenging MO problems, including HPO, neural architecture search (NAS), and joint NAS and HPO, with objectives including accuracy, latency and algorithmic fairness. A comparative study against state-of-the-art MO optimizers demonstrates that MO-DEHB clearly achieves the best performance across our 15 benchmarks.
Event Knowledge Incorporation with Posterior Regularization for Event-Centric Question Answering
Venue: arXiv cs.CL 2023
Year: 2023

We propose a simple yet effective strategy to incorporate event knowledge extracted from event trigger annotations via posterior regularization to improve the event reasoning capability of mainstream question-answering (QA) models for event-centric QA. In particular, we define event-related knowledge constraints based on the event trigger annotations in the QA datasets, and subsequently use them to regularize the posterior answer output probabilities from the backbone pre-trained language models used in the QA setting. We explore two different posterior regularization strategies for extractive and generative QA separately. For extractive QA, the sentence-level event knowledge constraint is defined by assessing if a sentence contains an answer event or not, which is later used to modify the answer span extraction probability. For generative QA, the token-level event knowledge constraint is defined by comparing the generated token from the backbone language model with the answer event in order to introduce a reward or penalty term, which essentially adjusts the answer generative probability indirectly. We conduct experiments on two event-centric QA datasets, TORQUE and ESTER. The results show that our proposed approach can effectively inject event knowledge into existing pre-trained language models and achieves strong performance compared to existing QA models in answer evaluation. Code and models can be found: https://github.com/LuJunru/EventQAviaPR.