OpenRaiser/Intern-Atlas huggingface dataset:
Causal Influences Decouple From Their Underlying Network Structure In Echo State Networks
Venue: arXiv cs.LG 2022
Year: 2022

Echo State Networks (ESN) are versatile recurrent neural network models in which the hidden layer remains unaltered during training. Interactions among nodes of this static backbone produce diverse representations of the given stimuli that are harnessed by a read-out mechanism to perform computations needed for solving a given task. ESNs are accessible models of neuronal circuits, since they are relatively inexpensive to train. Therefore, ESNs have become attractive for neuroscientists studying the relationship between neural structure, function, and behavior. For instance, it is not yet clear how distinctive connectivity patterns of brain networks support effective interactions among their nodes and how these patterns of interactions give rise to computation. To address this question, we employed an ESN with a biologically inspired structure and used a systematic multi-site lesioning framework to quantify the causal contribution of each node to the network's output, thus providing a causal link between network structure and behavior. We then focused on the structure-function relationship and decomposed the causal influence of each node on all other nodes, using the same lesioning framework. We found that nodes in a properly engineered ESN interact largely irrespective of the network's underlying structure. However, in a network with the same topology and a non-optimal parameter set, the underlying connectivity patterns determine the node interactions. Our results suggest that causal structure-function relations in ESNs can be decomposed into two components, direct and indirect interactions. The former are based on influences relying on structural connections. The latter describe the effective communication between any two nodes through other intermediate nodes. These widely distributed indirect interactions may crucially contribute to the efficient performance of ESNs.
SHARP: Shape-Aware Reconstruction of People in Loose Clothing
Venue: arXiv cs.CV 2022
Year: 2022

Recent advancements in deep learning have enabled 3D human body reconstruction from a monocular image, which has broad applications in multiple domains. In this paper, we propose SHARP (SHape Aware Reconstruction of People in loose clothing), a novel end-to-end trainable network that accurately recovers the 3D geometry and appearance of humans in loose clothing from a monocular image. SHARP uses a sparse and efficient fusion strategy to combine parametric body prior with a non-parametric 2D representation of clothed humans. The parametric body prior enforces geometrical consistency on the body shape and pose, while the non-parametric representation models loose clothing and handle self-occlusions as well. We also leverage the sparseness of the non-parametric representation for faster training of our network while using losses on 2D maps. Another key contribution is 3DHumans, our new life-like dataset of 3D human body scans with rich geometrical and textural details. We evaluate SHARP on 3DHumans and other publicly available datasets and show superior qualitative and quantitative performance than existing state-of-the-art methods.
Exploiting Dynamic and Fine-grained Semantic Scope for Extreme Multi-label Text Classification
Venue: arXiv cs.AI 2022
Year: 2022

Extreme multi-label text classification (XMTC) refers to the problem of tagging a given text with the most relevant subset of labels from a large label set. A majority of labels only have a few training instances due to large label dimensionality in XMTC. To solve this data sparsity issue, most existing XMTC methods take advantage of fixed label clusters obtained in early stage to balance performance on tail labels and head labels. However, such label clusters provide static and coarse-grained semantic scope for every text, which ignores distinct characteristics of different texts and has difficulties modelling accurate semantics scope for texts with tail labels. In this paper, we propose a novel framework TReaderXML for XMTC, which adopts dynamic and fine-grained semantic scope from teacher knowledge for individual text to optimize text conditional prior category semantic ranges. TReaderXML dynamically obtains teacher knowledge for each text by similar texts and hierarchical label information in training sets to release the ability of distinctly fine-grained label-oriented semantic scope. Then, TReaderXML benefits from a novel dual cooperative network that firstly learns features of a text and its corresponding label-oriented semantic scope by parallel Encoding Module and Reading Module, secondly embeds two parts by Interaction Module to regularize the text's representation by dynamic and fine-grained label-oriented semantic scope, and finally find target labels by Prediction Module. Experimental results on three XMTC benchmark datasets show that our method achieves new state-of-the-art results and especially performs well for severely imbalanced and sparse datasets.
Analysing the Greek Parliament Records with Emotion Classification
Venue: arXiv cs.CL 2022
Year: 2022

In this project, we tackle emotion classification for the Greek language, presenting and releasing a new dataset in Greek. We fine-tune and assess Transformer-based masked language models that were pre-trained on monolingual and multilingual resources, and we present the results per emotion and by aggregating at the sentiment and subjectivity level. The potential of the presented resources is investigated by detecting and studying the emotion of `disgust' in the Greek Parliament records. We: (a) locate the months with the highest values from 1989 to present, (b) rank the Greek political parties based on the presence of this emotion in their speeches, and (c) study the emotional context shift of words used to stigmatise people.
Generalized Multi-cluster Game under Partial-decision Information with Applications to Management of Energy Internet
Venue: arXiv cs.GT 2022
Year: 2022

The decision making and management of many engineering networks involves multiple parties with conflicting interests, while each party is constituted with multiple agents. Such problems can be casted as a multi-cluster game. Each cluster is treated as a self-interested player in a non-cooperative game where agents in the same cluster cooperate together to optimize the payoff function of the cluster. In a large-scale network, the information of agents in a cluster can not be available immediately for agents beyond this cluster, which raise challenges to the existing Nash equilibrium seeking algorithms. Hence, we consider a partial-decision information scenario in generalized Nash equilibrium seeking for multi-cluster games in a distributed manner. We reformulate the problem as finding zeros of the sum of preconditioned monotone operators by the primal-dual analysis and graph Laplacian matrix. Then a distributed generalized Nash equilibrium seeking algorithm is proposed without requiring fully awareness of its opponent clusters' decisions based on a forward-backward-forward method. With the algorithm, each agent estimates the strategies of all the other clusters by communicating with neighbors via an undirected network. We show that the derived operators can be monotone when the communication strength parameter is sufficiently large. We prove the algorithm convergence resorting to the fixed point theory by providing a sufficient condition. We discuss its potential application in Energy Internet with numerical studies.
DNNAbacus: Toward Accurate Computational Cost Prediction for Deep Neural Networks
Venue: arXiv cs.LG 2022
Year: 2022

Deep learning is attracting interest across a variety of domains, including natural language processing, speech recognition, and computer vision. However, model training is time-consuming and requires huge computational resources. Existing works on the performance prediction of deep neural networks, which mostly focus on the training time prediction of a few models, rely on analytical models and result in high relative errors. %Optimizing task scheduling and reducing job failures in data centers are essential to improve resource utilization and reduce carbon emissions. This paper investigates the computational resource demands of 29 classical deep neural networks and builds accurate models for predicting computational costs. We first analyze the profiling results of typical networks and demonstrate that the computational resource demands of models with different inputs and hyperparameters are not obvious and intuitive. We then propose a lightweight prediction approach DNNAbacus with a novel network structural matrix for network representation. DNNAbacus can accurately predict both memory and time cost for PyTorch and TensorFlow models, which is also generalized to different hardware architectures and can have zero-shot capability for unseen networks. Our experimental results show that the mean relative error (MRE) is 0.9% with respect to time and 2.8% with respect to memory for 29 classic models, which is much lower than the state-of-the-art works.
Reliability Assessment of Neural Networks in GPUs: A Framework For Permanent Faults Injections
Venue: arXiv cs.NE 2022
Year: 2022

Currently, Deep learning and especially Convolutional Neural Networks (CNNs) have become a fundamental computational approach applied in a wide range of domains, including some safety-critical applications (e.g., automotive, robotics, and healthcare equipment). Therefore, the reliability evaluation of those computational systems is mandatory. The reliability evaluation of CNNs is performed by fault injection campaigns at different levels of abstraction, from the application level down to the hardware level. Many works have focused on evaluating the reliability of neural networks in the presence of transient faults. However, the effects of permanent faults have been investigated at the application level, only, e.g., targeting the parameters of the network. This paper intends to propose a framework, resorting to a binary instrumentation tool to perform fault injection campaigns, targeting different components inside the GPU, such as the register files and the functional units. This environment allows for the first time assessing the reliability of CNNs deployed on a GPU considering the presence of permanent faults.
Evidential Temporal-aware Graph-based Social Event Detection via Dempster-Shafer Theory
Venue: arXiv cs.AI 2022
Year: 2022

The rising popularity of online social network services has attracted lots of research on mining social media data, especially on mining social events. Social event detection, due to its wide applications, has now become a trivial task. State-of-the-art approaches exploiting Graph Neural Networks (GNNs) usually follow a two-step strategy: 1) constructing text graphs based on various views (\textit{co-user}, \textit{co-entities} and \textit{co-hashtags}); and 2) learning a unified text representation by a specific GNN model. Generally, the results heavily rely on the quality of the constructed graphs and the specific message passing scheme. However, existing methods have deficiencies in both aspects: 1) They fail to recognize the noisy information induced by unreliable views. 2) Temporal information which works as a vital indicator of events is neglected in most works. To this end, we propose ETGNN, a novel Evidential Temporal-aware Graph Neural Network. Specifically, we construct view-specific graphs whose nodes are the texts and edges are determined by several types of shared elements respectively. To incorporate temporal information into the message passing scheme, we introduce a novel temporal-aware aggregator which assigns weights to neighbours according to an adaptive time exponential decay formula. Considering the view-specific uncertainty, the representations of all views are converted into mass functions through evidential deep learning (EDL) neural networks, and further combined via Dempster-Shafer theory (DST) to make the final detection. Experimental results on three real-world datasets demonstrate the effectiveness of ETGNN in accuracy, reliability and robustness in social event detection.
Capacity Bounds for Optical WDM Channels with Peak Power Constraints
Venue: arXiv cs.IT 2022
Year: 2022

We investigate optical WDM transmission from the standpoint of an information-theoretic interference channel. Achievable rates that outperform treating interference as noise are presented and validated using split-step Fourier method simulations.
Forecasting Multilinear Data via Transform-Based Tensor Autoregression
Venue: arXiv cs.LG 2022
Year: 2022

In the era of big data, there is an increasing demand for new methods for analyzing and forecasting 2-dimensional data. The current research aims to accomplish these goals through the combination of time-series modeling and multilinear algebraic systems. We expand previous autoregressive techniques to forecast multilinear data, aptly named the L-Transform Tensor autoregressive (L-TAR for short). Tensor decompositions and multilinear tensor products have allowed for this approach to be a feasible method of forecasting. We achieve statistical independence between the columns of the observations through invertible discrete linear transforms, enabling a divide and conquer approach. We present an experimental validation of the proposed methods on datasets containing image collections, video sequences, sea surface temperature measurements, stock prices, and networks.
PoeLM: A Meter- and Rhyme-Controllable Language Model for Unsupervised Poetry Generation
Venue: arXiv cs.CL 2022
Year: 2022

Formal verse poetry imposes strict constraints on the meter and rhyme scheme of poems. Most prior work on generating this type of poetry uses existing poems for supervision, which are difficult to obtain for most languages and poetic forms. In this work, we propose an unsupervised approach to generate poems following any given meter and rhyme scheme, without requiring any poetic text for training. Our method works by splitting a regular, non-poetic corpus into phrases, prepending control codes that describe the length and end rhyme of each phrase, and training a transformer language model in the augmented corpus. During inference, we build control codes for the desired meter and rhyme scheme, and condition our language model on them to generate formal verse poetry. Experiments in Spanish and Basque show that our approach is able to generate valid poems, which are often comparable in quality to those written by humans.
Psychotic Relapse Prediction in Schizophrenia Patients using A Mobile Sensing-based Supervised Deep Learning Model
Venue: arXiv cs.LG 2022
Year: 2022

Mobile sensing-based modeling of behavioral changes could predict an oncoming psychotic relapse in schizophrenia patients for timely interventions. Deep learning models could complement existing non-deep learning models for relapse prediction by modeling latent behavioral features relevant to the prediction. However, given the inter-individual behavioral differences, model personalization might be required for a predictive model. In this work, we propose RelapsePredNet, a Long Short-Term Memory (LSTM) neural network-based model for relapse prediction. The model is personalized for a particular patient by training using data from patients most similar to the given patient. Several demographics and baseline mental health scores were considered as personalization metrics to define patient similarity. We investigated the effect of personalization on training dataset characteristics, learned embeddings, and relapse prediction performance. We compared RelapsePredNet with a deep learning-based anomaly detection model for relapse prediction. Further, we investigated if RelapsePredNet could complement ClusterRFModel (a random forest model leveraging clustering and template features proposed in prior work) in a fusion model, by identifying latent behavioral features relevant for relapse prediction. The CrossCheck dataset consisting of continuous mobile sensing data obtained from 63 schizophrenia patients, each monitored for up to a year, was used for our evaluations. The proposed RelapsePredNet outperformed the deep learning-based anomaly detection model for relapse prediction. The F2 score for prediction were 0.21 and 0.52 in the full test set and the Relapse Test Set (consisting of data from patients who have had relapse only), respectively. These corresponded to a 29.4% and 38.8% improvement compared to the existing deep learning-based model for relapse prediction.
The Economic Viability of an In-Home Monitoring System in the context of an Aged Care Setting
Venue: arXiv cs.HC 2022
Year: 2022

The aged care sector in Australia faces significant challenges. While many of these issues have been clearly identified, their urgency has been further highlighted during the COVID-19 pandemic. Technology such as in-home monitoring is one way to address some of these challenges. However, the efficacy of technology must be considered together with its implementation and running costs to ensure that there is a return on investment, and it is economically viable as a solution. A pilot program was run in-home monitoring system to test the efficacy of this system. This paper focuses on an economic analysis to better understand the financial viability of such systems. Using a secondary analysis approach, the findings identified that revenue could be generated by providing carers with additional services such as real-time monitoring of the client, which can foster deeper relationships with the customer, along with savings of healthcare costs to carers, service providers and Government. Savings are related to the earlier intervention of critical events that are identified by the system, as delays in treatment of some critical events can create much more severe and costly health outcomes. Further health costs savings can be made via trend analysis which can show more nuanced health deterioration that is often missed. The implementation of preventative measures via this identification can reduce the chances of critical events occurring which have much higher costs. Overall, monitoring systems lead to a transition from a reactive to a preventative services offering, delivering more targeted and personalised care.
Multilevel sentiment analysis in arabic
Venue: arXiv cs.CL 2022
Year: 2022

In this study, we aimed to improve the performance results of Arabic sentiment analysis. This can be achieved by investigating the most successful machine learning method and the most useful feature vector to classify sentiments in both term and document levels into two (positive or negative) categories. Moreover, specification of one polarity degree for the term that has more than one is investigated. Also to handle the negations and intensifications, some rules are developed. According to the obtained results, Artificial Neural Network classifier is nominated as the best classifier in both term and document level sentiment analysis (SA) for Arabic Language. Furthermore, the average F-score achieved in the term level SA for both positive and negative testing classes is 0.92. In the document level SA, the average F-score for positive testing classes is 0.94, while for negative classes is 0.93.
Image Based Password Authentication System
Venue: arXiv cs.CR 2022
Year: 2022

Preservation of information and computer security is broadly dependent on the secured authentication system which is underpinned by password. Text based password is a commonly used and available system for authentication. But it bears many limitations like shoulder surfing, dictionary attack, Phishing, guessing the password etc. In order to overwhelm these vulnerabilities of ancient textual password, many graphical or image based password authentication system has been introduced form last few years. But none of this graphical system is considered as enough adventurous to keep pace with these issues. Here we have proposed an image based password authentication system which is more methodical and can cope up with every vulnerability of recent password authentication system. To make our system hassle free and more reliable, we will only take username from an user for registration purpose as our system will generate a unique key number for that particular user and this key will be used as password for later login procedure. The user name and key both will be encrypted using a cryptography algorithm to prevent database hacking. There will be a randomized clickable image grid in our system. By clicking on this image grid, user will input the password key for login purpose. Here we have developed another method namely shoulder surfing resistant password. To prevent the attack of shoulder surfing, if any user wishes to change our system provided password key then he or she is allowed to do so by using this method. Besides this method allows user to change the password every single time of login. A user doesn't need to enter any textual password for authentication in our recent module and hence combination of all these features improve the security, usability and user friendliness of our system.
recommenderlab: An R Framework for Developing and Testing Recommendation Algorithms
Venue: arXiv cs.IR 2022
Year: 2022

Algorithms that create recommendations based on observed data have significant commercial value for online retailers and many other industries. Recommender systems have a significant research community, and studying such systems is part of most modern data science curricula. While there is an abundance of software that implements recommendation algorithms, there is little in terms of supporting recommender system research and education. This paper describes the open-source software recommenderlab which was created with supporting research and education in mind. The package can be directly installed in R or downloaded from https://github.com/mhahsler/recommenderlab.
TorchNTK: A Library for Calculation of Neural Tangent Kernels of PyTorch Models
Venue: arXiv cs.LG 2022
Year: 2022

We introduce torchNTK, a python library to calculate the empirical neural tangent kernel (NTK) of neural network models in the PyTorch framework. We provide an efficient method to calculate the NTK of multilayer perceptrons. We compare the explicit differentiation implementation against autodifferentiation implementations, which have the benefit of extending the utility of the library to any architecture supported by PyTorch, such as convolutional networks. A feature of the library is that we expose the user to layerwise NTK components, and show that in some regimes a layerwise calculation is more memory efficient. We conduct preliminary experiments to demonstrate use cases for the software and probe the NTK.