Reviewerly/exHarmony huggingface dataset:
Adversarial examples for extreme multilabel text classification

Abstract Extreme Multilabel Text Classification (XMTC) is a text classification problem in which, (i) the output space is extremely large, (ii) each data point may have multiple positive labels, and (iii) the data follows a strongly imbalanced distribution. With applications in recommendation systems and automatic tagging of web-scale documents, the research on XMTC has been focused on improving prediction accuracy and dealing with imbalanced data. However, the robustness of deep learning based XMTC models against adversarial examples has been largely underexplored. In this paper, we investigate the behaviour of XMTC models under adversarial attacks. To this end, first, we define adversarial attacks in multilabel text classification problems. We categorize attacking multilabel text classifiers as (a) positive-to-negative, where the target positive label should fall out of top-k predicted labels, and (b) negative-to-positive, where the target negative label should be among the top-k predicted labels. Then, by experiments on APLC-XLNet and AttentionXML, we show that XMTC models are highly vulnerable to positive-to-negative attacks but more robust to negative-to-positive ones. Furthermore, our experiments show that the success rate of positive-to-negative adversarial attacks has an imbalanced distribution. More precisely, tail classes are highly vulnerable to adversarial attacks for which an attacker can generate adversarial samples with high similarity to the actual data-points. To overcome this problem, we explore the effect of rebalanced loss functions in XMTC where not only do they increase accuracy on tail classes, but they also improve the robustness of these classes against adversarial attacks. The code for our experiments is available at https://github.com/xmc-aalto/adv-xmtc .
Enriched entity representation of knowledge graph for text generation

Abstract Text generation is a key tool in natural language applications. Generating texts which could express rich ideas through several sentences needs a structured representation of their content. Many works utilize graph-based methods for graph-to-text generation, like knowledge-graph-to-text generation. However, generating texts from knowledge graph still faces problems, such as repetitions and the entity information is not fully utilized in the generated text. In this paper, we focus on knowledge-graph-to-text generation, and develop a multi-level entity fusion representation (MEFR) model to address the above problems, aiming to generate high-quality text from knowledge graph. Our model introduces a fusion mechanism, which is capable of aggregating node representations from word level and phrase level to obtain rich entity representations of the knowledge graph. Then, Graph Transformer is adopted to encode the graph and outputs contextualized node representations. Besides, we develop a vanilla beam search-based comparison mechanism during decoding procedure, which further considers similarity to reduce repetitive information of the generated text. Experimental results show that the proposed MEFR model could effectively improve generation performance, and outperform other baselines on AGENDA and WebNLG datasets. The results also demonstrate the importance to further explore information contained in knowledge graph.
Incremental Learning of Human Activities in Smart Homes

Sensor-based human activity recognition has been extensively studied. Systems learn from a set of training samples to classify actions into a pre-defined set of ground truth activities. However, human behaviours vary over time, and so a recognition system should ideally be able to continuously learn and adapt, while retaining the knowledge of previously learned activities, and without failing to highlight novel, and therefore potentially risky, behaviours. In this paper, we propose a method based on compression that can incrementally learn new behaviours, while retaining prior knowledge. Evaluation was conducted on three publicly available smart home datasets.
THOR: A Hybrid Recommender System for the Personalized Travel Experience

One of the travelers’ main challenges is that they have to spend a great effort to find and choose the most desired travel offer(s) among a vast list of non-categorized and non-personalized items. Recommendation systems provide an effective way to solve the problem of information overload. In this work, we design and implement “The Hybrid Offer Ranker” (THOR), a hybrid, personalized recommender system for the transportation domain. THOR assigns every traveler a unique contextual preference model built using solely their personal data, which makes the model sensitive to the user’s choices. This model is used to rank travel offers presented to each user according to their personal preferences. We reduce the recommendation problem to one of binary classification that predicts the probability with which the traveler will buy each available travel offer. Travel offers are ranked according to the computed probabilities, hence to the user’s personal preference model. Moreover, to tackle the cold start problem for new users, we apply clustering algorithms to identify groups of travelers with similar profiles and build a preference model for each group. To test the system’s performance, we generate a dataset according to some carefully designed rules. The results of the experiments show that the THOR tool is capable of learning the contextual preferences of each traveler and ranks offers starting from those that have the higher probability of being selected.
A situation awareness assessment method based on fuzzy cognitive maps

The status of an operator's situation awareness is one of the critical factors that influence the quality of the missions. Thus the measurement method of the situation awareness status is an important topic to research. So far, there are lots of methods designed for the measurement of situation awareness status, but there is no model that can measure it accurately in real-time, so this work is conducted to deal with such a gap. Firstly, collect the relevant physiological data of operators while they are performing a specific mission, simultaneously, measure their status of situation awareness by using the situation awareness global assessment technique (SAGAT), which is known for accuracy but cannot be used in real-time. And then, after the preprocessing of the raw data, use the physiological data as features, the SAGAT's results as a label to train a fuzzy cognitive map (FCM), which is an explainable and powerful intelligent model. Also, a hybrid learning algorithm of particle swarm optimization (PSO) and gradient descent is proposed for the FCM training. The final results show that the learned FCM can assess the status of situation awareness accurately in real-time, and the proposed hybrid learning algorithm has better efficiency and accuracy.
Scalable Nanophotonic-Electronic Spiking Neural Networks

Spiking neural networks (SNN) provide a new computational paradigm capable of highly parallelized, real-time processing. Photonic devices are ideal for the design of high-bandwidth, parallel architectures matching the SNN computational paradigm. Furthermore, the co-integration of CMOS and photonic elements combineslow-loss photonic devices with analog electronics for greater flexibility of nonlinear computational elements. We designed and simulated an optoelectronic spiking neuron circuit on a monolithic silicon photonics (SiPh) process that replicates useful spiking behaviors beyond the leaky integrate-and-fire (LIF). Additionally, we explored two learning algorithms with the potential for on-chip learning using Mach-Zehnder Interferometric (MZI) meshes as synaptic interconnects. A variation of Random Backpropagation (RPB) was experimentally demonstrated on-chip and matched the performance of a standard linear regression on a simple classification task. In addition, we applied the Contrastive Hebbian Learning (CHL) rule to a simulated neural network composed of MZI meshes for a random input-output mapping task. The CHL-trained MZI network performed better than random guessing but did not match the performance of the ideal neural network (without the constraints imposed by the MZI meshes). Through these efforts, we demonstrate that co-integrated CMOS and SiPh technologies are well-suited to the design of scalable SNN computing architectures.
Reportive evidentiality. A perception-based complement approach to digital discourse in Spanish and English

The present study builds on the evidential overtones of the main verbs of physical perception (visual) ver and see taking decir and say-reportive complements retrieved from a self-compiled Google corpus in Spanish and English. This opens up for research the possibility of a verdecir and seesay evidentiality in Spanish and English digital discourse. The analysis of the results confirms that the main verbs of direct vision ver and see take evidential values and provide justificatory support to the reported and quoted information provided by the decir and say post-complements based on the characteristics of the visual sources of knowledge, the meanings taken by the main perception verbs and the verbs of saying, and their complementation patterns of evidential perception.
Projection concept factorization with self-representation for data clustering

In recent years, matrix factorization-based techniques have received much attention in the data analysis field since it can perform dimensionality reduction and clustering simultaneously. Despite the great success achieved by the Non-negative matrix factorization (NMF) and concept factorization (CF) methods, they suffer from the out-of-sample problem and are sensitive to the noise. Some recent studies have indicated that the similarity relationship is capable of revealing the local structure. In this paper, a similarity graph is constructed to reflect the geometric information of manifold structure, while the concept factorization is employed to capture the global structure. In addition, the projection matrix is incorporated into the concept factorization model to eliminate the noise and avoid the out-of-sample problem. An iterative algorithm is introduced to solve the model. The experimental results obtained on both human face and text data sets verify the high efficiency of the proposed method.
Exploring Gender Differences in Computational Thinking Learning in a VR Classroom: Developing Machine Learning Models Using Eye-Tracking Data and Explaining the Models

Abstract Understanding existing gender differences in the development of computational thinking skills is increasingly important for gaining valuable insights into bridging the gender gap. However, there are few studies to date that have examined gender differences based on the learning process in a realistic classroom context. In this work, we aim to investigate gender classification using students’ eye movements that reflect temporal human behavior during a computational thinking lesson in an immersive VR classroom. We trained several machine learning classifiers and showed that students’ eye movements provide discriminative information for gender classification. In addition, we employed a Shapley additive explanation (SHAP) approach for feature selection and further model interpretation. The classification model trained with the selected (best) eye movement feature set using SHAP achieved improved performance with an average accuracy of over $$70\%$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mn>70</mml:mn> <mml:mo>%</mml:mo> </mml:mrow> </mml:math> . The SHAP values further explained the classification model by identifying important features and their impacts on the model output, namely gender. Our findings provide insights into the use of eye movements for in-depth investigations of gender differences in learning activities in VR classroom setups that are ecologically valid and may provide clues for providing personalized learning support and tutoring in such educational systems or optimizing system design.
A deep learning loss based on additive cosine margin: Application to fashion style and face recognition

Recently, loss functions based on angular spans improved the performance of deep visual recognition. These losses converted Euclidean cross entropy to angular cross entropy loss. Fashion style recognition deals with the problem of assigning a person’s outfit to a fashion style category. Due to the high similarity between different clothing items and the use of softmax-based loss functions, many of the current methods that address this problem show relatively poor performance and cannot guarantee sufficient inter-class margins in the fashion domain. In this work, we propose an end-to-end method for deep visual recognition by combining a standard CNN architecture with a novel loss function, which we call Additive Cosine Margin Loss (ACML). The proposed function not only projects feature vectors of different classes into different regions of the embedding, but also enforces compactness of the projections within each class. Our experiments were conducted on two public and well-known fashion style recognition datasets FashionStyle14 and HipsterWars, and on the face verification and identification datasets LFW, YTF, and MegaFace. These experiments demonstrate the superiority of the proposed loss function over: (1) existing angular margin-based loss functions (2) state-of-the-art methods for clothing style recognition as well as face analysis tasks.
Semi-HFL: semi-supervised federated learning for heterogeneous devices

Abstract In the vanilla federated learning (FL) framework, the central server distributes a globally unified model to each client and uses labeled samples for training. However, in most cases, clients are equipped with different devices and are exposed to a variety of situations. There are great differences between clients in storage, computing, communication, and other resources, which makes unified deep models used in traditional FL cannot fit clients’ personalized resource conditions. Furthermore, a great deal of labeled data is needed in traditional FL, whereas data labeling requires a great investment of time and resources, which is hard to do for individual clients. As a result, clients only have a vast amount of unlabeled data, which goes against the federated learning needs. To address the aforementioned two issues, we propose Semi-HFL, a semi-supervised federated learning approach for heterogeneous devices, which divides a deep model into a series of small submodels by inserting early exit branches to meet the resource requirements of different devices. Furthermore, considering the availability of labeled data, Semi-HFL introduces semi-supervised techniques for training in the above heterogeneous learning process. Specifically, two training phases are included in the semi-supervised learning process, unsupervised learning on clients and supervised learning on the server, which makes full use of clients’ unlabeled data. Through image classification, text classification, next-word prediction, and multi-task FL experiments based on five kinds of datasets, it is verified that compared with the traditional homogeneous learning method, Semi-HFL not only achieves higher accuracies but also significantly reduces the global resource overhead.
Forecasting cryptocurrencies’ price with the financial stress index: a graph neural network prediction strategy

This article proposes a graph neural network strategy (GNN), in which the long short-term memory (LSTM) and graph convolution network (GCN) are applied to capture both temporal and spatial features to forecast the price of Bitcoin, Litecoin, Ethereum, and Dash Coin with the ‘stable-coin’ Tether (USDT) and financial stress index (FSI). The main results show that the GNN strategy has better performance than univariate LSTM and multivariate LSTM in all of the seven steps forward forecasting. A sensitivity check shows that USDT and FSI/sub-FSI are important factors in the construction of the graphs and they verify the validity of the results.
rDSA : an intelligent tool for data science assignments

Tools supporting the teaching and learning of programming may help professors in correcting assignments and students in receiving immediate feedback, thus improving the solution before the final submission. This paper describes the rDSA tool, which was designed, developed, and evaluated to support students in completing assignments concerning (i) the execution of statistical analyses in the R language and (ii) commenting on the results in natural language. The paper focuses on the feedback provided by the tool to students and how it was designed/evaluated/improved over the years. The paper also presents the results of two studies that indicate the advantages of using the tool in terms of engagement and learning outcomes. To conclude, we provide a discussion on the characteristics of the tool, its use in similar courses, and the scope for future work.
Multiagent Reinforcement Learning With Heterogeneous Graph Attention Network

Most recent research on multiagent reinforcement learning (MARL) has explored how to deploy cooperative policies for homogeneous agents. However, realistic multiagent environments may contain heterogeneous agents that have different attributes or tasks. The heterogeneity of the agents and the diversity of relationships cause the learning of policy excessively tough. To tackle this difficulty, we present a novel method that employs a heterogeneous graph attention network to model the relationships between heterogeneous agents. The proposed method can generate an integrated feature representation for each agent by hierarchically aggregating latent feature information of neighbor agents, with the importance of the agent level and the relationship level being entirely considered. The method is agnostic to specific MARL methods and can be flexibly integrated with diverse value decomposition methods. We conduct experiments in predator-prey and StarCraft Multiagent Challenge (SMAC) environments, and the empirical results demonstrate that the performance of our method is superior to existing methods in several heterogeneous scenarios.
Toxicologic Pathology Forum: A Roadmap for Building State-of-the-Art Digital Image Data Resources for Toxicologic Pathology in the Pharmaceutical Industry

Digitization of histologic slides brings with it the promise of enhanced toxicologic pathology practice through the increased application of computational methods. However, the development of these advanced methods requires access to substrate image data, that is, whole slide images (WSIs). Deep learning methods, in particular, rely on extensive training data to develop robust algorithms. As a result, pharmaceutical companies interested in leveraging computational methods in their digital pathology workflows must first invest in data infrastructure to enable data access for both data scientists and pathologists. The process of building robust image data resources is challenging and includes considerations of generation, curation, and storage of WSI files, and WSI access including via linked metadata. This opinion piece describes the collective experience of building resources for WSI data in the Roche group. We elaborate on the challenges encountered and solutions developed with the goal of providing examples of how to build a data resource for digital pathology analytics in the pharmaceutical industry.
Considerations for (Teaching) Facilitator Roles for Movement-Based Design

Given the emergence of many new movement-based design methods, our work explores the problem of facilitating the activities and sessions that are part of these methods. We look at literature as well as our own experiences with facilitating movement-based design sessions, draw lessons regarding the various important facets of this facilitation, and present first thoughts regarding how to make competencies in that type of facilitation transferable in teaching.
Poster MPClan:

The growing volumes of data collected and its analysis to provide better services create worries about digital privacy. The literature has relied on secure multiparty computation techniques to address privacy concerns and give practical solutions. However, recent research has mostly focused on the small-party honest-majority setting of up to four parties, noting efficiency concerns. In this work, we extend the strategies to support a larger number of participants in honest-majority setting with efficiency at the center stage.
Data Augmentation Using Spectral Warping for Low Resource Children ASR

Abstract In low resource children automatic speech recognition (ASR) the performance is degraded due to limited acoustic and speaker variability available in small datasets. In this paper, we propose a spectral warping based data augmentation method to capture more acoustic and speaker variability. This is carried out by warping the linear prediction (LP) spectra computed from speech data. The warped LP spectra computed in a frame-based manner are used with the corresponding LP residuals to synthesize speech to capture more variability. The proposed augmentation method is shown to improve the ASR system performance over the baseline system. We have compared the proposed method with four well-known data augmentation methods: pitch scaling, speaking rate, SpecAug and vocal tract length perturbation (VTLP), and found that the proposed method performs the best. Further, we have combined the proposed method with these existing data augmentation methods to improve the ASR system performance even more. The combined system consisting of the original data, VTLP, SpecAug and the proposed spectral warping method gave the best performance by a relative word error rate reduction of 32.13% and 10.51% over the baseline system for Punjabi children and TLT-school corpus, respectively. The proposed spectral warping method is publicly available at https://github.com/kathania/Spectral-Warping .
Unbounded Gradients in Federated Learning with Buffered Asynchronous Aggregation

Synchronous updates may compromise the efficiency of cross-device federated learning once the number of active clients increases. The FedBuff algorithm (Nguyen et al. [1]) alleviates this problem by allowing asynchronous updates (staleness), which enhances the scalability of training while preserving privacy via secure aggregation. We revisit the FedBuff algorithm for asynchronous federated learning and extend the existing analysis by removing the boundedness assumptions from the gradient norm. This paper presents a theoretical analysis of the convergence rate of this algorithm when heterogeneity in data, batch size, and delay are considered.
An Efficient Data Protection Scheme Based on Hierarchical ID-Based Encryption for MQTT

As Internet of Things (IoT) thrives over the whole world, more and more IoT devices and IoT-based protocols have been designed and proposed in order to meet people’s needs. Among those protocols, message queueing telemetry transport (MQTT) is one of the most emerging and promising protocols, which provides many-to-many message transmissions based on the “publish/subscribe” mechanism. It has been widely used in industries such as the energy industry, chemical engineering, self-driving, and so on. While transporting important messages, MQTT specification recommends the use of TLS protocol. However, the computation cost of TLS is too heavy. Since topics in a broker are stored with a hierarchical structure, in this manuscript, we propose a novel data protection protocol for MQTT from hierarchical ID-based encryption. Our protocol adopts the intrinsic hierarchical structures of MQTT, and achieves constant-size keys, i.e., independent of the depth in hierarchical structures. Besides, the formal security model for the proposed protocol have been defined in the manuscript. The proposed protocol have been formally proven chosen-plaintext secure under the ℓ-wBDHI assumption.