OpenRaiser/Intern-Atlas huggingface dataset:
Energy Loss Prediction in IoT Energy Services
Venue: arXiv cs.DC 2023
Year: 2023

We propose a novel Energy Loss Prediction(ELP) framework that estimates the energy loss in sharing crowdsourced energy services. Crowdsourcing wireless energy services is a novel and convenient solution to enable the ubiquitous charging of nearby IoT devices. Therefore, capturing the wireless energy sharing loss is essential for the successful deployment of efficient energy service composition techniques. We propose Easeformer, a novel attention-based algorithm to predict the battery levels of IoT devices in a crowdsourced energy sharing environment. The predicted battery levels are used to estimate the energy loss. A set of experiments were conducted to demonstrate the feasibility and effectiveness of the proposed framework. We conducted extensive experiments on real wireless energy datasets to demonstrate that our framework significantly outperforms existing methods.
Sharpness & Shift-Aware Self-Supervised Learning
Venue: arXiv cs.LG 2023
Year: 2023

Self-supervised learning aims to extract meaningful features from unlabeled data for further downstream tasks. In this paper, we consider classification as a downstream task in phase 2 and develop rigorous theories to realize the factors that implicitly influence the general loss of this classification task. Our theories signify that sharpness-aware feature extractors benefit the classification task in phase 2 and the existing data shift between the ideal (i.e., the ideal one used in theory development) and practical (i.e., the practical one used in implementation) distributions to generate positive pairs also remarkably affects this classification task. Further harvesting these theoretical findings, we propose to minimize the sharpness of the feature extractor and a new Fourier-based data augmentation technique to relieve the data shift in the distributions generating positive pairs, reaching Sharpness & Shift-Aware Contrastive Learning (SSA-CLR). We conduct extensive experiments to verify our theoretical findings and demonstrate that sharpness & shift-aware contrastive learning can remarkably boost the performance as well as obtaining more robust extracted features compared with the baselines.
Towards More Robust NLP System Evaluation: Handling Missing Scores in Benchmarks
Venue: arXiv cs.CL 2023
Year: 2023

The evaluation of natural language processing (NLP) systems is crucial for advancing the field, but current benchmarking approaches often assume that all systems have scores available for all tasks, which is not always practical. In reality, several factors such as the cost of running baseline, private systems, computational limitations, or incomplete data may prevent some systems from being evaluated on entire tasks. This paper formalize an existing problem in NLP research: benchmarking when some systems scores are missing on the task, and proposes a novel approach to address it. Our method utilizes a compatible partial ranking approach to impute missing data, which is then aggregated using the Borda count method. It includes two refinements designed specifically for scenarios where either task-level or instance-level scores are available. We also introduce an extended benchmark, which contains over 131 million scores, an order of magnitude larger than existing benchmarks. We validate our methods and demonstrate their effectiveness in addressing the challenge of missing system evaluation on an entire task. This work highlights the need for more comprehensive benchmarking approaches that can handle real-world scenarios where not all systems are evaluated on the entire task.
Controllable Speaking Styles Using a Large Language Model
Venue: arXiv cs.CL 2023
Year: 2023

Reference-based Text-to-Speech (TTS) models can generate multiple, prosodically-different renditions of the same target text. Such models jointly learn a latent acoustic space during training, which can be sampled from during inference. Controlling these models during inference typically requires finding an appropriate reference utterance, which is non-trivial. Large generative language models (LLMs) have shown excellent performance in various language-related tasks. Given only a natural language query text (the prompt), such models can be used to solve specific, context-dependent tasks. Recent work in TTS has attempted similar prompt-based control of novel speaking style generation. Those methods do not require a reference utterance and can, under ideal conditions, be controlled with only a prompt. But existing methods typically require a prompt-labelled speech corpus for jointly training a prompt-conditioned encoder. In contrast, we instead employ an LLM to directly suggest prosodic modifications for a controllable TTS model, using contextual information provided in the prompt. The prompt can be designed for a multitude of tasks. Here, we give two demonstrations: control of speaking style; prosody appropriate for a given dialogue context. The proposed method is rated most appropriate in 50% of cases vs. 31% for a baseline model.
Vulnerability Propagation in Package Managers Used in iOS Development
Venue: arXiv cs.SE 2023
Year: 2023

Although using third-party libraries is common practice when writing software, vulnerabilities may be found even in well-known libraries. Detected vulnerabilities are often fixed quickly in the library code. The easiest way to include these fixes in a dependent software application, is to update the used library version. Package managers provide automated solutions for updating library dependencies. However, library dependencies can have dependencies to other libraries resulting in a dependency network with several levels of indirections. Assessing vulnerability risks induced by dependency networks is a non-trivial task for software developers. The library dependency network in the Swift ecosystem encompasses libraries from CocoaPods, Carthage and Swift Package Manager. We analysed how vulnerabilities propagate in the library dependency network of the Swift ecosystem, how vulnerable dependencies could be fixed via dependency upgrades, and if third party vulnerability analysis could be made more precise given public information on these vulnerabilities. We found that only 5.9% of connected libraries had a direct or transitive dependency to a vulnerable library. Although we found that most libraries with publicly reported vulnerabilities are written in C, the highest impact of publicly reported vulnerabilities originated from libraries written in native iOS languages. We found that around 30% of vulnerable dependencies could have been fixed via upgrading the library dependency. In case of critical vulnerabilities and latest library versions, over 70% of vulnerable dependencies would have been fixed via a dependency upgrade. Lastly, we checked whether the analysis of vulnerable dependency use could be refined using publicly available information on the code location (method or class) of a reported vulnerability. We found that such information is not available most of the time.
Confidence-Guided Semi-supervised Learning in Land Cover Classification
Venue: arXiv cs.CV 2023
Year: 2023

Semi-supervised learning has been well developed to help reduce the cost of manual labelling by exploiting a large quantity of unlabelled data. Especially in the application of land cover classification, pixel-level manual labelling in large-scale imagery is labour-intensive, time-consuming and expensive. However, existing semi-supervised learning methods pay limited attention to the quality of pseudo-labels during training even though the quality of training data is one of the critical factors determining network performance. In order to fill this gap, we develop a confidence-guided semi-supervised learning (CGSSL) approach to make use of high-confidence pseudo labels and reduce the negative effect of low-confidence ones for land cover classification. Meanwhile, the proposed semi-supervised learning approach uses multiple network architectures to increase the diversity of pseudo labels. The proposed semi-supervised learning approach significantly improves the performance of land cover classification compared to the classic semi-supervised learning methods and even outperforms fully supervised learning with a complete set of labelled imagery of the benchmark Potsdam land cover dataset.
Logit-Based Ensemble Distribution Distillation for Robust Autoregressive Sequence Uncertainties
Venue: arXiv cs.LG 2023
Year: 2023

Efficiently and reliably estimating uncertainty is an important objective in deep learning. It is especially pertinent to autoregressive sequence tasks, where training and inference costs are typically very high. However, existing research has predominantly focused on tasks with static data such as image classification. In this work, we investigate Ensemble Distribution Distillation (EDD) applied to large-scale natural language sequence-to-sequence data. EDD aims to compress the superior uncertainty performance of an expensive (teacher) ensemble into a cheaper (student) single model. Importantly, the ability to separate knowledge (epistemic) and data (aleatoric) uncertainty is retained. Existing probability-space approaches to EDD, however, are difficult to scale to large vocabularies. We show, for modern transformer architectures on large-scale translation tasks, that modelling the ensemble logits, instead of softmax probabilities, leads to significantly better students. Moreover, the students surprisingly even outperform Deep Ensembles by up to ~10% AUROC on out-of-distribution detection, whilst matching them at in-distribution translation.
Cultural Differences in Signed Ego Networks on Twitter: An Investigatory Analysis
Venue: arXiv cs.SI 2023
Year: 2023

Human social behaviour has been observed to adhere to certain structures. One such structure, the Ego Network Model (ENM), has been found almost ubiquitously in human society. Recently, this model has been extended to include signed connections. While the unsigned ENM has been rigorously observed for decades, the signed version is still somewhat novel and lacks the same breadth of observation. Therefore, the main aim of this paper is to examine this signed structure across various categories of individuals from a swathe of culturally distinct regions. Minor differences in the distribution of signs across the SENM can be observed between cultures. However, these can be overwhelmed when the network is centred around a specific topic. Indeed, users who are engaged with specific themes display higher levels of negativity in their networks. This effect is further supported by a significant negative correlation between the number of "general" topics discussed in a network and that network's percentage of negative connections. These findings suggest that the negativity of communications and relationships on Twitter are very dependent on the topics being discussed and, furthermore, these relationships are more likely to be negative when they are based around a specific topic.
$\mathbb{F}_q\mathcal{R}$-skew cyclic codes and their application to quantum codes
Venue: arXiv cs.IT 2023
Year: 2023

Let $p$ be a prime and $\mathbb{F}_q$ be the finite field of order $q=p^m$. In this paper, we study $\mathbb{F}_q\mathcal{R}$-skew cyclic codes where $\mathcal{R}=\mathbb{F}_q+u\mathbb{F}_q$ with $u^2=u$. To characterize $\mathbb{F}_q\mathcal{R}$-skew cyclic codes, we first establish their algebraic structure and then discuss the dual-containing properties by considering a non-degenerate inner product. Further, we define a Gray map over $\mathbb{F}_q\mathcal{R}$ and obtain their $\mathbb{F}_q$-Gray images. As an application, we apply the CSS (Calderbank-Shor-Steane) construction on Gray images of dual containing $\mathbb{F}_q\mathcal{R}$-skew cyclic codes and obtain many quantum codes with better parameters than the best-known codes available in the literature.
An Allocation Model for Attributing Emissions in Multi-tenant Cloud Data Centers
Venue: arXiv cs.DC 2023
Year: 2023

Cloud computing has become the de facto paradigm for delivering software to system users, with organizations and enterprises of all sizes making use of cloud services in some way. On the surface, adopting the cloud appears to be a very efficient approach for offloading concerns such as infrastructure management, logistics, and most importantly for this work, energy consumption and consequent carbon emissions to the cloud service provider. However, this is in many ways not an appropriately accountable solution to managing the contribution of the ICT sector to global emissions. To this effect, in this paper we report on an exploratory case study done in collaboration with a Software as a Service provider operating globally in the telecommunications sector. The study reckons with the service provider using multi-tenant, that is, shared, off-premises data centers for hosting their private cloud infrastructure towards developing a fair model of allocating operational emissions among the service tenants -- customer companies with many distinct users. The developed emissions model has to account for allocating in an appropriate manner the generated emissions between the tenants of the software provider services, and among the different tenants of the same data center. A carbon footprint report generator is developed building on the proposed model which is, in turn, used to present sustainability reports to involved stakeholders for evaluation purposes. Our results show that the model is perceived as transparent, informative, and fair, with requested improvements focusing mainly on the generated reports and the information contained therein.
Intelligent multicast routing method based on multi-agent deep reinforcement learning in SDWN
Venue: arXiv cs.NI 2023
Year: 2023

Multicast communication technology is widely applied in wireless environments with a high device density. Traditional wireless network architectures have difficulty flexibly obtaining and maintaining global network state information and cannot quickly respond to network state changes, thus affecting the throughput, delay, and other QoS requirements of existing multicasting solutions. Therefore, this paper proposes a new multicast routing method based on multiagent deep reinforcement learning (MADRL-MR) in a software-defined wireless networking (SDWN) environment. First, SDWN technology is adopted to flexibly configure the network and obtain network state information in the form of traffic matrices representing global network links information, such as link bandwidth, delay, and packet loss rate. Second, the multicast routing problem is divided into multiple subproblems, which are solved through multiagent cooperation. To enable each agent to accurately understand the current network state and the status of multicast tree construction, the state space of each agent is designed based on the traffic and multicast tree status matrices, and the set of AP nodes in the network is used as the action space. A novel single-hop action strategy is designed, along with a reward function based on the four states that may occur during tree construction: progress, invalid, loop, and termination. Finally, a decentralized training approach is combined with transfer learning to enable each agent to quickly adapt to dynamic network changes and accelerate convergence. Simulation experiments show that MADRL-MR outperforms existing algorithms in terms of throughput, delay, packet loss rate, etc., and can establish more intelligent multicast routes.
Emotion Recognition based on Psychological Components in Guided Narratives for Emotion Regulation
Venue: arXiv cs.CL 2023
Year: 2023

Emotion regulation is a crucial element in dealing with emotional events and has positive effects on mental health. This paper aims to provide a more comprehensive understanding of emotional events by introducing a new French corpus of emotional narratives collected using a questionnaire for emotion regulation. We follow the theoretical framework of the Component Process Model which considers emotions as dynamic processes composed of four interrelated components (behavior, feeling, thinking and territory). Each narrative is related to a discrete emotion and is structured based on all emotion components by the writers. We study the interaction of components and their impact on emotion classification with machine learning methods and pre-trained language models. Our results show that each component improves prediction performance, and that the best results are achieved by jointly considering all components. Our results also show the effectiveness of pre-trained language models in predicting discrete emotion from certain components, which reveal differences in how emotion components are expressed.
How does agency impact human-AI collaborative design space exploration? A case study on ship design with deep generative models
Venue: arXiv cs.LG 2023
Year: 2023

Typical parametric approaches restrict the exploration of diverse designs by generating variations based on a baseline design. In contrast, generative models provide a solution by leveraging existing designs to create compact yet diverse generative design spaces (GDSs). However, the effectiveness of current exploration methods in complex GDSs, especially in ship hull design, remains unclear. To that end, we first construct a GDS using a generative adversarial network, trained on 52,591 designs of various ship types. Next, we constructed three modes of exploration, random (REM), semi-automated (SAEM) and automated (AEM), with varying levels of user involvement to explore GDS for novel and optimised designs. In REM, users manually explore the GDS based on intuition. In SAEM, both the users and optimiser drive the exploration. The optimiser focuses on exploring a diverse set of optimised designs, while the user directs the exploration towards their design preference. AEM uses an optimiser to search for the global optimum based on design performance. Our results revealed that REM generates the most diverse designs, followed by SAEM and AEM. However, the SAEM and AEM produce better-performing designs. Specifically, SAEM is the most effective in exploring designs with a high trade-off between novelty and performance. In conclusion, our study highlights the need for innovative exploration approaches to fully harness the potential of GDS in design optimisation.
Comparison of classifiers in challenge scheme
Venue: arXiv cs.LG 2023
Year: 2023

In recent decades, challenges have become very popular in scientific research as these are crowdsourcing schemes. In particular, challenges are essential for developing machine learning algorithms. For the challenges settings, it is vital to establish the scientific question, the dataset (with adequate quality, quantity, diversity, and complexity), performance metrics, as well as a way to authenticate the participants' results (Gold Standard). This paper addresses the problem of evaluating the performance of different competitors (algorithms) under the restrictions imposed by the challenge scheme, such as the comparison of multiple competitors with a unique dataset (with fixed size), a minimal number of submissions and, a set of metrics chosen to assess performance. The algorithms are sorted according to the performance metric. Still, it is common to observe performance differences among competitors as small as hundredths or even thousandths, so the question is whether the differences are significant. This paper analyzes the results of the MeOffendEs@IberLEF 2021 competition and proposes to make inference through resampling techniques (bootstrap) to support Challenge organizers' decision-making.
Topology Optimization using Neural Networks with Conditioning Field Initialization for Improved Efficiency
Venue: arXiv cs.LG 2023
Year: 2023

We propose conditioning field initialization for neural network based topology optimization. In this work, we focus on (1) improving upon existing neural network based topology optimization, (2) demonstrating that by using a prior initial field on the unoptimized domain, the efficiency of neural network based topology optimization can be further improved. Our approach consists of a topology neural network that is trained on a case by case basis to represent the geometry for a single topology optimization problem. It takes in domain coordinates as input to represent the density at each coordinate where the topology is represented by a continuous density field. The displacement is solved through a finite element solver. We employ the strain energy field calculated on the initial design domain as an additional conditioning field input to the neural network throughout the optimization. The addition of the strain energy field input improves the convergence speed compared to standalone neural network based topology optimization.
Edge Directionality Improves Learning on Heterophilic Graphs
Venue: arXiv cs.LG 2023
Year: 2023

Graph Neural Networks (GNNs) have become the de-facto standard tool for modeling relational data. However, while many real-world graphs are directed, the majority of today's GNN models discard this information altogether by simply making the graph undirected. The reasons for this are historical: 1) many early variants of spectral GNNs explicitly required undirected graphs, and 2) the first benchmarks on homophilic graphs did not find significant gain from using direction. In this paper, we show that in heterophilic settings, treating the graph as directed increases the effective homophily of the graph, suggesting a potential gain from the correct use of directionality information. To this end, we introduce Directed Graph Neural Network (Dir-GNN), a novel general framework for deep learning on directed graphs. Dir-GNN can be used to extend any Message Passing Neural Network (MPNN) to account for edge directionality information by performing separate aggregations of the incoming and outgoing edges. We prove that Dir-GNN matches the expressivity of the Directed Weisfeiler-Lehman test, exceeding that of conventional MPNNs. In extensive experiments, we validate that while our framework leaves performance unchanged on homophilic datasets, it leads to large gains over base models such as GCN, GAT and GraphSage on heterophilic benchmarks, outperforming much more complex methods and achieving new state-of-the-art results.
Analytic relationship of relative synchronizability to network structure and motifs
Venue: arXiv cs.SI 2023
Year: 2023

Synchronization phenomena on networks have attracted much attention in studies of neural, social, economic, and biological systems, yet we still lack a systematic understanding of how relative synchronizability relates to underlying network structure. Indeed, this question is of central importance to the key theme of how dynamics on networks relate to their structure more generally. We present an analytic technique to directly measure the relative synchronizability of noise-driven time-series processes on networks, in terms of the directed network structure. We consider both discrete-time auto-regressive processes and continuous-time Ornstein-Uhlenbeck dynamics on networks. Our technique builds on computation of the network covariance matrix in the space orthogonal to the synchronized state, enabling it to be more general than previous work in not requiring either symmetric (undirected) or diagonalizable connectivity matrices, and allowing arbitrary self-link weights. More importantly, our approach quantifies the relative synchronisation specifically in terms of the contribution of process motif (walk) structures. We demonstrate that in general the relative abundance of process motifs with convergent directed walks (including feedback and feedforward loops) hinders synchronizability. We also reveal subtle differences between the motifs involved for discrete or continuous-time dynamics. Our insights analytically explain several known general results regarding synchronizability of networks, including that small-world and regular networks are less synchronizable than random networks.