Reviewerly/exHarmony huggingface dataset:
Progressive-fidelity computation of the genetic algorithm for energy-efficient virtual machine placement in cloud data centers

Energy efficiency is a critical issue in the management and operation of data centers, which form the backbone of cloud computing. Virtual machine placement has a significant impact on the energy efficiency of virtualized data centers. Among various methods to solve the virtual-machine placement problem, the genetic algorithm has been well accepted for its quality of solution. However, it is computationally demanding largely due to the complex form of fitness function, limiting its applications in data centers. To enhance the computational efficiency of the genetic algorithm while maintaining its quality of solution, a progressive-fidelity approach is developed in this paper for genetic-algorithm computation. It starts with a low-fidelity genetic algorithm with a simple fitness function. Then, for solution refinement, it switches to a medium-fidelity genetic algorithm with a more complicated fitness function. Finally, it progresses to the fine tuning of solution through a high-fidelity genetic algorithm with the energy consumption of data centers as fitness function. Heuristics are presented for the adaptive switching of genetic-algorithm computation from low fidelity to medium fidelity and finally to high fidelity. Experiments show that compared with the standard genetic algorithm of high fidelity, our progressive-fidelity approach of genetic algorithm computation is 50% faster for large-scale data centers while maintaining similar quality of solution in terms of the energy consumption of data centers.
Developments on metaheuristic-based optimization for numerical and engineering optimization problems: Analysis, design, validation, and applications

Optimization problems are prevalent in a variety of real-world applications, including medical, engineering, chemical, and others, and must be precisely solved to enhance the performance of these applications. Unfortunately, finding near-optimal solutions to these problems is regarded as a hard challenge due to their various characteristics. In a new attempt to solve these problems, this paper presents a new variant of the artificial gorilla troops optimizer (GTO) called ranking-based GTO (RGTO). This variant uses two strategies known as the ranking-based update strategy and the convergence acceleration strategy to improve both the classical GTO's exploitation and exploration capabilities. The first strategy is proposed to enhance each gorilla's local and global search abilities, whereas the latter is intended to enhance GTO's global search abilities to reach better solutions as quickly as possible. First, a recent and challenging benchmark, namely CEC-2017, is utilized to assess the RGTO's explorative and exploitative capabilities. After that, RGTO is used to solve three engineering optimization problems, including parameter estimation problems for both photovoltaic (PV) models and proton exchange membrane fuel cells (PEMFCs), as well as some engineering design problems, to demonstrate how well it performs for real-world optimization problems. Compared to several rival optimizers, the proposed algorithm provides outstanding outcomes for the three engineering optimization benchmark problems considered.
VideoDoodles: Hand-Drawn Animations on Videos with Scene-Aware Canvases

We present an interactive system to ease the creation of so-called video doodles - videos on which artists insert hand-drawn animations for entertainment or educational purposes. Video doodles are challenging to create because to be convincing, the inserted drawings must appear as if they were part of the captured scene. In particular, the drawings should undergo tracking, perspective deformations and occlusions as they move with respect to the camera and to other objects in the scene - visual effects that are difficult to reproduce with existing 2D video editing software. Our system supports these effects by relying on planar canvases that users position in a 3D scene reconstructed from the video. Furthermore, we present a custom tracking algorithm that allows users to anchor canvases to static or dynamic objects in the scene, such that the canvases move and rotate to follow the position and direction of these objects. When testing our system, novices could create a variety of short animated clips in a dozen of minutes, while professionals praised its speed and ease of use compared to existing tools.
A white paper on good research practices in benchmarking: The case of cluster analysis

Note: A revised version of this is now published. Please cite and read (it's open access): Van Mechelen, I., Boulesteix, A.-L., Dangl, R., Dean, N., Hennig, C., Leisch, F., Steinley, D., Warrens, M. J. (2023). A white paper on good research practices in benchmarking: The case of cluster analysis. WIREs Data Mining and Knowledge Discovery, e1511. https://doi.org/10.1002/widm.1511 To achieve scientific progress in terms of building a cumulative body of knowledge, careful attention to benchmarking is of the utmost importance. This means that proposals of new methods of data pre-processing, new data-analytic techniques, and new methods of output post-processing, should be extensively and carefully compared with existing alternatives, and that existing methods should be subjected to neutral comparison studies. To date, benchmarking and recommendations for benchmarking have been frequently seen in the context of supervised learning. Unfortunately, there has been a dearth of guidelines for benchmarking in an unsupervised setting, with the area of clustering as an important subdomain. To address this problem, discussion is given to the theoretical conceptual underpinnings of benchmarking in the field of cluster analysis by means of simulated as well as empirical data. Subsequently, the practicalities of how to address benchmarking questions in clustering are dealt with, and foundational recommendations are made.
AYNEXT - tools for streamlining the evaluation of link prediction techniques

AYNEXT is an open source Python suite aimed towards researchers in the field of link prediction in Knowledge Graphs. Link prediction consists of predicting missing edges in a Knowledge Graph, which usually involves the application of different techniques to generate negative examples (false triples) to fit a model, and splitting edges into training, testing and validation sets. Setting up a correct evaluation setup or testing new negatives-generation strategies becomes more challenging as more complex strategies and considerations (e.g., removal of inverse relations) develop. AYNEXT makes it easy to configure and customize the creation of evaluation datasets and the computation of evaluation metrics and statistical significance tests for each pair of link prediction techniques. AYNEXT has been designed to be simple to use, but modular enough to enable customization of the main steps in the evaluation process. AYNEXT-DataGen covers the pre-processing, splitting, and negatives generation steps of the evaluation process, while AYNEXT-ResTest covers the metrics computing and the statistical tests. AYNEXT offers a simple to use command line interface that takes as input either a Knowledge Graph in standard formats or the results of applying existing techniques, but can be used programmatically for in-depth customization.
Cheating Automatic Short Answer Grading with the Adversarial Usage of Adjectives and Adverbs

Abstract Automatic grading models are valued for the time and effort saved during the instruction of large student bodies. Especially with the increasing digitization of education and interest in large-scale standardized testing, the popularity of automatic grading has risen to the point where commercial solutions are widely available and used. However, for short answer formats, automatic grading is challenging due to natural language ambiguity and versatility. While automatic short answer grading models are beginning to compare to human performance on some datasets, their robustness, especially to adversarially manipulated data, is questionable. Exploitable vulnerabilities in grading models can have far-reaching consequences ranging from cheating students receiving undeserved credit to undermining automatic grading altogether—even when most predictions are valid. In this paper, we devise a black-box adversarial attack tailored to the educational short answer grading scenario to investigate the grading models’ robustness. In our attack, we insert adjectives and adverbs into natural places of incorrect student answers, fooling the model into predicting them as correct. We observed a loss of prediction accuracy between 10 and 22 percentage points using the state-of-the-art models BERT and T5. While our attack made answers appear less natural to humans in our experiments, it did not significantly increase the graders’ suspicions of cheating. Based on our experiments, we provide recommendations for utilizing automatic grading systems more safely in practice.
Expectations over Unspoken Alternatives Predict Pragmatic Inferences

Abstract Scalar inferences (SI) are a signature example of how humans interpret language based on unspoken alternatives. While empirical studies have demonstrated that human SI rates are highly variable—both within instances of a single scale, and across different scales—there have been few proposals that quantitatively explain both cross- and within-scale variation. Furthermore, while it is generally assumed that SIs arise through reasoning about unspoken alternatives, it remains debated whether humans reason about alternatives as linguistic forms, or at the level of concepts. Here, we test a shared mechanism explaining SI rates within and across scales: context-driven expectations about the unspoken alternatives. Using neural language models to approximate human predictive distributions, we find that SI rates are captured by the expectedness of the strong scalemate as an alternative. Crucially, however, expectedness robustly predicts cross-scale variation only under a meaning-based view of alternatives. Our results suggest that pragmatic inferences arise from context-driven expectations over alternatives, and these expectations operate at the level of concepts.1
Special issue on technology assisted review systems

When faced with an information need, an exhaustive search aim to retrieve all the relevant information. The need for comprehensive research is fundamental electronic discovery, systematic reviews, investigation, research in general, and also in the creation of evaluation collections in information retrieval. To support these high-recall retrieval tasks, technology relies on machine learning to develop systems that rank and classify documents with greater efficiency than traditional boolean keyword searches. Technology-Assisted Review (TAR) is one of the key technologies which can help to improve this process. The aim of this special issue is to investigate the most recent approaches of TAR systems applied in a wide variety of contexts where high recall is necessary: from the classification of emails to the compilation of systematic reviews up to the evaluation of the quality of the assessments.
Generating Presence-Absence Matrices by Quantum Annealing

We consider the problem of generating binary matrices with fixed sums for their rows and columns coefficients, i.e. with fixed margins. Such presence-absence (0/1) matrices are widely used in ecological research, for instance to represent the presence or absence of particular species in a particular habitat. Generating random matrices with fixed sums for their rows and their columns is an important issue in order to compare some given matrix presenting field data versus randomly generated matrices with similar characteristics (same sums on rows and columns) in order to test some hypothesis, i.e. when performing null model statistical analysis. We propose to model this problem in QUBO (Quadratic Unconstrained Binary Optimization) in order to solve it by quantum annealing. QUBO is the input language of quantum computers based on quantum annealing such as the D-Wave systems and of “quantum-inspired” annealing solvers based on dedicated classical hardware. We present some experimental results achieved on the D-Wave Advantage quantum computer and on the Fixstars Amplify Annealing Engine.
A Low Complexity Approximation of Gradient Descent for Learning over Single and Multi-Agent Systems

Gradient descent (GD) is an iterative optimization method to minimize a differentiable cost function. A main drawback of GD is its use of the entire dataset to perform a parameter update per learning step, which is cost-prohibitive for large-scale problems. Variants of GD such as mini-batch GD and stochastic GD compute the update direction with respect to a randomly sampled selection of the training dataset per iteration. This reduces the computational burden at the cost of adding jitters to the learning direction. Alternatively, we propose an iterative optimization algorithm that performs cheap parameter updates at minimal perturbation of the GD direction. The new method computes a deterministic summary of the training dataset, and then computes the learning direction per iteration with respect to the summary. We provide a convergence analysis for the proposed method and show that the thoroughness of the summary can be tweaked to optimize the trade-off between computational complexity and convergence rate. Simulation results are illustrated numerically for single and multi-agent systems.
A parallel deep learning-based code clone detection model

Code clone detection is a crucial task in software development and maintenance. While deep learning-based methods have been proposed to tackle this problem, most of them neglect the time and memory consumption issues which can be significant when working with limited computational resources. Given the inability of recurrent neural networks to train in a parallel manner, this paper presents a parallel code clone detection model based on temporal convolutional networks. The proposed method splits the corresponding abstract syntax tree into a set of code statement sequences, utilizes a temporal convolutional neural network to generate representations containing complexity features found in the source code, and finally measures the distance between these representations. The proposed method is evaluated on a real-world dataset for code clone detection, and the experimental results demonstrate that it performs comparably to state-of-the-art methods while requiring significantly less time and memory costs.
Hakuin: Optimizing Blind SQL Injection with Probabilistic Language Models

SQL Injection (SQLI) is a pervasive web attack where a malicious input is used to dynamically build SQL queries in a way that tricks the database (DB) engine into performing unintended harmful operations. Among many potential exploitations, an attacker may opt to exfiltrate the application data. The exfiltration process is straightforward when the web application responds to injected queries with their results. In case the content is not exposed, the adversary can still deduce it using Blind SQLI (BSQLI), an inference technique based on response differences or time delays. Unfortunately, a common drawback of BSQLI is its low inference rate (one bit per request), which severely limits the volume of data that can be extracted this way. To address this limitation, the state-of-the-art BSQLI tools optimize the inference of textual data with binary search. However, this approach has two major limitations: it assumes a uniform distribution of characters and does not take into account the history of previously inferred characters. Consequently, the technique is inefficient for natural languages used ubiquitously in DBs. This paper presents Hakuin - a new framework for optimizing BSQLI with probabilistic language models. Hakuin employs domain-specific pre-trained and adaptive models to predict the next characters based on the inference history and prioritizes characters with a higher probability of being the right ones. It also tracks statistical information to opportunistically guess strings as a whole instead of inferring the characters separately. We benchmark Hakuin against 3 state-of-the-art BSQLI tools using 20 industry-standard DB schemas and a generic DB. The results show that Hakuin is about 6 times more efficient in inferring schemas, up to 3.2 times more efficient with generic data, and up to 26 times more efficient on columns with limited values compared to the second-best performing tool. To the best of our knowledge, Hakuin is the first solution that combines domain-specific pre-trained and adaptive language models to optimize BSQLI. We release its full source code, datasets, and language models to facilitate further research.
Performance Comparison of Binary Machine Learning Classifiers in Identifying Code Comment Types: An Exploratory Study

Code comments are vital to source code as they help developers with program comprehension tasks. Written in natural language (usually English), code comments convey a variety of different information, which are grouped into specific categories. In this study, we construct 19 binary machine learning classifiers for code comment categories that belong to three different programming languages. We present a comparison of performance scores for different types of machine learning classifiers and show that the Linear SV C classifier has the highest average Fl score of 0.5474.
STACC: Code Comment Classification using SentenceTransformers

Code comments are a key resource for information about software artefacts. Depending on the use case, only some types of comments are useful. Thus, automatic approaches to clas-sify these comments have been proposed. In this work, we address this need by proposing, STACC, a set of SentenceTransformers- based binary classifiers. These lightweight classifiers are trained and tested on the NLBSE Code Comment Classification tool competition dataset, and surpass the baseline by a significant margin, achieving an average Fl score of 0.74 against the baseline of 0.31, which is an improvement of 139%. A replication package, as well as the models themselves, are publicly available.
Applying information theory to software evolution

Although information theory has found success in disciplines, the literature on its applications to software evolution is limit. We are still missing artifacts that leverage the data and tooling available to measure how the information content of a project can be a proxy for its complexity. In this work, we explore two definitions of entropy, one structural and one textual, and apply it to the historical progression of the commit history of 25 open source projects. We produce evidence that they generally are highly correlated. We also observed that they display weak and unstable correlations with other complexity metrics. Our preliminary investigation of outliers shows an unexpected high frequency of events where there is considerable change in the information content of the project, suggesting that such outliers may inform a definition of surprisal.
Hop-Constrained s-t Simple Path Enumeration on Large Dynamic Graphs

Hop-constrained s-t simple path (k-st path) enumeration is a fundamental problem in graph databases and plays an important role in many real-world applications. Given a dynamic graph G, a source-target pair s-t, and a hop constraint k, we aim to efficiently compute k-st paths: list all simple paths within length k from s to t, and then continuously maintain the results against edge updates. Although the k-st path enumeration has been well studied in static setting, the existing works on static graphs cannot be applied or adapted to handle dynamic graphs efficiently. To address the challenges on dynamic computation, we propose a partial path-based index structure and an efficient enumeration algorithm based on the index. We also propose several well-designed techniques to efficiently maintain the index and locate the affected results with graph updates. Comprehensive experiments verify that our proposed CPE <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">update</inf> algorithm outperforms the state-of-the-art methods by up to 4 orders of magnitude on dynamic graphs. The experiment results also show that the time cost of our initialization step CPE <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">startup</inf> (including index construction) is similar to the state-of-the-art static method.
Zero-shot Prompting for Code Complexity Prediction Using GitHub Copilot

Code generation models are gaining popularity because they can produce correct code from a prompt, speeding up the software development process. GitHub Copilot is currently one of the most commonly used tools for code generation. This tool is based on GPT3, a Large Language Model (LLM), and can perform zero-shot prompting tasks i.e., tasks for which the model is not specifically trained. In this paper, we describe a preliminary study that investigates whether GitHub Copilot can predict the runtime complexity of a given program using zero- shot prompting. In our study, we found that GitHub Copilot can correctly predict the runtime complexity 45.44% times in the first suggestion and 56.38 % times considering all suggestions. We also compared Copilot to other machine learning, neural network, and transformer-based approaches for code complexity prediction. We observed that Copilot outperformed other approaches for predicting code with linear complexity <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{O}(n)$</tex> .
Distributed Near-Maximum Independent Set Maintenance over Large-scale Dynamic Graphs

Computing the maximum independent set (MIS) in a graph is a fundamental NP-hard problem, which is widely adopted in many real-world applications. Extensive works have been done on computing an approximate MIS. While the highly dynamic property of real-world graphs calls for efficient MIS maintenance solutions, existing works for dynamic MIS computation in the literature mainly focus on the single-machine scenario. The assumption that a single machine can access the whole graph makes them difficult to be straightforwardly applied for large-scale graphs in distributed environment. Motivated by this, in this paper, we study the problem of maintaining approximate MIS over large-scale dynamic graphs in distributed environments. We propose a new vertex centric algorithm OIMIS. Compared with existing solutions, OIMIS avoids the strong order dependency in distributed computation, which makes it easy to handle dynamic graph updates. OIMIS computes and maintains MIS with high effectiveness and efficiency. In terms of high effectiveness, OIMIS maintains consistent MIS results with the state-of-the-art distributed algorithm to compute MIS in static graphs. In terms of high efficiency, each vertex in OIMIS only updates MIS status according to its neighbor attributes. Novel optimization techniques are also designed to reduce communication and computation cost. We conduct extensive experiments to prove the effectiveness and efficiency of our distributed algorithms.