Reviewerly/exHarmony huggingface dataset:
Superpixel Segmentation: From Theory to Applications*

Superpixel segmentation consists of partitioning images into regions composed of similar and connected pixels. Its methods have been widely used in many computer vision applications since it allows for reducing the workload, removing redundant information, and preserving regions with meaningful features. Due to the rapid progress in this area, the literature fails to catch up on more recent works among the compared ones and to categorize the methods according to all existing strategies. We revisit the recent and popular literature according to our taxonomy and evaluate 20 strategies based on nine criteria: connectivity, compactness, delineation, control over the number of superpixels, color homogeneity, robustness, running time, stability, and visual quality. Thus, this document presents a proposal for a tutorial to be presented at SIBGRAPI 2023.
A Generic Construction of an Anonymous Reputation System and Instantiations from Lattices

With an anonymous reputation system one can realize the process of rating sellers anonymously in an online shop. While raters can stay anonymous, sellers still have the guarantee that they can only be reviewed by raters who bought their product. We present the first generic construction of a reputation system from basic building blocks, namely digital signatures, encryption schemes, non-interactive zero-knowledge proofs, and linking indistinguishable tags. We then show the security of the reputation system in a strong security model. Among others, we instantiate the generic construction with building blocks based on lattice problems, leading to the first module lattice-based reputation system in the random oracle model.
Biphasic Face Photo-Sketch Synthesis via Semantic-Driven Generative Adversarial Network With Graph Representation Learning

Biphasic face photo-sketch synthesis has significant practical value in wide-ranging fields such as digital entertainment and law enforcement. Previous approaches directly generate the photo-sketch in a global view, they always suffer from the low quality of sketches and complex photograph variations, leading to unnatural and low-fidelity results. In this article, we propose a novel semantic-driven generative adversarial network to address the above issues, cooperating with graph representation learning. Considering that human faces have distinct spatial structures, we first inject class-wise semantic layouts into the generator to provide style-based spatial information for synthesized face photographs and sketches. In addition, to enhance the authenticity of details in generated faces, we construct two types of representational graphs via semantic parsing maps upon input faces, dubbed the intraclass semantic graph (IASG) and the interclass structure graph (IRSG). Specifically, the IASG effectively models the intraclass semantic correlations of each facial semantic component, thus producing realistic facial details. To preserve the generated faces being more structure-coordinated, the IRSG models interclass structural relations among every facial component by graph representation learning. To further enhance the perceptual quality of synthesized images, we present a biphasic interactive cycle training strategy by fully taking advantage of the multilevel feature consistency between the photograph and sketch. Extensive experiments demonstrate that our method outperforms the state-of-the-art competitors on the CUHK Face Sketch (CUFS) and CUHK Face Sketch FERET (CUFSF) datasets.
Cross-linking breast tumor transcriptomic states and tissue histology

Identification of the gene expression state of a cancer patient from routine pathology imaging and characterization of its phenotypic effects have significant clinical and therapeutic implications. However, prediction of expression of individual genes from whole slide images (WSIs) is challenging due to co-dependent or correlated expression of multiple genes. Here, we use a purely data-driven approach to first identify groups of genes with co-dependent expression and then predict their status from WSIs using a bespoke graph neural network. These gene groups allow us to capture the gene expression state of a patient with a small number of binary variables that are biologically meaningful and carry histopathological insights for clinical and therapeutic use cases. Prediction of gene expression state based on these gene groups allows associating histological phenotypes (cellular composition, mitotic counts, grading, etc.) with underlying gene expression patterns and opens avenues for gaining biological insights from routine pathology imaging directly.
Cumulative Capacitated Colored Traveling Salesman Problem

A colored traveling salesman problem (CTSP) is a generalization of the well-known multiple traveling salesman problem, which introduces colors to distinguish the accessibility of its cities to salesmen. This work proposes a city/customer-centric model called cumulative capacitated CTSP ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${\rm C}^{2}$</tex-math> </inline-formula> -CTSP) to tackle some practical problems with fast response requirements. Its hypergraph and mathematical programming formulations are developed for the first time. A general variable neighborhood search (GVNS) metaheuristic is designed to solve it. Specifically, greedy backtracking is proposed to initialize a solution taking into account the cumulative cost and two constraints including colors and capacities. Next, 2-swap, reinsertion, and double-bridge operations are randomly selected and carried out to execute the perturbation. Moreover, neighborhood-list-2-opt, relocation move, and generalized partition crossover are organized as variable neighborhood descent to constitute the local search for better solutions. Extensive experiments are conducted to compare the proposed GVNS with four genetic algorithms, two hybrid ant colony systems, two variable neighborhood search methods, and a perturb-based local search in 20 regular and random cases. The statistical results demonstrate that GVNS is superior to all competitors tuned by irace package in terms of both search ability and convergence rate. In addition, the study of six GVNS variants lacking different operators validates the significant role of each corresponding operator in GVNS’s outstanding performance.
CMCF-Net: An End-to-End Context Multiscale Cross-Fusion Network for Robust Copy-Move Forgery Detection

Image copy-move forgery detection (CMFD) has become a challenging problem due to increasingly powerful editing software that makes forged images increasingly realistic. Existing algorithms that directly connect multiple scales of features in the encoder part may not effectively aggregate contextual information, resulting in poor performance. In this paper, an end-to-end context multiscale cross-fusion network (CMCF-Net) is proposed to detect image copy-move forgery. The proposed network consists of a multiscale feature extraction fusion (MSF) module and a multi-information fusion decoding (MFD) module. Multiscale information is efficiently extracted and fused in the MSF module utilizing stacked-scale feature fusion, which improves the network's forgery localization ability on objects of different scales. The MFD module employs contextual information combination and weighted fusion of multiscale information to guide the network in obtaining relevant clues from correlated information at multiple different scales. Experimental results and analysis have demonstrated that the proposed CMCF-Net achieves the best localization results with higher robustness.
Prompt Sapper: A LLM-Empowered Production Tool for Building AI Chains

The emergence of foundation models, such as large language models (LLMs) GPT-4 and text-to-image models DALL-E, has opened up numerous possibilities across various domains. People can now use natural language (i.e. prompts) to communicate with AI to perform tasks. While people can use foundation models through chatbots (e.g., ChatGPT), chat, regardless of the capabilities of the underlying models, is not a production tool for building reusable AI services. APIs like LangChain allow for LLM-based application development but require substantial programming knowledge, thus posing a barrier. To mitigate this, we systematically review, summarise, refine and extend the concept of AI chain by incorporating the best principles and practices that have been accumulated in software engineering for decades into AI chain engineering, to systematize AI chain engineering methodology. We also develop a no-code integrated development environment, Prompt Sapper , which embodies these AI chain engineering principles and patterns naturally in the process of building AI chains, thereby improving the performance and quality of AI chains. With Prompt Sapper, AI chain engineers can compose prompt-based AI services on top of foundation models through chat-based requirement analysis and visual programming. Our user study evaluated and demonstrated the efficiency and correctness of Prompt Sapper.
Fuzzy Perspective of Online Games by Using Cryptography and Cooperative Game Theory

In this paper, we mathematically associate crypto-cloud computing with cooperative game theory in the presence of fuzzy uncertainty. For this purpose, we retrieve information from the database of Amazon Web Service. Then we construct a secure crypto-cloud game theoretical model and apply this model to online games under fuzzy uncertainty. Further, we suggest some fuzzy game theoretical solutions by proposing a novel twisted Edwards curve pairing-based scheme over finite fields having the property of indistinguishable under chosen ciphertext attacks. Finally, it is seen that costs of the numerical fuzzy solutions are reduced and their efficiency is increased compared with the numerical crisp ones.
University Teachers’ Agency in Relation to Technology Use in Teaching: A Quantitative Investigation

This study investigates Danish university teachers’ agency in using digital technologies in teaching. Furthermore, it explores how teachers’ conduct and experienced constraints are connected to different dimensions of teachers’ agency.&#x0D; The paper is based on survey responses from 344 teachers about their agentic will and power to influence aspects of technology usage in teaching. Using an exploratory factor analysis, we identify three distinct factors and their relations: 1) wanting to control digital data and having the power to do so is distinct from 2) teachers’ power and will to control the technologies adopted and 3) how technologies are applied in teaching. Furthermore, all factors correlate with the teachers’ conduct (extent to which technology facilitates education interactions), while there are factor-specific patterns to the constraints (e.g., time, experience).&#x0D; The results provide a basis for revisiting teachers’ role in university decision-making. We include a discussion on the entangledness of agency across institutional levels.
Deep variational Luenberger-type observer with dynamic objects channel-attention for stochastic video prediction

Considering the inherent stochasticity and uncertainty, predicting future video frames is exceptionally challenging. In this work, we study the problem of video prediction by combining interpretability of stochastic state space models and representation learning of deep neural networks with channel-attention. Our model which based on a variational encoder introduces a novel channel-attention block to enhance the ability to capture dynamic objects in video frames and a Luenberger-type observer which catches the dynamic evolution of the latent features. These enable the features of moving objects more conspicuous and decomposition of videos into static features and dynamics in an unsupervised manner. By deriving the stability theory of the nonlinear Luenberger-type observer, the hidden states in the feature space become insensitive with respect to the initial values, which improves the robustness of the overall model. Furthermore, the variational lower bound on the data log-likelihood can be derived to obtain the tractable posterior prediction distribution based on the variational principle. Finally, the experiments including the Rotaton Numbers dataset and the Kth dataset are provided to demonstrate the proposed model outperforms concurrent works.
Beyond Words: An Intelligent Human-Machine Dialogue System with Multimodal Generation and Emotional Comprehension

Intelligent service robots have become an indispensable aspect of modern-day society, playing a crucial role in various domains ranging from healthcare to hospitality. Among these robotic systems, human-machine dialogue systems are particularly noteworthy as they deliver both auditory and visual services to users, effectively bridging the communication gap between humans and machines. Despite their utility, the majority of existing approaches to these systems primarily concentrate on augmenting the logical coherence of the system’s responses, inadvertently neglecting the significance of user emotions in shaping a comprehensive communication experience. To tackle this shortcoming, we propose the development of an innovative human-machine dialogue system that is both intelligent and emotionally sensitive, employing multimodal generation techniques. This system is architecturally comprised of three components: (1) data collection and processing, responsible for gathering and preparing relevant information, (2) a dialogue engine, which generates contextually appropriate responses, and (3) an interaction module, responsible for facilitating the communication interface between users and the system. To validate our proposed approach, we have constructed a prototype system and conducted an evaluation of the performance of the core dialogue engine by utilizing an open dataset. The results of our study indicate that our system demonstrates a remarkable level of multimodal generation response, ultimately offering a more human-like dialogue experience.
ScanGuard-YOLO: Enhancing X-ray Prohibited Item Detection with Significant Performance Gains

To address the problem of low recall rate in the detection of prohibited items in X-ray images due to the severe object occlusion and complex background, an X-ray prohibited item detection network, ScanGuard-YOLO, based on the YOLOv5 architecture, is proposed to effectively improve the model’s recall rate and the comprehensive metric F1 score. Firstly, the RFB-s module was added to the end part of the backbone, and dilated convolution was used to increase the receptive field of the backbone network to better capture global features. In the neck section, the efficient RepGFPN module was employed to fuse multiscale information from the backbone output. This aimed to capture details and contextual information at various scales, thereby enhancing the model’s understanding and representation capability of the object. Secondly, a novel detection head was introduced to unify scale-awareness, spatial-awareness, and task-awareness altogether, which significantly improved the representation ability of the object detection heads. Finally, the bounding box regression loss function was defined as the WIOUv3 loss, effectively balancing the contribution of low-quality and high-quality samples to the loss. ScanGuard-YOLO was tested on OPIXray and HiXray datasets, showing significant improvements compared to the baseline model. The mean average precision (mAP@0.5) increased by 2.3% and 1.6%, the recall rate improved by 4.5% and 2%, and the F1 score increased by 2.3% and 1%, respectively. The experimental results demonstrate that ScanGuard-YOLO effectively enhances the detection capability of prohibited items in complex backgrounds and exhibits broad prospects for application.
Application of AI for Short-Term PV Generation Forecast

The efficient use of the photovoltaic power requires a good estimation of the PV generation. That is why the use of good techniques for forecast is necessary. In this research paper, Long Short-Term Memory, Bidirectional Long Short-Term Memory and the Temporal convolutional network are studied in depth to forecast the photovoltaic power, voltage and efficiency of a 1320 Wp amorphous plant installed in the Technology Support Centre in the University Rey Juan Carlos, Madrid (Spain). The accuracy of these techniques are compared using experimental data along one year, applying 1 timestep or 15 min and 96 step times or 24 h, showing that TCN exhibits outstanding performance, compared with the two other techniques. For instance, it presents better results in all forecast variables and both forecast horizons, achieving an overall Mean Squared Error (MSE) of 0.0024 for 15 min forecasts and 0.0058 for 24 h forecasts. In addition, the sensitivity analyses for the TCN technique is performed and shows that the accuracy is reduced as the forecast horizon increases and that the 6 months of dataset is sufficient to obtain an adequate result with an MSE value of 0.0080 and a coefficient of determination of 0.90 in the worst scenarios (24 h of forecast).
Sim2Real: Generative AI to Enhance Photorealism through Domain Transfer with GAN and Seven-Chanel-360°-Paired-Images Dataset

This work aims at providing a solution to data scarcity by allowing end users to generate new images while carefully controlling building shapes and environments. While Generative Adversarial Networks (GANs) are the most common network type for image generation tasks, recent studies have only focused on RGB-to-RGB domain transfer tasks. This study utilises a state-of-the-art GAN network for domain transfer that effectively transforms a multi-channel image from a 3D scene into a photorealistic image. It relies on a custom dataset that pairs 360° images from a simulated domain with corresponding 360° street views. The simulated domain includes depth, segmentation map, and surface normal (stored in seven-channel images), while the target domain is composed of photos from Paris. Samples come in pairs thanks to careful virtual camera positioning. To enhance the simulated images into photorealistic views, the generator is designed to preserve semantic information throughout the layers. The study concludes with photorealistic-generated samples from the city of Paris, along with strategies to further refine model performance. The output samples are realistic enough to be used to train and improve future AI models.
Neural Architecture Search for Text Classification With Limited Computing Resources Using Efficient Cartesian Genetic Programming

Cartesian Genetic Programming (CGP) has often been applied for Neural Architecture Search (NAS). However, the performance of CGP is less than ideal when searching for architectures with limited computing resources. To better facilitate NAS with limited computing resources, this paper proposes a crossover operator, a light-weighted age mechanism, and two adaptive mutation operators as the novel components in our Efficient Cartesian Genetic Programming (ECGP) method. To assess the performance of ECGP, we conduct extensive experiments on three text classification task datasets. The experimental results demonstrate that ECGP outperforms other NAS methods, requiring only hundreds of fitness evaluations to find architectures with competitive accuracy compared with human-designed models. Additionally, the ECGP-evolved architectures are shown as converging fast and stably, and having high-level transferability with merely a 1-2% accuracy drop. Ablation studies demonstrate the effectiveness of the proposed operators and age mechanism, and identify GRU as the most critical function in the text classification task. Finally, we summarize three design principles observed from the ECGP-evolved architectures that are in line with human-design strategies. To the best of our knowledge, this work introduces the first attention-derived NAS benchmark for the text classification task.
From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inference

Large language models (LLMs) have exploded in popularity due to their new generative capabilities that go far beyond prior state-of-the-art. These technologies are increasingly being leveraged in various domains such as law, finance, and medicine. However, these models carry significant computational challenges, especially the compute and energy costs required for inference. Inference energy costs already receive less attention than the energy costs of training LLMs-despite how often these large models are called on to conduct inference in reality (e.g., ChatGPT). As these state-of-the-art LLMs see increasing usage and deployment in various domains, a better understanding of their resource utilization is crucial for cost-savings, scaling performance, efficient hardware usage, and optimal inference strategies. In this paper, we describe experiments conducted to study the computational and energy utilization of inference with LLMs. We benchmark and conduct a preliminary analysis of the inference performance and inference energy costs of different sizes of LLaMA-a recent state-of-the-art LLM-developed by Meta AI on two generations of popular GPUs (NVIDIA V100 & A100) and two datasets (Alpaca and GSM8K) to reflect the diverse set of tasks/benchmarks for LLMs in research and practice. We present the results of multi-node, multi-GPU inference using model sharding across up to 32 GPUs. To our knowledge, our work is the one of the first to study LLM inference performance from the perspective of computational and energy resources at this scale.
SelfGraphVQA: A Self-Supervised Graph Neural Network for Scene-based Question Answering

The intersection of vision and language is of major interest due to the increased focus on seamless integration between recognition and reasoning.Scene graphs (SGs) have emerged as a useful tool for multimodal image analysis, showing impressive performance in tasks such as Visual Question Answering (VQA).In this work, we demonstrate that despite the effectiveness of scene graphs in VQA tasks, current methods that utilize idealized annotated scene graphs struggle to generalize when using predicted scene graphs extracted from images.To address this issue, we introduce the SelfGraphVQA framework.Our approach extracts a scene graph from an input image using a pretrained scene graph generator and employs semanticallypreserving augmentation with self-supervised techniques.This method improves the utilization of graph representations in VQA tasks by circumventing the need for costly and potentially biased annotated data.By creating alternative views of the extracted graphs through image augmentations, we can learn joint embeddings by optimizing the informational content in their representations using an un-normalized contrastive approach.As we work with SGs, we experiment with three distinct maximization strategies: node-wise, graph-wise, and permutation-equivariant regularization.We empirically showcase the effectiveness of the extracted scene graph for VQA and demonstrate that these approaches enhance overall performance by highlighting the significance of visual information.This offers a more practical solution for VQA tasks that rely on SGs for complex reasoning questions.