Reviewerly/exHarmony huggingface dataset:
Performance analysis of saliency detection in images

Detecting salient objects in an image has a vast number of applications across the web, and mostly it is done manually. Since the internet is growing faster than ever, it is not a feasible solution for high-scale dynamic applications. Hence, image processing automation with computer vision and machine learning has become a burning topic of research in the past few years. Saliency based image cropping is a task of identifying the notable segments in an image and be able to accommodate these parts in the crop ratio for any given specific viewport. It helps not only in image cropping but also with object recognition, visual tracking and visual restoration, etc. To address this, a novel approach for saliency detection is proposed based on heat map analysis and boundary prior. In this approach, the images go through heat map analysis to identify salient objects based on machine learning model. After that, the images are segmented based on color and texture difference and then major contours are identified. The contours and the obtained saliency coordinates are accommodated inside the crop for each viewport requirement. The crop coordinates then go through two more processes, firstly a shift center process where the crop center is moved towards the important but lesser salient object, and then an inclusivity rule checks that the image is not cropped at coordinates without any salient objects. The simulation results reveal that the proposed algorithm attains better results than the cutting-edge algorithm of Twitter with similarity index of 89.89%.
Weakly-Supervised TILs Segmentation Based on Point Annotations Using Transfer Learning with Point Detector and Projected-Boundary Regressor

In Whole Slide Image (WSI) analysis, detecting nuclei sub-types such as Tumor Infiltrating Lymphocytes (TILs) which are a primary bio-marker for cancer diagnosis, is an important yet challenging task. Though several conventional methods have been proposed and applied to target user’s nuclei sub-types (e.g., TILs), they often fail to detect subtle differences between instances due to similar morphology across sub-types. To address this, we propose a novel decoupled segmentation architecture that leverages point annotations in a weakly-supervised manner to adapt to the nuclei sub-type. Our design consists of an encoder for feature extraction, a boundary regressor that learns prior knowledge from nuclei boundary masks, and a point detector that predicts the center positions of nuclei, respectively. Moreover, employing a frozen pre-trained nuclei segmenter facilitates easier adaptation to TILs segmentation via fine-tuning, while learning a decoupled point detector. To demonstrate the effectiveness of our approach, we evaluated on an in-house Melanoma TIL dataset, and report significant improvements over a state-of-the-art weakly-supervised TILs segmentation method, including conventional approaches based on pseudo-label construction.
Predicting Brain Multigraph Population from a Single Graph Template for Boosting One-Shot Classification

A central challenge in training one-shot learning models is the limited representativeness of the available shots of the data space. Particularly in the field of network neuroscience where the brain is represented as a graph, such models may lead to low performance when classifying brain states (e.g., typical vs. autistic). To cope with this, most of the existing works involve a data augmentation step to increase the size of the training set, its diversity and representativeness. Though effective, such augmentation methods are limited to generating samples with the same size as the input shots (e.g., generating brain connectivity matrices from a single shot matrix). To the best of our knowledge, the problem of generating brain multigraphs capturing multiple types of connectivity between pairs of nodes (i.e., anatomical regions) from a single brain graph remains unsolved. In this paper, we unprecedentedly propose a hybrid graph neural network (GNN) architecture, namely Multigraph Generator Network or briefly MultigraphGNet, comprising two subnetworks: (1) a many-to-one GNN which integrates an input population of brain multigraphs into a single template graph, namely a connectional brain temple (CBT), and (2) a reverse one-to-many U-Net network which takes the learned CBT in each training step and outputs the reconstructed input multigraph population. Both networks are trained in an end-to-end way using a cyclic loss. Experimental results demonstrate that our MultigraphGNet boosts the performance of an independent classifier when trained on the augmented brain multigraphs in comparison with training on a single CBT from each class. We hope that our framework can shed some light on the future research of multigraph augmentation from a single graph. Our MultigraphGNet source code is available at https://github.com/basiralab/MultigraphGNet .
Conditional Feature Embedding by Visual Clue Correspondence Graph for Person Re-Identification

Although Person Re-Identification has made impressive progress, difficult cases like occlusion, change of view-point, and similar clothing still bring great challenges. In order to tackle these challenges, extracting discriminative feature representation is crucial. Most of the existing methods focus on extracting ReID features from individual images separately. However, when matching two images, we propose that the ReID features of a query image should be dynamically adjusted based on the contextual information from the gallery image it matches. We call this type of ReID features conditional feature embedding. In this paper, we propose a novel ReID framework that extracts conditional feature embedding based on the aligned visual clues between image pairs, called Clue Alignment based Conditional Embedding (CACE-Net). CACE-Net applies an attention module to build a detailed correspondence graph between crucial visual clues in image pairs and uses discrepancy-based GCN to embed the obtained complex correspondence information into the conditional features. The experiments show that CACE-Net achieves state-of-the-art performance on three public datasets
Impact of Software Engineering Research in Practice: A Patent and Author Survey Analysis

Existing work on the practical impact of software engineering (SE) research examines industrial relevance rather than adoption of study results, hence the question of how results have been practically applied remains open. To answer this and investigate the outcomes of impactful research, we performed a quantitative and qualitative analysis of 4,354 SE patents citing 1,690 SE papers published in four leading SE venues between 1975-2017. Moreover, we conducted a survey on 475 authors of 593 top-cited and awarded publications, achieving 26% response rate. Overall, researchers have equipped practitioners with various tools, processes, and methods, and improved many existing products. SE practice values knowledge-seeking research and is impacted by diverse cross-disciplinary SE areas. Practitioner-oriented publication venues appear more impactful than researcher-oriented ones, while industry-related tracks in conferences could enhance their impact. Some research works did not reach a wide footprint due to limited funding resources or unfavorable cost-benefit trade-off of the proposed solutions. The need for higher SE research funding could be corroborated through a dedicated empirical study. In general, the assessment of impact is subject to its definition. Therefore, academia and industry could jointly agree on a formal description to set a common ground for subsequent research on the topic.
Improving DNN-based 2048 Players with Global Embedding

2048 is a popular game for which plenty of computer players have been created. However, many created 2048 players, especially all DNN-based ones, only implicitly use tile values as inputs and access tile position information. In this study, we take one of the best DNN-based 2048 players as a baseline and propose a 2048 player directly using both tile values and tile positions as inputs. Additionally, we explore the possibility of embedding all tile values and positions that we then concatenate with the network's regular value inputs. We first train these variations in a short session and then select the best two models with the baseline to be further trained in a long session. Our best two methods performed better than the baseline DNN player in both short and long training sessions.
Neighborly: A Sandbox for Simulation-based Emergent Narrative

This paper presents Neighborly, a customizable, community-scale social simulation engine for procedurally generating settlements of characters for use in research experimentation or entertainment media. Neighborly is a rational reconstruction of Talk of the Town (TotT), an earlier social simulation for emergent narrative focused on simulating small American towns and the townspeople’s lives. Based on Talk of the Town’s previous success as part of the experimental game Bad News, we wanted to reconstruct it as a general-use social simulation authoring tool. In this paper, we delineate the design space of TotT-like social simulation and compare TotT-likes to other academic projects and commercial social simulation games. Finally, we provide an overview of how Neighborly embodies the essence of TotT while offering users a customizable tool for creating community-scale social simulations.
Optical Character Recognition of Electrical Equipment Nameplate with Contrast Enhancement

Automatic identification of the electrical equipment via optical character recognition (OCR) of the electrical equipment nameplate images has been widely used as a primary inspection task of a substation. However, most existing OCR methods assume that the input image is obtained under ideal ambient lighting conditions and only focus on text detection and recognition. This commonly results in poor OCR performance when dealing with images with, e.g., reflection effects that are captured under complex ambient light conditions. In this paper, we propose an OCR system that can reliably handle the common reflection effects shown in the electrical equipment nameplate images captured under challenging ambient light conditions. Specifically, we did this by applying a robust contrast enhancement method that is based on a logarithmic mapping function (LMF) to the electrical equipment nameplate images prior to OCR with one of the state-of-the-art OCR networks, PPOCRv3. Experimental results with 24 test electrical equipment nameplate images show that the LMF contrast enhancement not only improves the image quality in terms of the lighting condition and overall scene statistics but also improves the overall OCR sentence recognition accuracy.
Agent X: Improving Exploration vs Exploitation in the State of the Art Angry Birds AI

AI agents have successfully employed deep reinforcement learning methods to surpass human performance in various new tasks over the past decade, notably including the domain of games. However, Angry Birds requires complex physical and spacial reasoning that is yet to be captured by such means. We present our logic-based Angry Birds AI which won the 2021 IJCAI AIBIRDS competition and propose a simple new method we call Second Order Thompson Sampling (SOTS) which allows for fine-tuning the balance between exploration and exploitation. We cover the competition scores of our entrant Agent X, its predecessor - the former state of the art Bambirds 2019, and the new and improved Bambirds 2021. We find that our agent has the best all-round performance, but would gain a lot by incorporating the improvements of Bambirds 2021. We list other potential areas of improvement for a future superhuman Angry Birds AI.
On the Verge of Solving Rocket League using Deep Reinforcement Learning and Sim-to-sim Transfer

Autonomously trained agents that are supposed to play video games reasonably well rely either on fast simulation speeds or heavy parallelization across thousands of machines running concurrently. This work explores a third way that is established in robotics, namely sim-to-real transfer, or if the game is considered a simulation itself, sim-to-sim transfer. In the case of Rocket League, we demonstrate that single behaviors of goalies and strikers can be successfully learned using Deep Reinforcement Learning in the simulation environment and transferred back to the original game. Although the implemented training simulation is to some extent inaccurate, the goalkeeping agent saves nearly 100% of its faced shots once transferred, while the striking agent scores in about 75% of cases. Therefore, the trained agent is robust enough and able to generalize to the target domain of Rocket League.
Turning Zeroes into Non-Zeroes: Sample Efficient Exploration with Monte Carlo Graph Search

Monte Carlo Tree Search (MCTS) has proven to be a staple method in Game Artificial Intelligence for creating agents that can perform well in complex environments without requiring domain-specific knowledge. The main downside of this planning based algorithm is the high computational budget needed to recommend an action. The fundamental cause of this is a vast search space caused by a high branching factor, and the difficulty to create a good heuristic function to guide the search without leveraging domain-specific knowledge. Recent advances in the field proposed a new planning based method called Monte Carlo Graph Search (MCGS), which uses a graph instead of a tree to plan its next action, reducing the branching factor and consequently increasing the performance of the search. In this paper, we propose several modifications that optimize the performance by increasing the sample efficiency of MCGS. The use of frontier for node selection, improving the rollout phase by doing stored rollouts, and a generalized approach to guide the search by incorporating a domain-independent online novelty detection method. Together these enhancements enable MCGS to solve sparse reward environments while using a significantly lower computational budget than MCTS.
Fostering integrative thinking through an online AR-based robot system analysis

This study explores an approach to learning robotics with augmented reality (AR) technology. The authors developed a learning environment for AR experiences with the virtual model of the mobile robot TurtleBot2. The environment was implemented in a workshop given online to second-year industrial engineering students as part of the course ‘Engineering of Production Systems’. The experience that was developed by the authors using Vuforia Studio was practiced by the student from their homes, using Vuforia View. In this practice, the students disassembled the virtual robot, learned about its main components, assembled it again, equipped it with a new component, and performed maintenance operations. In the evaluation study, the authors found that AR experiences contributed to students’ understanding of the robot system and to fostering their integrative thinking. Results of the study indicate the effectiveness of the proposed approach to learning robot systems, especially when students cannot access robotics labs.
Ethermal - Lightweight Thermal Feedback for VR Games

One of the major benefits of virtual reality (VR) lies in its ability to create a heightened sense of presence through visual and auditory modalities. To further enhance presence, research has investigated various ways to allow for tactile perception of objects in virtual environments. Among the different dimensions of tactile perception, thermal feedback is less explored, especially in the context of games which renders many existing approaches impractical due to their interference with game controllers. In this paper, we present Ethermal - a thermal feedback solution - designed to a) convey temperature information through the player’s hands while b) still permitting the use of standard VR game controllers. An evaluation with 10 players, using the device in a custom-made virtual environment to demonstrate different use cases of thermal feedback, suggests that an increased feel of general and spatial presence and experienced realism can be achieved even with simple technological means.
Salience as a Narrative Planning Step Cost Function

Psychological research has demonstrated that as we experience a story several features affect the salience of its events in memory. These features correspond to who? where? when? how? and why? questions about those events. Computational models of salience have been used in interactive narratives to measure which events people most easily remember from the past and which they expect more readily from the future. We use three example domains to show that events in sequences that are solutions to narrative planning problems are generally more salient with each other, and events in non-solution sequences are less salient with each other. This means that measuring the salience of a sequence of actions during planning can serve as an efficient cost function to improve the speed, and perhaps also the quality, of a narrative planner.
Infrared Image Recognition based on Hierarchical Attention Perceptron Model

Infrared image recognition is one of the key technologies in intelligent substations. Compared with tradition image processing methods, deep-learning algorithms are preferred due to their stronger modelling power. However, few training samples impair the model performance, and brings Transformer network into consideration. Transformer network utilizes context information such as object relations in an effective way, but the computation complexity is not controlled for dense predictions on high-resolution pictures. To solve this problem, a Hierarchical Attention Perceptron (HAP) module which conducts window-based self-attention manipulations is introduced. This local self-attention strategy is further enhanced by shifted window partition configuration which connects neighboring windows to provide more information. By replacing the original backbone network and encoder with HAP module, the Transformer network has higher recognition accuracy and lower computation complexity. Experiments based on field data help find the best architecture, and prove the effectiveness of the proposed method with the comparison to mainstream models.
PIFE: Permutation Invariant Feature Extractor for Danmaku Games

Dealing with unstructured complex patterns provides a challenge to existing reinforcement patterns. In this research, we propose a new model to overcome the difficulty in challenging danmaku games. Touhou Project is one of the bestknown games in the bullet hell genre also known as danmaku, where a player has to dodge complex patterns of bullets on the screen. Furthermore, the agent needs to react to the environment in real-time, which made existing methods having difficulties processing the high-volume data of objects; bullets, enemies, etc. We introduce an environment for the Touhou Project game‘東方花映塚~Phantasmagoria of Flower View.’ which manipulates the memory of the running game and enables to control the character. However, the game state information consists of unstructured and unordered data not amenable for training existing reinforcement learning models, as they are not invariant to order changes in the input. To overcome this issue, we propose a new pooling-based reinforcement learning approach that is able to handle permutation invariant inputs by extracting abstract values and merging them in an order-independent way. Experimental results corroborate the effectiveness of our approach which shows significantly increased scores compared to existing baseline approaches.
Adaptive Optimization with Fewer Epochs Improves Across-Scanner Generalization of U-Net Based Medical Image Segmentation

The U-Net architecture is widely used for medical image segmentation. However, accuracy has been observed to drop, sometimes dramatically, when U-Nets are trained on images that have been acquired with a specific scanner, and are applied to images from another scanner. This indicates an overfitting to image characteristics that are irrelevant to the semantic contents, and is usually mitigated with data augmentation. We argue that early stopping additionally improves across-scanner generalization, while greatly reducing training times. For this, we first observe that the widely used stochastic gradient descent (SGD) trains different U-Net layers at different speeds, and demonstrate that this problem is reduced by switching to AvaGrad, a recently proposed adaptive optimizer. On two different datasets, this allows us to match accuracies from nnUNets with default settings, 1000 epochs of SGD, by training for only 50 epochs with AvaGrad, and to exceed their results in the across-scanner setting. This benefit is specific to combining adaptive optimization and early stopping, since it can be matched neither by SGD with a low number of epochs, nor by Avagrad with many epochs. Finally, we demonstrate that the choice of optimizer can have important implications for domain adaptation. In particular, the SpotTUnet, which was recently proposed to automatically select layers for fine-tuning, arrives at very different policies depending on the optimizer.
Financial Inclusion and Information Availability on Banking Products and Services

Financial inclusion is one of the essential prerequisites for achieving personal and societal well-being. To achieve this, the customers at all levels be given timely information about the banking products and services on time. The paper seeks to analyze the information availability of rural people in Kerala, one of the most literate states in India, on various banking products and services. The study attempted to compare the opinion of 150 customers of public, private, and regional rural banks concerning information availability on banking products and services using Principal Component Analysis. The study found that the financial literacy rate and proper information availability on banking products and services of banking customers, particularly people at the bottom level are poor.