Reviewerly/exHarmony huggingface dataset:
Conjugated Discrete Distributions for Distributional Reinforcement  Learning

In this work we continue to build upon recent advances in reinforcement learning for finite Markov processes. A common approach among previous existing algorithms, both single-actor and distributed, is to either clip rewards or to apply a transformation method on Q-functions to handle a large variety of magnitudes in real discounted returns. We theoretically show that one of the most successful methods may not yield an optimal policy if we have a non-deterministic process. As a solution, we argue that distributional reinforcement learning lends itself to remedy this situation completely. By the introduction of a conjugated distributional operator we may handle a large class of transformations for real returns with guaranteed theoretical convergence. We propose an approximating single-actor algorithm based on this operator that trains agents directly on unaltered rewards using a proper distributional metric given by the Cram\'er distance. To evaluate its performance in a stochastic setting we train agents on a suite of 55 Atari 2600 games using sticky-actions and obtain state-of-the-art performance compared to other well-known algorithms in the Dopamine framework.
Exploring and Mitigating Gender Bias in Recommender Systems with  Explicit Feedback

Recommender systems are indispensable because they influence our day-to-day behavior and decisions by giving us personalized suggestions. Services like Kindle, Youtube, and Netflix depend heavily on the performance of their recommender systems to ensure that their users have a good experience and to increase revenues. Despite their popularity, it has been shown that recommender systems reproduce and amplify the bias present in the real world. The resulting feedback creates a self-perpetuating loop that deteriorates the user experience and results in homogenizing recommendations over time. Further, biased recommendations can also reinforce stereotypes based on gender or ethnicity, thus reinforcing the filter bubbles that we live in. In this paper, we address the problem of gender bias in recommender systems with explicit feedback. We propose a model to quantify the gender bias present in book rating datasets and in the recommendations produced by the recommender systems. Our main contribution is to provide a principled approach to mitigate the bias being produced in the recommendations. We theoretically show that the proposed approach provides unbiased recommendations despite biased data. Through empirical evaluation on publicly available book rating datasets, we further show that the proposed model can significantly reduce bias without significant impact on accuracy. Our method is model agnostic and can be applied to any recommender system. To demonstrate the performance of our model, we present the results on four recommender algorithms, two from the K-nearest neighbors family, UserKNN and ItemKNN, and the other two from the matrix factorization family, Alternating least square and Singular value decomposition.
A Structured Analysis of Information Security Incidents in the Maritime  Sector

Cyber attacks in the maritime sector can have a major impact on world economy. However, the severity of this threat can be underestimated because many attacks remain unknown or unnoticed. We present an overview about publicly known cyber incidents in the maritime sector from the past 20 years. In total, we found 90 publicly reported attacks and 15 proof of concepts. Furthermore, we interviewed five IT security experts from the maritime sector. The interviews put the results of our research in perspective and confirm that our view is comprehensive. However, the interviewees highlight that there is a high dark figure of unreported incidents and argue that threat information sharing may potentially be helpful for attack prevention. From these results, we extract threats for players in the maritime sector.
Generalized Net Model of Ant Lion Optimizer

In the presented paper, the functioning and the results of the work of a metaheuristic algorithm, known as Ant Lion Optimizer (ALO) is described using the apparatus of generalized nets (GNs). The developed GN-model mimic the optimization processes based on the behaviour of antlions and their prey in nature. To model such interactions, ants are required to move over the search space, and antlions are allowed to hunt them and become fitter using traps. The proposed GN-model executes the ALO algorithm conducting abovementioned basic steps and thus performing an optimal search. ALO GN-model considered here is another contribution to the open, namely “to present each of the artificial intelligence areas by GNs”.
Resolution to Sutner's Conjecture

Consider a game played on a simple graph $G = (V,E)$ where each vertex consists of a clickable light. Clicking any vertex $v$ toggles the on/off state of $v$ and its neighbors. One wins the game by finding a sequence of clicks that turns off all the lights. When $G$ is a $5 \times 5$ grid, this game was commercially available from Tiger Electronics as Lights Out. Sutner was one of the first to study these games mathematically. He found that when $d(G) = \text{dim}(\text{ker}(A + I))$ over the field $GF(2)$, where $A$ is the adjacency matrix of $G$, is 0 all initial configurations are solvable. When investigating $n \times n$ grid graphs, Sutner conjectured that $d_{2n+1} = 2d_{n} + \delta_{n}, \delta_{n} \in \{0,2\}, \delta_{2n+1} = \delta_{n}$, where $d_n = d(G)$ for $G$ an $n \times n$ grid graph. We resolve this conjecture in the affirmative. We use results from Sutner that give $d_n$ as the GCD of two polynomials in the ring $\mathbb{Z}_2[x]$. We then apply identities from Hunziker, Machiavelo, and Park that relate the polynomials of $(2n+1) \times (2n+1)$ grids and $n \times n$ grids. Finally, we use a result from Ore about the GCD of two products. Together these results allow us to prove Sutner's conjecture. We then go further and show for exactly which values of $n$ $\delta_n$ is 0 or 2.
Hybrid Acceleration Scheme for Variance Reduced Stochastic Optimization  Algorithms

Stochastic variance reduced optimization methods are known to be globally convergent while they suffer from slow local convergence, especially when moderate or high accuracy is needed. To alleviate this problem, we propose an optimization algorithm -- which we refer to as a hybrid acceleration scheme -- for a class of proximal variance reduced stochastic optimization algorithms. The proposed optimization scheme combines a fast locally convergent algorithm, such as a quasi--Newton method, with a globally convergent variance reduced stochastic algorithm, for instance SAGA or L--SVRG. Our global convergence result of the hybrid acceleration method is based on specific safeguard conditions that need to be satisfied for a step of the locally fast convergent method to be accepted. We prove that the sequence of the iterates generated by the hybrid acceleration scheme converges almost surely to a solution of the underlying optimization problem. We also provide numerical experiments that show significantly improved convergence of the hybrid acceleration scheme compared to the basic stochastic variance reduced optimization algorithm.
Procedural Kernel Networks

In the last decade Convolutional Neural Networks (CNNs) have defined the state of the art for many low level image processing and restoration tasks such as denoising, demosaicking, upscaling, or inpainting. However, on-device mobile photography is still dominated by traditional image processing techniques, and uses mostly simple machine learning techniques or limits the neural network processing to producing low resolution masks. High computational and memory requirements of CNNs, limited processing power and thermal constraints of mobile devices, combined with large output image resolutions (typically 8--12 MPix) prevent their wider application. In this work, we introduce Procedural Kernel Networks (PKNs), a family of machine learning models which generate parameters of image filter kernels or other traditional algorithms. A lightweight CNN processes the input image at a lower resolution, which yields a significant speedup compared to other kernel-based machine learning methods and allows for new applications. The architecture is learned end-to-end and is especially well suited for a wide range of low-level image processing tasks, where it improves the performance of many traditional algorithms. We also describe how this framework unifies some previous work applying machine learning for common image restoration tasks.
SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Video

We propose SelfRecon, a clothed human body reconstruction method that combines implicit and explicit representations to recover space-time coherent geometries from a monocular self-rotating human video. Explicit methods require a predefined template mesh for a given sequence, while the template is hard to acquire for a specific subject. Meanwhile, the fixed topology limits the reconstruction accuracy and clothing types. Implicit representation supports arbitrary topology and can represent high-fidelity geometry shapes due to its continuous nature. However, it is difficult to integrate multi-frame information to produce a consistent registration sequence for downstream applications. We propose to combine the advantages of both representations. We utilize differential mask loss of the explicit mesh to obtain the coherent overall shape, while the details on the implicit surface are refined with the differentiable neural rendering. Meanwhile, the explicit mesh is updated periodically to adjust its topology changes, and a consistency loss is designed to match both representations. Compared with existing methods, SelfRecon can produce high-fidelity surfaces for arbitrary clothed humans with self-supervised optimization. Extensive experimental results demonstrate its effectiveness on real captured monocular videos. The source code is available at https://github.com/jby1993/SelfReconCode.
Virtual Classes of Character Stacks

In this paper, we extend the Topological Quantum Field Theory developed by Gonz\'alez-Prieto, Logares, and Mu\~noz for computing virtual classes of $G$-representation varieties of closed orientable surfaces in the Grothendieck ring of varieties to the setting of the character stacks. To this aim, we define a suitable Grothendieck ring of representable stacks, over which this Topological Quantum Field Theory is defined. In this way, we compute the virtual class of the character stack over $BG$, that is, a motivic decomposition of the representation variety with respect to the natural adjoint action. We apply this framework in two cases providing explicit expressions for the virtual classes of the character stacks of closed orientable surfaces of arbitrary genus. First, in the case of the affine linear group of rank $1$, the virtual class of the character stack fully remembers the natural adjoint action, in particular, the virtual class of the character variety can be straightforwardly derived. Second, we consider the non-connected group $\mathbb{G}_m \rtimes \mathbb{Z}/2\mathbb{Z}$, and we show how our theory allows us to compute motivic information of the character stacks where the classical na\"ive point-counting method fails.
Inexact Graph Matching Using Centrality Measures

Graph matching is the process of computing the similarity between two graphs. Depending on the requirement, it can be exact or inexact. Exact graph matching requires a strict correspondence between nodes of two graphs, whereas inexact matching allows some flexibility or tolerance during the graph matching. In this chapter, we describe an approximate inexact graph matching by reducing the size of the graphs using different centrality measures. Experimental evaluation shows that it can reduce running time for inexact graph matching.
Finding, Scoring and Explaining Arguments in Bayesian Networks

We propose a new approach to explain Bayesian Networks. The approach revolves around a new definition of a probabilistic argument and the evidence it provides. We define a notion of independent arguments, and propose an algorithm to extract a list of relevant, independent arguments given a Bayesian Network, a target node and a set of observations. To demonstrate the relevance of the arguments, we show how we can use the extracted arguments to approximate message passing. Finally, we show a simple scheme to explain the arguments in natural language.
Probing Pretrained Models of Source Code

Deep learning models are widely used for solving challenging code processing tasks, such as code generation or code summarization. Traditionally, a specific model architecture was carefully built to solve a particular code processing task. However, recently general pretrained models such as CodeBERT or CodeT5 have been shown to outperform task-specific models in many applications. While pretrained models are known to learn complex patterns from data, they may fail to understand some properties of source code. To test diverse aspects of code understanding, we introduce a set of diagnosting probing tasks. We show that pretrained models of code indeed contain information about code syntactic structure and correctness, the notions of identifiers, data flow and namespaces, and natural language naming. We also investigate how probing results are affected by using code-specific pretraining objectives, varying the model size, or finetuning.
Numerical solution of the mix quantum rabi model

The mixed quantum rabi model is generated based on the interaction of single photon and two photon. Based on the mix QRM under one-qubit, this paper extended it to two-qubit Hamiltonian and solve it, then analyzed the variation law of different parameter variables and the symmetry of matrix.
A Surrogate-Assisted Controller for Expensive Evolutionary Reinforcement Learning

The integration of Reinforcement Learning (RL) and Evolutionary Algorithms (EAs) aims at simultaneously exploiting the sample efficiency as well as the diversity and robustness of the two paradigms. Recently, hybrid learning frameworks based on this principle have achieved great success in various challenging robot control tasks. However, in these methods, policies from the genetic population are evaluated via interactions with the real environments, limiting their applicability in computationally expensive problems. In this work, we propose Surrogate-assisted Controller (SC), a novel and efficient module that can be integrated into existing frameworks to alleviate the computational burden of EAs by partially replacing the expensive policy evaluation. The key challenge in applying this module is to prevent the optimization process from being misled by the possible false minima introduced by the surrogate. To address this issue, we present two strategies for SC to control the workflow of hybrid frameworks. Experiments on six continuous control tasks from the OpenAI Gym platform show that SC can not only significantly reduce the cost of fitness evaluations, but also boost the performance of the original hybrid frameworks with collaborative learning and evolutionary processes.
In the Arms of a Robot: Designing Autonomous Hugging Robots with Intra-Hug Gestures

Hugs are complex affective interactions that often include gestures like squeezes. We present six new guidelines for designing interactive hugging robots, which we validate through two studies with our custom robot. To achieve autonomy, we investigated robot responses to four human intra-hug gestures: holding, rubbing, patting, and squeezing. Thirty-two users each exchanged and rated sixteen hugs with an experimenter-controlled HuggieBot 2.0. The robot's inflated torso's microphone and pressure sensor collected data of the subjects' demonstrations that were used to develop a perceptual algorithm that classifies user actions with 88\% accuracy. Users enjoyed robot squeezes, regardless of their performed action, they valued variety in the robot response, and they appreciated robot-initiated intra-hug gestures. From average user ratings, we created a probabilistic behavior algorithm that chooses robot responses in real time. We implemented improvements to the robot platform to create HuggieBot 3.0 and then validated its gesture perception system and behavior algorithm with sixteen users. The robot's responses and proactive gestures were greatly enjoyed. Users found the robot more natural, enjoyable, and intelligent in the last phase of the experiment than in the first. After the study, they felt more understood by the robot and thought robots were nicer to hug.
PERANCANGAN SISTEM INFORMASI PELAYANAN PEMESANAN BERBASIS WEB RUMAH MAKAN CIBIUK SERANG

Cibiuk Restaurant is a typical Sundanese restaurant that is currently the most popular both in the culinary field, but in ordering services it still uses a manual process in its operations. From the food and beverage ordering service process as well as transaction reports, the number of customers who come to order sometimes makes the waiter make mistakes in writing the amount and the service is still less efficient, this is a problem for the waiters at Cibiuk Restaurant. With the increasing development of increasingly high technology, demands for services that are increasingly easy, fast, and practical so as not to depend on notepads. So the purpose of this study is to create a website-based customer service ordering information system using the FCFS method. To make it easier to assist orders in providing First Come First Served services or prioritizing orders that come first is one of the goals of service satisfaction to customers. This researcher collected data using interview and observation techniques at Cibiuk Restaurant. The method used in designing the system is the waterfall method and uses a programming language, namely PHP and MySQL database, with the modeling applied as a prototype. From the results of the design of this service information system, it is hoped that it can help cashiers to be more effective and efficient in terms of time and to improve ordering services at Cibiuk Serang Restaurant.
You May Not Need Ratio Clipping in PPO

Proximal Policy Optimization (PPO) methods learn a policy by iteratively performing multiple mini-batch optimization epochs of a surrogate objective with one set of sampled data. Ratio clipping PPO is a popular variant that clips the probability ratios between the target policy and the policy used to collect samples. Ratio clipping yields a pessimistic estimate of the original surrogate objective, and has been shown to be crucial for strong performance. We show in this paper that such ratio clipping may not be a good option as it can fail to effectively bound the ratios. Instead, one can directly optimize the original surrogate objective for multiple epochs; the key is to find a proper condition to early stop the optimization epoch in each iteration. Our theoretical analysis sheds light on how to determine when to stop the optimization epoch, and call the resulting algorithm Early Stopping Policy Optimization (ESPO). We compare ESPO with PPO across many continuous control tasks and show that ESPO significantly outperforms PPO. Furthermore, we show that ESPO can be easily scaled up to distributed training with many workers, delivering strong performance as well.
Continuous-Time Video Generation via Learning Motion Dynamics with Neural ODE

In order to perform unconditional video generation, we must learn the distribution of the real-world videos. In an effort to synthesize high-quality videos, various studies attempted to learn a mapping function between noise and videos, including recent efforts to separate motion distribution and appearance distribution. Previous methods, however, learn motion dynamics in discretized, fixed-interval timesteps, which is contrary to the continuous nature of motion of a physical body. In this paper, we propose a novel video generation approach that learns separate distributions for motion and appearance, the former modeled by neural ODE to learn natural motion dynamics. Specifically, we employ a two-stage approach where the first stage converts a noise vector to a sequence of keypoints in arbitrary frame rates, and the second stage synthesizes videos based on the given keypoints sequence and the appearance noise vector. Our model not only quantitatively outperforms recent baselines for video generation, but also demonstrates versatile functionality such as dynamic frame rate manipulation and motion transfer between two datasets, thus opening new doors to diverse video generation applications.
OPD: Single-view 3D Openable Part Detection

We address the task of predicting what parts of an object can open and how they move when they do so. The input is a single image of an object, and as output we detect what parts of the object can open, and the motion parameters describing the articulation of each openable part. To tackle this task, we create two datasets of 3D objects: OPDSynth based on existing synthetic objects, and OPDReal based on RGBD reconstructions of real objects. We then design OPDRCNN, a neural architecture that detects openable parts and predicts their motion parameters. Our experiments show that this is a challenging task especially when considering generalization across object categories, and the limited amount of information in a single image. Our architecture outperforms baselines and prior work especially for RGB image inputs. Short video summary at https://www.youtube.com/watch?v=P85iCaD0rfc
BEAS: Blockchain Enabled Asynchronous &amp; Secure Federated Machine Learning

Federated Learning (FL) enables multiple parties to distributively train a ML model without revealing their private datasets. However, it assumes trust in the centralized aggregator which stores and aggregates model updates. This makes it prone to gradient tampering and privacy leakage by a malicious aggregator. Malicious parties can also introduce backdoors into the joint model by poisoning the training data or model gradients. To address these issues, we present BEAS, the first blockchain-based framework for N-party FL that provides strict privacy guarantees of training data using gradient pruning (showing improved differential privacy compared to existing noise and clipping based techniques). Anomaly detection protocols are used to minimize the risk of data-poisoning attacks, along with gradient pruning that is further used to limit the efficacy of model-poisoning attacks. We also define a novel protocol to prevent premature convergence in heterogeneous learning environments. We perform extensive experiments on multiple datasets with promising results: BEAS successfully prevents privacy leakage from dataset reconstruction attacks, and minimizes the efficacy of poisoning attacks. Moreover, it achieves an accuracy similar to centralized frameworks, and its communication and computation overheads scale linearly with the number of participants.
Unsupervised Feature Selection Method Based on Sensitivity and Correlation Concepts for Multiclass Problems

In this chapter, we describe an unsupervised filter method, called Sensitivity and Correlation-based Feature Selection (SCFS). It is based on subtractive clustering and the concepts of sensitivity and Pearson’s correlation. We show how this method is employed as the fitness function in a genetic algorithm (GA) in order to evaluate feature subsets. Informally, the method works as follows. First, the sensitivity index of each feature is computed by applying the subtractive clustering technique to the other features. Then, the results of the sensitivity index are used to calculate the relevance of the features.