OpenRaiser/Intern-Atlas huggingface dataset:
Oscillatory Neural Network as Hetero-Associative Memory for Image Edge Detection
Venue: arXiv cs.NE 2022
Year: 2022

The increasing amount of data to be processed on edge devices, such as cameras, has motivated Artificial Intelligence (AI) integration at the edge. Typical image processing methods performed at the edge, such as feature extraction or edge detection, use convolutional filters that are energy, computation, and memory hungry algorithms. But edge devices and cameras have scarce computational resources, bandwidth, and power and are limited due to privacy constraints to send data over to the cloud. Thus, there is a need to process image data at the edge. Over the years, this need has incited a lot of interest in implementing neuromorphic computing at the edge. Neuromorphic systems aim to emulate the biological neural functions to achieve energy-efficient computing. Recently, Oscillatory Neural Networks (ONN) present a novel brain-inspired computing approach by emulating brain oscillations to perform autoassociative memory types of applications. To speed up image edge detection and reduce its power consumption, we perform an in-depth investigation with ONNs. We propose a novel image processing method by using ONNs as a hetero-associative memory (HAM) for image edge detection. We simulate our ONN-HAM solution using first, a Matlab emulator, and then a fully digital ONN design. We show results on gray scale square evaluation maps, also on black and white and gray scale 28x28 MNIST images and finally on black and white 512x512 standard test images. We compare our solution with standard edge detection filters such as Sobel and Canny. Finally, using the fully digital design simulation results, we report on timing and resource characteristics, and evaluate its feasibility for real-time image processing applications. Our digital ONN-HAM solution can process images with up to 120x120 pixels (166 MHz system frequency) respecting real-time camera constraints. This work is the first to explore ONNs as hetero-associative memory for image processing applications.
Raman Spectrum Matching with Contrastive Representation Learning
Venue: arXiv cs.LG 2022
Year: 2022

Raman spectroscopy is an effective, low-cost, non-intrusive technique often used for chemical identification. Typical approaches are based on matching observations to a reference database, which requires careful preprocessing, or supervised machine learning, which requires a fairly large number of training observations from each class. We propose a new machine learning technique for Raman spectrum matching, based on contrastive representation learning, that requires no preprocessing and works with as little as a single reference spectrum from each class. On three datasets we demonstrate that our approach significantly improves or is on par with the state of the art in prediction accuracy, and we show how to compute conformal prediction sets with specified frequentist coverage. Based on our findings, we believe contrastive representation learning is a promising alternative to existing methods for Raman spectrum matching.
Kron Reduction and Effective Resistance of Directed Graphs
Venue: arXiv cs.DM 2022
Year: 2022

In network theory, the concept of effective resistance is a distance measure on a graph that relates the global network properties to individual connections between nodes. In addition, the Kron reduction method is a standard tool for reducing or eliminating the desired nodes, which preserves the interconnection structure and the effective resistance of the original graph. Although these two graph-theoretic concepts stem from the electric network on an undirected graph, they also have a number of applications throughout a wide variety of other fields. In this study, we propose a generalization of a Kron reduction for directed graphs. Furthermore, we prove that this reduction method preserves the structure of the original graphs, such as the strong connectivity or weight balance. In addition, we generalize the effective resistance to a directed graph using Markov chain theory, which is invariant under a Kron reduction. Although the effective resistance of our proposal is asymmetric, we prove that it induces two novel graph metrics in general strongly connected directed graphs. In particular, the effective resistance captures the commute and covering times for strongly connected weight balanced directed graphs. Finally, we compare our method with existing approaches and relate the hitting probability metrics and effective resistance in a stochastic case. In addition, we show that the effective resistance in a doubly stochastic case is the same as the resistance distance in an ergodic Markov chain.
NeuralKG: An Open Source Library for Diverse Representation Learning of Knowledge Graphs
Venue: arXiv cs.LG 2022
Year: 2022

NeuralKG is an open-source Python-based library for diverse representation learning of knowledge graphs. It implements three different series of Knowledge Graph Embedding (KGE) methods, including conventional KGEs, GNN-based KGEs, and Rule-based KGEs. With a unified framework, NeuralKG successfully reproduces link prediction results of these methods on benchmarks, freeing users from the laborious task of reimplementing them, especially for some methods originally written in non-python programming languages. Besides, NeuralKG is highly configurable and extensible. It provides various decoupled modules that can be mixed and adapted to each other. Thus with NeuralKG, developers and researchers can quickly implement their own designed models and obtain the optimal training methods to achieve the best performance efficiently. We built an website in http://neuralkg.zjukg.cn to organize an open and shared KG representation learning community. The source code is all publicly released at https://github.com/zjukg/NeuralKG.
Evolutionary scheduling of university activities based on consumption forecasts to minimise electricity costs
Venue: arXiv cs.LG 2022
Year: 2022

This paper presents a solution to a predict then optimise problem which goal is to reduce the electricity cost of a university campus. The proposed methodology combines a multi-dimensional time series forecast and a novel approach to large-scale optimization. Gradient-boosting method is applied to forecast both generation and consumption time-series of the Monash university campus for the month of November 2020. For the consumption forecasts we employ log transformation to model trend and stabilize variance. Additional seasonality and trend features are added to the model inputs when applicable. The forecasts obtained are used as the base load for the schedule optimisation of university activities and battery usage. The goal of the optimisation is to minimize the electricity cost consisting of the price of electricity and the peak electricity tariff both altered by the load from class activities and battery use as well as the penalty of not scheduling some optional activities. The schedule of the class activities is obtained through evolutionary optimisation using the covariance matrix adaptation evolution strategy and the genetic algorithm. This schedule is then improved through local search by testing possible times for each activity one-by-one. The battery schedule is formulated as a mixed-integer programming problem and solved by the Gurobi solver. This method obtains the second lowest cost when evaluated against 6 other methods presented at an IEEE competition that all used mixed-integer programming and the Gurobi solver to schedule both the activities and the battery use. The code and data used for the paper are publicly available.
Dynamic Traffic Allocation in IEEE 802.11be Multi-link WLANs
Venue: arXiv cs.NI 2022
Year: 2022

The multi-link operation (MLO) is a key feature of the next IEEE 802.11be Extremely High Throughput amendment. Through its adoption, it is expected to enhance users' experience by improving throughput rates and latency. However, potential MLO gains are tied to how traffic is distributed across the multiple radio interfaces. In this paper, we introduce a traffic manager atop MLO, and evaluate different high-level traffic-to-link allocation policies to distribute incoming traffic over the set of enabled interfaces. Following a flow-level approach, we compare both non-dynamic and dynamic traffic balancing policy types. The results show that the use of a dynamic policy, along with MLO, allows to significantly reduce the congestion suffered by traffic flows, enhancing the traffic delivery in all the evaluated scenarios, and in particular improving the quality of service received by video flows. Moreover, we show that the adoption of MLO in future Wi-Fi networks improves also the coexistence with non-MLO networks, which results in performance gains for both MLO and non-MLO networks.
Deep Dirichlet uncertainty for unsupervised out-of-distribution detection of eye fundus photographs in glaucoma screening
Venue: arXiv cs.CV 2022
Year: 2022

The development of automatic tools for early glaucoma diagnosis with color fundus photographs can significantly reduce the impact of this disease. However, current state-of-the-art solutions are not robust to real-world scenarios, providing over-confident predictions for out-of-distribution cases. With this in mind, we propose a model based on the Dirichlet distribution that allows to obtain class-wise probabilities together with an uncertainty estimation without exposure to out-of-distribution cases. We demonstrate our approach on the AIROGS challenge. At the start of the final test phase (8 Feb. 2022), our method had the highest average score among all submissions.
New Antenna Selection Schemes for Full-Duplex Cooperative MIMO-NOMA Systems
Venue: arXiv cs.IT 2022
Year: 2022

In this paper, we address the antenna selection (AS) problem in full-duplex (FD) cooperative non-orthogonal multiple access (NOMA) systems, where a multi-antenna FD relay bridges the connection between the multi-antenna base station and NOMA far user. Specifically, two AS schemes, namely max-$\SUu$ and max-$\SUuu$, are proposed to maximize the end-to-end signal-to-interference-plus-noise ratio at either or both near and far users, respectively. Moreover, a two-stage AS scheme, namely quality-of-service (QoS) provisioning scheme, is designed to realize a specific rate at the far user while improving the near user's rate. To enhance the performance of the QoS provisioning AS scheme, the idea of dynamic antenna clustering is applied at the relay to adaptively partition the relay's antennas into transmit and receive subsets. The proposed AS schemes' exact outage probability and achievable rate expressions are derived. To provide more insight, closed-form asymptotic outage probability expressions for the max-$\SUu$ and max-$\SUuu$ AS schemes are obtained. Our results show that while the QoS provisioning AS scheme can deliver a near-optimal performance for static antenna setup at the relay, it provides up to $12\%$ average sum rate gain over the optimum AS selection with fixed antenna setup.
Bayesian autoencoders with uncertainty quantification: Towards trustworthy anomaly detection
Venue: arXiv cs.LG 2022
Year: 2022

Despite numerous studies of deep autoencoders (AEs) for unsupervised anomaly detection, AEs still lack a way to express uncertainty in their predictions, crucial for ensuring safe and trustworthy machine learning systems in high-stake applications. Therefore, in this work, the formulation of Bayesian autoencoders (BAEs) is adopted to quantify the total anomaly uncertainty, comprising epistemic and aleatoric uncertainties. To evaluate the quality of uncertainty, we consider the task of classifying anomalies with the additional option of rejecting predictions of high uncertainty. In addition, we use the accuracy-rejection curve and propose the weighted average accuracy as a performance metric. Our experiments demonstrate the effectiveness of the BAE and total anomaly uncertainty on a set of benchmark datasets and two real datasets for manufacturing: one for condition monitoring, the other for quality inspection.
Automorphisms of Set Families and of Families of Cliques in an Interval Graph in FPT Time
Venue: arXiv cs.DM 2022
Year: 2022

We consider the following problem closely related to graph isomorphism. In a simplified version, the task is to compute the automorphism group of a given set family (or a hypergraph), that is, the group of all automorphisms of the given sets which are compatible with some permutation of their elements. In a general setting, the set family in question is a collection of cliques (called marked cliques) of a given interval graph, and the task is to compute the group of all permutations of the cliques which result from some automorpism of the underlying interval graph. This problem is obviously at least as hard as the graph isomorphism (GI-hard) already in the simplified version -- consider the set family of edges of a graph, and we give an FPT-time algorithm parameterized by the maximum number of sets in the family which are incomparable by inclusion (its antichain size). To our best knowledge, the general version of the problem has not been formulated in the literature so far. The problem has been inspired by the research of special cases of the isomorphism problem of chordal graphs; namely, the simplified set-family version is the core of our FPT algorithm for the isomorphism of so-called Sd-graphs [MFCS 2021], and the general version extends and improves a cumbersome technical step in our FPT algorithm for the isomorphism of chordal graphs of bounded leafage [WALCOM 2022]. The new algorithm combines two classical tools -- PQ-trees of interval graphs and Babai's tower-of-groups, in a nontrivial way.
Improving Amharic Handwritten Word Recognition Using Auxiliary Task
Venue: arXiv cs.CV 2022
Year: 2022

Amharic is one of the official languages of the Federal Democratic Republic of Ethiopia. It is one of the languages that use an Ethiopic script which is derived from Gee'z, ancient and currently a liturgical language. Amharic is also one of the most widely used literature-rich languages of Ethiopia. There are very limited innovative and customized research works in Amharic optical character recognition (OCR) in general and Amharic handwritten text recognition in particular. In this study, Amharic handwritten word recognition will be investigated. State-of-the-art deep learning techniques including convolutional neural networks together with recurrent neural networks and connectionist temporal classification (CTC) loss were used to make the recognition in an end-to-end fashion. More importantly, an innovative way of complementing the loss function using the auxiliary task from the row-wise similarities of the Amharic alphabet was tested to show a significant recognition improvement over a baseline method. Such findings will promote innovative problem-specific solutions as well as will open insight to a generalized solution that emerges from problem-specific domains.
Modeling moving boundary value problems in electrochemical machining
Venue: arXiv cs.CE 2022
Year: 2022

This work presents a new approach to efficiently model the cathode in the moving boundary value problem of electrochemical machining. Until recently, the process simulation with finite elements had the drawback of remeshing required by the changing surface geometries. This disadvantage was overcome by a novel model formulation for the anodic dissolution that utilizes effective material parameters as well as the dissolution level as an internal variable and, thereby, does not require remeshing. Now, we extend this concept to model arbitrarily shaped and moving cathodes. Two methodologies are investigated to describe the time varying position of the cathode. In the first approach, we change the electric conductivity of elements within the cathode and, in a second approach, we apply Dirichlet boundary conditions on the nodes of corresponding elements. For both methods, elements on the cathode's surface are treated with effective material parameters. This procedure allows for the efficient simulation of industrially relevant, complex geometries without mesh adaptation. The model's performance is validated by means of analytical, numerical and experimental results from the literature. The short computation times make the approach interesting for industrial applications.
Learning Relative Return Policies With Upside-Down Reinforcement Learning
Venue: arXiv cs.LG 2022
Year: 2022

Lately, there has been a resurgence of interest in using supervised learning to solve reinforcement learning problems. Recent work in this area has largely focused on learning command-conditioned policies. We investigate the potential of one such method -- upside-down reinforcement learning -- to work with commands that specify a desired relationship between some scalar value and the observed return. We show that upside-down reinforcement learning can learn to carry out such commands online in a tabular bandit setting and in CartPole with non-linear function approximation. By doing so, we demonstrate the power of this family of methods and open the way for their practical use under more complicated command structures.
FedCAT: Towards Accurate Federated Learning via Device Concatenation
Venue: arXiv cs.LG 2022
Year: 2022

As a promising distributed machine learning paradigm, Federated Learning (FL) enables all the involved devices to train a global model collaboratively without exposing their local data privacy. However, for non-IID scenarios, the classification accuracy of FL models decreases drastically due to the weight divergence caused by data heterogeneity. Although various FL variants have been studied to improve model accuracy, most of them still suffer from the problem of non-negligible communication and computation overhead. In this paper, we introduce a novel FL approach named Fed-Cat that can achieve high model accuracy based on our proposed device selection strategy and device concatenation-based local training method. Unlike conventional FL methods that aggregate local models trained on individual devices, FedCat periodically aggregates local models after their traversals through a series of logically concatenated devices, which can effectively alleviate the model weight divergence problem. Comprehensive experimental results on four well-known benchmarks show that our approach can significantly improve the model accuracy of state-of-the-art FL methods without causing extra communication overhead.
Stochastic Games with Synchronizing Objectives
Venue: arXiv cs.GT 2022
Year: 2022

We consider two-player stochastic games played on a finite graph for infinitely many rounds. Stochastic games generalize both Markov decision processes (MDP) by adding an adversary player, and two-player deterministic games by adding stochasticity. The outcome of the game is a sequence of distributions over the states of the game graph. We consider synchronizing objectives, which require the probability mass to accumulate in a set of target states, either always, once, infinitely often, or always after some point in the outcome sequence; and the winning modes of sure winning (if the accumulated probability is equal to 1) and almost-sure winning (if the accumulated probability is arbitrarily close to 1). We present algorithms to compute the set of winning distributions for each of these synchronizing modes, showing that the corresponding decision problem is PSPACE-complete for synchronizing once and infinitely often, and PTIME-complete for synchronizing always and always after some point. These bounds are remarkably in line with the special case of MDPs, while the algorithmic solution and proof technique are considerably more involved, even for deterministic games. This is because those games have a flavour of imperfect information, in particular they are not determined and randomized strategies need to be considered, even if there is no stochastic choice in the game graph. Moreover, in combination with stochasticity in the game graph, finite-memory strategies are not sufficient in general (for synchronizing infinitely often).
Core-periphery detection in hypergraphs
Venue: arXiv cs.SI 2022
Year: 2022

Core-periphery detection is a key task in exploratory network analysis where one aims to find a core, a set of nodes well-connected internally and with the periphery, and a periphery, a set of nodes connected only (or mostly) with the core. In this work we propose a model of core-periphery for higher-order networks modeled as hypergraphs and we propose a method for computing a core-score vector that quantifies how close each node is to the core. In particular, we show that this method solves the corresponding non-convex core-periphery optimization problem globally to an arbitrary precision. This method turns out to coincide with the computation of the Perron eigenvector of a nonlinear hypergraph operator, suitably defined in term of the incidence matrix of the hypergraph, generalizing recently proposed centrality models for hypergraphs. We perform several experiments on synthetic and real-world hypergraphs showing that the proposed method outperforms alternative core-periphery detection algorithms, in particular those obtained by transferring established graph methods to the hypergraph setting via clique expansion.
Behaviorally Grounded Model-Based and Model Free Cost Reduction in a Simulated Multi-Echelon Supply Chain
Venue: arXiv cs.LG 2022
Year: 2022

Amplification and phase shift in ordering signals, commonly referred to as bullwhip, are responsible for both excessive strain on real world inventory management systems, stock outs, and unnecessary capital reservation though safety stock building. Bullwhip is a classic, yet persisting, problem with reverberating consequences in inventory management. Research on bullwhip has consistently emphasized behavioral influences for this phenomenon and leveraged behavioral ordering models to suggest interventions. However more recent model-free approaches have also seen success. In this work, the author develops algorithmic approaches towards mitigating bullwhip using both behaviorally grounded model-based approaches alongside a model-free dual deep Q-network reinforcement learning approach. In addition to exploring the utility of this specific model-free architecture to multi-echelon supply chains with imperfect information sharing and information delays, the author directly compares the performance of these model-based and model-free approaches. In doing so, this work highlights both the insights gained from exploring model-based approaches in the context of prior behavioral operations management literature and emphasizes the complementary nature of model-based and model-free approaches in approaching behaviorally grounded supply chain management problems.
GenéLive! Generating Rhythm Actions in Love Live!
Venue: arXiv cs.LG 2022
Year: 2022

This article presents our generative model for rhythm action games together with applications in business operations. Rhythm action games are video games in which the player is challenged to issue commands at the right timings during a music session. The timings are rendered in the chart, which consists of visual symbols, called notes, flying through the screen. We introduce our deep generative model, GenéLive!, which outperforms the state-of-the-art model by taking into account musical structures through beats and temporal scales. Thanks to its favorable performance, GenéLive! was put into operation at KLab Inc., a Japan-based video game developer, and reduced the business cost of chart generation by as much as half. The application target included the phenomenal "Love Live!," which has more than 10 million users across Asia and beyond, and is one of the few rhythm action franchises that has led the online era of the genre. In this article, we evaluate the generative performance of GenéLive! using production datasets at KLab as well as open datasets for reproducibility, while the model continues to operate in their business. Our code and the model, tuned and trained using a supercomputer, are publicly available.