OpenRaiser/Intern-Atlas huggingface dataset:
CV4Code: Sourcecode Understanding via Visual Code Representations
Venue: arXiv cs.SE 2022
Year: 2022

We present CV4Code, a compact and effective computer vision method for sourcecode understanding. Our method leverages the contextual and the structural information available from the code snippet by treating each snippet as a two-dimensional image, which naturally encodes the context and retains the underlying structural information through an explicit spatial representation. To codify snippets as images, we propose an ASCII codepoint-based image representation that facilitates fast generation of sourcecode images and eliminates redundancy in the encoding that would arise from an RGB pixel representation. Furthermore, as sourcecode is treated as images, neither lexical analysis (tokenisation) nor syntax tree parsing is required, which makes the proposed method agnostic to any particular programming language and lightweight from the application pipeline point of view. CV4Code can even featurise syntactically incorrect code which is not possible from methods that depend on the Abstract Syntax Tree (AST). We demonstrate the effectiveness of CV4Code by learning Convolutional and Transformer networks to predict the functional task, i.e. the problem it solves, of the source code directly from its two-dimensional representation, and using an embedding from its latent space to derive a similarity score of two code snippets in a retrieval setup. Experimental results show that our approach achieves state-of-the-art performance in comparison to other methods with the same task and data configurations. For the first time we show the benefits of treating sourcecode understanding as a form of image processing task.
Cyclic Proofs, Hypersequents, and Transitive Closure Logic
Venue: arXiv cs.LO 2022
Year: 2022

We propose a cut-free cyclic system for Transitive Closure Logic (TCL) based on a form of hypersequents, suitable for automated reasoning via proof search. We show that previously proposed sequent systems are cut-free incomplete for basic validities from Kleene Algebra (KA) and Propositional Dynamic Logic (PDL), over standard translations. On the other hand, our system faithfully simulates known cyclic systems for KA and PDL, thereby inheriting their completeness results. A peculiarity of our system is its richer correctness criterion, exhibiting 'alternating traces' and necessitating a more intricate soundness argument than for traditional cyclic proofs.
DPO: Dynamic-Programming Optimization on Hybrid Constraints
Venue: arXiv cs.LO 2022
Year: 2022

In Bayesian inference, the most probable explanation (MPE) problem requests a variable instantiation with the highest probability given some evidence. Since a Bayesian network can be encoded as a literal-weighted CNF formula $φ$, we study Boolean MPE, a more general problem that requests a model $τ$ of $φ$ with the highest weight, where the weight of $τ$ is the product of weights of literals satisfied by $τ$. It is known that Boolean MPE can be solved via reduction to (weighted partial) MaxSAT. Recent work proposed DPMC, a dynamic-programming model counter that leverages graph-decomposition techniques to construct project-join trees. A project-join tree is an execution plan that specifies how to conjoin clauses and project out variables. We build on DPMC and introduce DPO, a dynamic-programming optimizer that exactly solves Boolean MPE. By using algebraic decision diagrams (ADDs) to represent pseudo-Boolean (PB) functions, DPO is able to handle disjunctive clauses as well as XOR clauses. (Cardinality constraints and PB constraints may also be compactly represented by ADDs, so one can further extend DPO's support for hybrid inputs.) To test the competitiveness of DPO, we generate random XOR-CNF formulas. On these hybrid benchmarks, DPO significantly outperforms MaxHS, UWrMaxSat, and GaussMaxHS, which are state-of-the-art exact solvers for MaxSAT.
Semantically Accurate Super-Resolution Generative Adversarial Networks
Venue: arXiv cs.CV 2022
Year: 2022

This work addresses the problems of semantic segmentation and image super-resolution by jointly considering the performance of both in training a Generative Adversarial Network (GAN). We propose a novel architecture and domain-specific feature loss, allowing super-resolution to operate as a pre-processing step to increase the performance of downstream computer vision tasks, specifically semantic segmentation. We demonstrate this approach using Nearmap's aerial imagery dataset which covers hundreds of urban areas at 5-7 cm per pixel resolution. We show the proposed approach improves perceived image quality as well as quantitative segmentation accuracy across all prediction classes, yielding an average accuracy improvement of 11.8% and 108% at 4x and 32x super-resolution, compared with state-of-the art single-network methods. This work demonstrates that jointly considering image-based and task-specific losses can improve the performance of both, and advances the state-of-the-art in semantic-aware super-resolution of aerial imagery.
Hyperparameter Optimization with Neural Network Pruning
Venue: arXiv cs.CV 2022
Year: 2022

Since the deep learning model is highly dependent on hyperparameters, hyperparameter optimization is essential in developing deep learning model-based applications, even if it takes a long time. As service development using deep learning models has gradually become competitive, many developers highly demand rapid hyperparameter optimization algorithms. In order to keep pace with the needs of faster hyperparameter optimization algorithms, researchers are focusing on improving the speed of hyperparameter optimization algorithm. However, the huge time consumption of hyperparameter optimization due to the high computational cost of the deep learning model itself has not been dealt with in-depth. Like using surrogate model in Bayesian optimization, to solve this problem, it is necessary to consider proxy model for a neural network (N_B) to be used for hyperparameter optimization. Inspired by the main goal of neural network pruning, i.e., high computational cost reduction and performance preservation, we presumed that the neural network (N_P) obtained through neural network pruning would be a good proxy model of N_B. In order to verify our idea, we performed extensive experiments by using CIFAR10, CFIAR100, and TinyImageNet datasets and three generally-used neural networks and three representative hyperparameter optmization methods. Through these experiments, we verified that N_P can be a good proxy model of N_B for rapid hyperparameter optimization. The proposed hyperparameter optimization framework can reduce the amount of time up to 37%.
It Isn't Sh!tposting, It's My CAT Posting
Venue: arXiv cs.CV 2022
Year: 2022

In this paper, we describe a novel architecture which can generate hilarious captions for a given input image. The architecture is split into two halves, i.e. image captioning and hilarious text conversion. The architecture starts with a pre-trained CNN model, VGG16 in this implementation, and applies attention LSTM on it to generate normal caption. These normal captions then are fed forward to our hilarious text conversion transformer which converts this text into something hilarious while maintaining the context of the input image. The architecture can also be split into two halves and only the seq2seq transformer can be used to generate hilarious caption by inputting a sentence.This paper aims to help everyday user to be more lazy and hilarious at the same time by generating captions using CATNet.
Customizing ML Predictions for Online Algorithms
Venue: arXiv cs.LG 2022
Year: 2022

A popular line of recent research incorporates ML advice in the design of online algorithms to improve their performance in typical instances. These papers treat the ML algorithm as a black-box, and redesign online algorithms to take advantage of ML predictions. In this paper, we ask the complementary question: can we redesign ML algorithms to provide better predictions for online algorithms? We explore this question in the context of the classic rent-or-buy problem, and show that incorporating optimization benchmarks in ML loss functions leads to significantly better performance, while maintaining a worst-case adversarial result when the advice is completely wrong. We support this finding both through theoretical bounds and numerical simulations.
An approach to translating Haskell programs to Agda and reasoning about them
Venue: arXiv cs.LO 2022
Year: 2022

We are using the Agda programming language and proof assistant to formally verify the correctness of a Byzantine Fault Tolerant consensus implementation based on HotStuff / LibraBFT. The Agda implementation is a translation of our Haskell implementation based on LibraBFT. This short paper focuses on one aspect of this work. We have developed a library that enables the translated Agda implementation to closely mirror the Haskell code on which it is based. This makes it easier and more efficient to review the translation for accuracy, and to maintain the translated Agda code when the Haskell code changes, thereby reducing the risk of translation errors. We also explain how we capture the semantics of the syntactic features provided by our library, thus enabling formal reasoning about programs that use them; details of how we reason about the resulting Agda implementation will be presented in a future paper. The library that we present is independent of our particular verification project, and is available, open-source, for others to use and extend.
Probability trees and the value of a single intervention
Venue: arXiv cs.LG 2022
Year: 2022

The most fundamental problem in statistical causality is determining causal relationships from limited data. Probability trees, which combine prior causal structures with Bayesian updates, have been suggested as a possible solution. In this work, we quantify the information gain from a single intervention and show that both the anticipated information gain, prior to making an intervention, and the expected gain from an intervention have simple expressions. This results in an active-learning method that simply selects the intervention with the highest anticipated gain, which we illustrate through several examples. Our work demonstrates how probability trees, and Bayesian estimation of their parameters, offer a simple yet viable approach to fast causal induction.
Seeing Sounds, Hearing Shapes: a gamified study to evaluate sound-sketches
Venue: arXiv cs.MM 2022
Year: 2022

Sound-shape associations, a subset of cross-modal associations between the auditory and visual domain, have been studied mainly in the context of matching a set of purposefully crafted shapes to sounds. Recent studies have explored how humans represent sound through free-form sketching and how a graphical sketch input could be used for sound production. In this paper, the potential of communicating sound characteristics through these free-form sketches is investigated in a gamified study that was conducted with eighty-two participants at two online exhibition events. The results show that participants managed to recognise sounds at a higher rate than the random baseline would suggest, however it appeared difficult to visually encode nuanced timbral differences.
Financial Time Series Data Augmentation with Generative Adversarial Networks and Extended Intertemporal Return Plots
Venue: arXiv cs.CV 2022
Year: 2022

Data augmentation is a key regularization method to support the forecast and classification performance of highly parameterized models in computer vision. In the time series domain however, regularization in terms of augmentation is not equally common even though these methods have proven to mitigate effects from small sample size or non-stationarity. In this paper we apply state-of-the art image-based generative models for the task of data augmentation and introduce the extended intertemporal return plot (XIRP), a new image representation for time series. Multiple tests are conducted to assess the quality of the augmentation technique regarding its ability to synthesize time series effectively and improve forecast results on a subset of the M4 competition. We further investigate the relationship between data set characteristics and sampling results via Shapley values for feature attribution on the performance metrics and the optimal ratio of augmented data. Over all data sets, our approach proves to be effective in reducing the return forecast error by 7% on 79% of the financial data sets with varying statistical properties and frequencies.
i-MYO: A Hybrid Prosthetic Hand Control System based on Eye-tracking, Augmented Reality and Myoelectric signal
Venue: arXiv cs.RO 2022
Year: 2022

Dexterous prosthetic hands have better grasp performance than traditional ones. However, patients still find it difficult to use these hands without a suitable control system. A new hybrid myoelectric control system, termed i-MYO, is presented and evaluated to solve this problem. The core component of the i-MYO is a novel grasp-type switching interface based on eye-tracking and augmented reality (AR), termed i-GSI. With the i-GSI, the user can easily switch a grasp type (six total) for a prosthetic hand by gazing at a GazeButton. The i-GSI is implemented in an AR helmet and is integrated, as an individual module, into the i-MYO system. In the i-MYO system, the myoelectric signal was used to control hand opening /closing proportionally. The operation of the i-MYO was tested on nine healthy subjects who wore HIT-V hand on the forearm and manipulated objects in a reach-and-grasp task. It was also tested on one patient who had an inferior myoelectric signal and was required to control the HIT-V hand to grasp objects. Results showed that in 91.6% of the trials, inexperienced healthy subjects accomplished the task within 5.9 s, and most failed trials were caused by a lack of experience in fine grasping. In addition, in about 1.5% of trials, the subjects also successfully transferred the objects but with a non-optimal grasp type. In 97.0% of the trials, the subjects spent ~1.3 s switching the optimal grasp types. A higher success rate in grasp type (99.1%) for the untrained patient has been observed thanks to more trials conducted. In 98.7 % of trials, the patient only needed another 2 s to control the hand to grasp the object after switching to the optimal grasp type. The tests demonstrate the control capability of the new system in multi-DOF prosthetics, and all inexperienced subjects were able to master the operation of the i-MYO quickly within a few pieces of training and apply it easily.
Zigzag normalisation for associative $n$-categories
Venue: arXiv cs.LO 2022
Year: 2022

The theory of associative $n$-categories has recently been proposed as a strictly associative and unital approach to higher category theory. As a foundation for a proof assistant, this is potentially attractive, since it has the potential to allow simple formal proofs of complex high-dimensional algebraic phenomena. However, the theory relies on an implicit term normalisation procedure to recognize correct composites, with no recursive method available for computing it. Here we describe a new approach to term normalisation in associative $n$-categories, based on the categorical zigzag construction. This radically simplifies the theory, and yields a recursive algorithm for normalisation, which we prove is correct. Our use of categorical lifting properties allows us to give efficient proofs of our results. This normalisation algorithm forms a core component of the proof assistant homotopy.io, and we illustrate our scheme with worked examples.
Monitoring Security of Enterprise Hosts via DNS Data Analysis
Venue: arXiv cs.CR 2022
Year: 2022

Enterprise Networks are growing in scale and complexity, with heterogeneous connected assets needing to be secured in different ways. Nevertheless, virtually all connected assets use the Domain Name System (DNS) for address resolution, and DNS has thus become a convenient vehicle for attackers to covertly perform Command and Control (C&C) communication, data theft, and service disruption across a wide range of assets. Enterprise security appliances that monitor network traffic typically allow all DNS traffic through as it is vital for accessing any web service; they may at best match against a database of known malicious patterns, and are therefore ineffective against zero-day attacks. This thesis focuses on three high-impact cyber-attacks that leverage DNS, specifically data exfiltration, malware C&C communication, and service disruption. Using big data (over 10B packets) of DNS network traffic collected from a University campus and a Government research organization over a 6-month period, we illustrate the anatomy of these attacks, train machines for automatically detecting such attacks, and evaluate their efficacy in the field.
Personalized Behaviour Models: A Survey Focusing on Autism Therapy Applications
Venue: arXiv cs.RO 2022
Year: 2022

Children with Autism Spectrum Disorder find robots easier to communicate with than humans. Thus, robots have been introduced in autism therapies. However, due to the environmental complexity, the used robots often have to be controlled manually. This is a significant drawback of such systems and it is required to make them more autonomous. In particular, the robot should interpret the child's state and continuously adapt its actions according to the behaviour of the child under therapy. This survey elaborates on different forms of personalized robot behaviour models. Various approaches from the field of Human-Robot Interaction, as well as Child-Robot Interaction, are discussed. The aim is to compare them in terms of their deficits, feasibility in real scenarios, and potential usability for autism-specific Robot-Assisted Therapy. The general challenge for algorithms based on which the robot learns proper interaction strategies during therapeutic games is to increase the robot's autonomy, thereby providing a basis for a robot's decision-making.
Fast Neural Network based Solving of Partial Differential Equations
Venue: arXiv cs.LG 2022
Year: 2022

We present a novel method for using Neural Networks (NNs) for finding solutions to a class of Partial Differential Equations (PDEs). Our method builds on recent advances in Neural Radiance Field research (NeRFs) and allows for a NN to converge to a PDE solution much faster than classic Physically Informed Neural Network (PINNs) approaches.
Sparse superposition codes with rotational invariant coding matrices for memoryless channels
Venue: arXiv cs.IT 2022
Year: 2022

We recently showed in [1] the superiority of certain structured coding matrices ensembles (such as partial row-orthogonal) for sparse superposition codes when compared with purely random matrices with i.i.d. entries, both information-theoretically and under practical vector approximate message-passing decoding. Here we generalize this result to binary input channels under generalized vector approximate message-passing decoding [2].We focus on specific binary output channels for concreteness but our analysis based on the replica symmetric method from statistical physics applies to any memoryless channel. We confirm that the "spectral criterion" introduced in [1], a coding-matrix design principle which allows the code to be capacity-achieving in the "large section size" asymptotic limit, extends to generic memoryless channels. Moreover, we also show that the vanishing error floor property [3] of this coding scheme is universal for arbitrary spectrum of the coding matrix.
Toward Timed-Release Encryption in Web3 An Efficient Dual-Purpose Proof-of-Work Consensus
Venue: arXiv cs.CR 2022
Year: 2022

Many existing timed-release encryption schemes uses time-lock puzzles to avoid relying on a trusted timeserver or a key holder which could be a weak spot in data security. However, it is unavoidable to consume massive computing power for solving time-lock puzzles and it is difficult for encryptors to predict the amount of time to solve a puzzle by decryptors. In this study, an efficient dual-purpose proof-of-work consensus allows users to release a time-locked content, which is encrypted by an asymmetric key encryption scheme on a blockchain, without trust in any third-party agents. The release time is predictable as the block time in a proof-of-work blockchain is adaptively controlled. The mining work is reproposed so that once a new block was mined on the blockchain network, time-lock puzzles were also solved immediately. No additional work is required to reveal the time-locked contents and the encryption is secured by monetary incentive mechanisms since it would be very costly to arrange an attack attempt, which must overtake the total hash rate of the whole blockchain network.
VLC Physical Layer Security through RIS-aided Jamming Receiver for 6G Wireless Networks
Venue: arXiv cs.CR 2022
Year: 2022

Visible Light Communication (VLC) is one the most promising enabling technology for future 6G networks to overcome Radio-Frequency (RF)-based communication limitations thanks to a broader bandwidth, higher data rate, and greater efficiency. However, from the security perspective, VLCs suffer from all known wireless communication security threats (e.g., eavesdropping and integrity attacks). For this reason, security researchers are proposing innovative Physical Layer Security (PLS) solutions to protect such communication. Among the different solutions, the novel Reflective Intelligent Surface (RIS) technology coupled with VLCs has been successfully demonstrated in recent work to improve the VLC communication capacity. However, to date, the literature still lacks analysis and solutions to show the PLS capability of RIS-based VLC communication. In this paper, we combine watermarking and jamming primitives through the Watermark Blind Physical Layer Security (WBPLSec) algorithm to secure VLC communication at the physical layer. Our solution leverages RIS technology to improve the security properties of the communication. By using an optimization framework, we can calculate RIS phases to maximize the WBPLSec jamming interference schema over a predefined area in the room. In particular, compared to a scenario without RIS, our solution improves the performance in terms of secrecy capacity without any assumption about the adversary's location. We validate through numerical evaluations the positive impact of RIS-aided solution to increase the secrecy capacity of the legitimate jamming receiver in a VLC indoor scenario. Our results show that the introduction of RIS technology extends the area where secure communication occurs and that by increasing the number of RIS elements the outage probability decreases.
Exploring Children's Preferences for Taking Care of a Social Robot
Venue: arXiv cs.RO 2022
Year: 2022

Research in child-robot interactions suggests that engaging in "care-taking" of a social robot, such as tucking the robot in at night, can strengthen relationships formed between children and robots. In this work, we aim to better understand and explore the design space of caretaking activities with 10 children, aged 8--12 from eight families, involving an exploratory design session followed by a preliminary feasibility testing of robot caretaking activities. The design sessions provided insight into children's current caretaking tasks, how they would take care of a social robot, and how these new caretaking activities could be integrated into their daily routines. The feasibility study tested two different types of robot caretaking tasks, which we call connection and utility, and measured their short term effects on children's perceptions of and closeness to the social robot. We discuss the themes and present interaction design guidelines of robot caretaking activities for children.