Reviewerly/exHarmony huggingface dataset:
Robust Optimization as Data Augmentation for Large-scale Graphs

Data augmentation helps neural networks generalize better by enlarging the training set, but it remains an open question how to effectively augment graph data to enhance the performance of GNNs (Graph Neural Networks). While most existing graph regularizers focus on manipulating graph topological structures by adding/removing edges, we offer a method to augment node features for better performance. We propose FLAG (Free Large-scale Adversarial Augmentation on Graphs), which iteratively augments node features with gradient-based adversarial perturbations during training. By making the model invariant to small fluctuations in input data, our method helps models generalize to out-of-distribution samples and boosts model performance at test time. FLAG is a general-purpose approach for graph data, which universally works in node classification, link prediction, and graph classification tasks. FLAG is also highly flexible and scalable, and is deployable with arbitrary GNN backbones and large-scale datasets. We demon-strate the efficacy and stability of our method through ex-tensive experiments and ablation studies. We also provide intuitive observations for a deeper understanding of our method. We open source our implementation at https://github.com/devnkong/FLAG.
Cross-Modal Transferable Adversarial Attacks from Images to Videos

Recent studies have shown that adversarial examples handcrafted on one white-box model can be used to at-tack other black-box models. Such cross-model transferability makes it feasible to perform black-box attacks, which has raised security concerns for real-world DNNs applications. Nevertheless, existing works mostly focus on investigating the adversarial transferability across different deep models that share the same modality of input data. The cross-modal transferability of adversarial perturbation has never been explored. This paper investigates the transferability of adversarial perturbation across different modalities, i.e., leveraging adversarial perturbation generated on white-box image models to attack black-box video models. Specifically, motivated by the observation that the low-level feature space between images and video frames are similar, we propose a simple yet effective cross-modal attack method, named as Image To Video (I2V) attack. I2V generates adversarial frames by minimizing the cosine similarity between features of pretrained image models from adversarial and benign examples, then combines the generated adversarial frames to perform black-box attacks on video recognition models. Extensive experiments demonstrate that I2V can achieve high attack success rates on different black-box video recognition models. On Kinetics-400 and UCF-101, I2V achieves an average attack success rate of 77.88% and 65.68%, respectively, which sheds light on the feasibility of cross-modal adversarial attacks.
Auction-Based Cluster Federated Learning in Mobile Edge Computing Systems

Federated Learning (FL), allowing data owners to conduct model training without sending their raw data to third-party servers, can enhance data privacy in Mobile Edge Computing (MEC) which brings data processing closer to the data sources. However, the heterogeneity of local data and constrained local resources in MEC bring new challenges hindering the development of FL. To this end, we propose an Auction-based Cluster Federated Learning scheme, called ACFL, comprising a clustered FL framework and an auction-based client selection strategy. Our clustered FL framework first introduces a mean-shift clustering algorithm to FL, which can intelligently cluster clients according to their local data distribution. Then, we select clients from each cluster using an auction mechanism to participate in FL training, which can mitigate the impact of data heterogeneity on model convergence and balance energy consumption. Moreover, we prove the proposed clustered FL framework converges at a sublinear rate. Extensive experiments conducted on real-world datasets demonstrate that the proposed FL scheme outperforms the conventional FL schemes in terms of convergence rate and energy balance.
Automatic generation of pathological benchmark dataset from hyperspectral images of double stained tissues

Artificial intelligence has been widely used for digital pathology diagnosis. However, the AI performance highly relies on the high-quality annotated datasets, pathological images need to be labeled by experienced pathologists manually, which is time consuming, laborious and expensive. In addition, small lesion areas are usually missed by human eyes, directly influencing the performance of those identification models trained by the data. This paper presents a new strategy for generating annotated pathological benchmark dataset from microscopic hyperspectral images of HE-CAM5.2 stained tissues. We design a Spatial-Spectral based Hyperspectral GAN (SSHGAN), which transforms hyperspectral images into standard histological images using networks trained by the cycle consistent adversarial model. Gradient boosting decision tree integrated with graph-cut method is used to automatically generate the annotations by adding the spectral prior. The proposed strategy can obtain both the standard H&E images and the corresponding annotation files simultaneously using spatial and spectral information of hyperspectral images. The methods have been tested on gastric cancer, lung adenocarcinoma, intrahepatic cholangiocarcinoma, and colorectal cancer tissues and evaluated by segmentation networks and experienced pathologists. Experimental results show that the proposed methods have desirable performance on small tumor targets and discrete regions, which is promising in automatically generating completely annotation pathology benchmark datasets.
SVFormer: Semi-supervised Video Transformer for Action Recognition

Semi-supervised action recognition is a challenging but critical task due to the high cost of video annotations. Existing approaches mainly use convolutional neural networks, yet current revolutionary vision transformer models have been less explored. In this paper, we investigate the use of transformer models under the SSL setting for action recognition. To this end, we introduce SVFormer, which adopts a steady pseudo-labeling framework (i.e., EMA-Teacher) to cope with unlabeled video samples. While a wide range of data augmentations have been shown effective for semi-supervised image classification, they generally produce limited results for video recognition. We therefore introduce a novel augmentation strategy, Tube Token-Mix, tailored for video data where video clips are mixed via a mask with consistent masked tokens over the temporal axis. In addition, we propose a temporal warping augmentation to cover the complex temporal variation in videos, which stretches selected frames to various temporal durations in the clip. Extensive experiments on three datasets Kinetics-400, UCF-101, and HMDB-51 verify the advantage of SVFormer. In particular, SVFormer outperforms the state-of-the-art by 31.5% with fewer training epochs under the 1% labeling rate of Kinetics-400. Our method can hopefully serve as a strong benchmark and encourage future search on semi-supervised action recognition with Transformer networks. Code is released at https://github.com/ChenHsing/SVFormer.
MSMDFusion: Fusing LiDAR and Camera at Multiple Scales with Multi-Depth Seeds for 3D Object Detection

Fusing LiDAR and camera information is essential for accurate and reliable 3D object detection in autonomous driving systems. This is challenging due to the difficulty of combining multi-granularity geometric and semantic features from two drastically different modalities. Recent approaches aim at exploring the semantic densities of camera features through lifting points in 2D camera images (referred to as “seeds”) into 3D space, and then incorporate 2D semantics via cross-modal interaction or fusion techniques. However, depth information is under-investigated in these approaches when lifting points into 3D space, thus 2D semantics can not be reliably fused with 3D points. Moreover, their multi-modal fusion strategy, which is implemented as concatenation or attention, either can not effectively fuse 2D and 3D information or is unable to perform fine-grained interactions in the voxel space. To this end, we propose a novel framework with better utilization of the depth information and fine-grained cross-modal interaction between LiDAR and camera, which consists of two important components. First, a Multi-Depth Unprojection (MDU) method is used to enhance the depth quality of the lifted points at each interaction level. Second, a Gated Modality-Aware Convolution (GMA-Conv) block is applied to modulate voxels involved with the camera modality in a fine-grained manner and then aggregate multi-modal features into a unified space. Together they provide the detection head with more comprehensive features from LiDAR and camera. On the nuScenes test benchmark, our proposed method, abbreviated as MSMD-Fusion, achieves state-of-the-art results on both 3D object detection and tracking tasks without using test-time-augmentation and ensemble techniques. The code is available at https://github.com/SxJyJay/MSMDFusion.
Goal-Directed Hierarchical Dynamic Scripting for RTS Games

Learning how to defeat human players is a challenging task in today's commercial computer games. This paper suggests a goal-directed hierarchical dynamic scripting approach for incorporating learning into real-time strategy games. Two alternatives for shortening the re-adaptation time when using dynamic scripting are also presented. Finally, this paper presents an effective way of throttling the performance of the adaptive artificial intelligence system. Put together, the approach entails the possibility of an artificial intelligence opponent to be challenging for a human player, but not too challenging.
From Synthetic Characters to Virtual Actors

We discuss the extension of an emotionally-driven agent architecture already applied to the creation of emergent narratives. Synthetic characters are enhanced to perform as actors by carrying out a second cognitive appraisal, based on the OCC model, of the emotional impact of their projected actions before execution.
Automatically Generating Summary Visualizations from Game Logs

In this paper we describe a system called ViGLS (Visualization of Game Log Summaries) that generates summaries of gameplay sessions from game logs. ViGLS automatically produces visualization of the summarized actions that are extracted based on cognitive models of summarization. ViGLS is implemented using a service-oriented architecture, de-coupling the summarization methods from any particular game engine being used. The camera code libraries used in visualization are based on constraint based camera control approaches and, in our implementation, make use of the scripting layer of the Unreal game engine.
Planning for Concurrent Action Executions Under Action Duration Uncertainty Using Dynamically Generated Bayesian Networks

An interesting class of planning domains, including planning for daily activities of Mars rovers, involves achievement of goals with time constraints and concurrent actions with probabilistic durations. Current probabilistic approaches, which rely on a discrete time model, introduce a blow up in the search state-space when the two factors of action concurrency and action duration uncertainty are combined. Simulation-based and sampling probabilistic planning approaches would cope with this state explosion by avoiding storing all the explored states in memory, but they remain approximate solution approaches. In this paper, we present an alternative approach relying on a continuous time model which avoids the state explosion caused by time stamping in the presence of action concurrency and action duration uncertainty. Time is represented as a continuous random variable. The dependency between state time variables is conveyed by a Bayesian network, which is dynamically generated by a state-based forward-chaining search based on the action descriptions. A generated plan is characterized by a probability of satisfying a goal. The evaluation of this probability is done by making a query the Bayesian network.
On the Application of Data Mining to Official Data

Retrieving valuable knowledge and statistical patterns from official data has a great potential in supporting strategic policy making.Data Mining (DM) techniques are well-known for providing flexible and efficient analytical tools for data processing.In this paper, we provide an introduction to applications of DM to official statistics and flag the important issues and challenges.Considering recent advancements in software projects for DM, we propose intelligent data control system design and specifications as an example of DM application in official data processing.
Coming Up With Good Excuses: What to do When no Plan Can be Found

When using a planner-based agent architecture, many things can go wrong. First and foremost, an agent might fail to execute one of the planned actions for some reasons. Even more annoying, however, is a situation where the agent is incompetent, i.e., unable to come up with a plan. This might be due to the fact that there are principal reasons that prohibit a successful plan or simply because the task's description is incomplete or incorrect. In either case, an explanation for such a failure would be very helpful. We will address this problem and provide a formalization of coming up with excuses for not being able to find a plan. Based on that, we will present an algorithm that is able to find excuses and demonstrate that such excuses can be found in practical settings in reasonable time.
Optimizing Player Experience in Interactive Narrative Planning: A Modular Reinforcement Learning Approach

Recent years have witnessed growing interest in data-driven approaches to interactive narrative planning and drama management. Reinforcement learning techniques show particular promise because they can automatically induce and refine models for tailoring game events by optimizing reward functions that explicitly encode interactive narrative experiences’ quality. Due to the inherently subjective nature of interactive narrative experience, designing effective reward functions is challenging. In this paper, we investigate the impacts of alternate formulations of reward in a reinforcement learning-based interactive narrative planner for the Crystal Island game environment. We formalize interactive narrative planning as a modular reinforcement-learning (MRL) problem. By decomposing interactive narrative planning into multiple independent sub-problems, MRL enables efficient induction of interactive narrative policies directly from a corpus of human players’ experience data. Empirical analyses suggest that interactive narrative policies induced with MRL are likely to yield better player outcomes than heuristic or baseline policies. Furthermore, we observe that MRL-based interactive narrative planners are robust to alternate reward discount parameterizations.
Using Sliding Windows to Generate Action Abstractions in Extensive-Form Games

In extensive-form games with a large number of actions, careful abstraction of the action space is critically important to performance. In this paper we extend previous work on action abstraction using no-limit poker games as our test domains. We show that in such games it is no longer necessary to choose, a priori, one specific range of possible bet sizes. We introduce an algorithm that adjusts the range of bet sizes considered for each bet individually in an iterative fashion. This flexibility results in a substantially improved game value in no-limit Leduc poker. When applied to no-limit Texas Hold'em our algorithm produces an action abstraction that is about one third the size of a state of the art hand-crafted action abstraction, yet has a better overall game value.
Fast Procedural Level Population with Playability Constraints

We examine the use of constraint propagation for populating indoor game levels with enemies and other objects. We introduce a notion of path constraints, which bound some function over the possible paths a player might take, and show how to efficiently place objects while guaranteeing path constraints. This allows the system to guarantee that power-ups are balanced to the number of enemies occurring in the level, that they’re placed early enough to be useful, that keys are not hidden behind the doors they are intended to unlock, and so on. We describe a constraint solver based on interval methods that allows natural processing of numeric constraints and show that it is efficient enough to be used even on very low-end platforms.
Limitations of Choice-Based Interactive Evolution for Game Level Design

This paper presents a tool geared towards the collaboration of a human and an artificial designer for the creation of game content. The framework combines procedural content generation using stochastic search with user input in the form of an initial goal statement as well as preference of generated results. Feedback from industry experts in a pilot user experiment showcased the limitations of this approach and the protocol chosen for evaluating the authoring tool. The limitations are discussed with respect to the suitability of interactive evolution for creative design and the design of experimental protocols for evaluating authoring tools for games.
Multinomial Relation Prediction in Social Data: A Dimension Reduction Approach

The recent popularization of social web services has made them one of the primary uses of the World Wide Web. An important concept in social web services is social actions such as making connections and communicating with others and adding annotations to web resources. Predicting social actions would improve many fundamental web applications, such as recommendations and web searches. One remarkable characteristic of social actions is that they involve multiple and heterogeneous objects such as users, documents, keywords, and locations. However, the high-dimensional property of such multinomial relations poses one fundamental challenge, that is, predicting multinomial relations with only a limited amount of data. In this paper, we propose a new multinomial relation prediction method, which is robust to data sparsity. We transform each instance of a multinomial relation into a set of binomial relations between the objects and the multinomial relation of the involved objects. We then apply an extension of a low-dimensional embedding technique to these binomial relations, which results in a generalized eigenvalue problem guaranteeing global optimal solutions. We also incorporate attribute information as side information to address the “cold start” problem in multinomial relation prediction. Experiments with various real-world social web service datasets demonstrate that the proposed method is more robust against data sparseness as compared to several existing methods, which can only find sub-optimal solutions.
Unsupervised learning from videos using temporal coherency deep networks

In this work we address the challenging problem of unsupervised learning from videos. Existing methods utilize the spatio-temporal continuity in contiguous video frames as regularization for the learning process. Typically, this temporal coherence of close frames is used as a free form of annotation, encouraging the learned representations to exhibit small differences between these frames. But this type of approach fails to capture the dissimilarity between videos with different content, hence learning less discriminative features. We here propose two Siamese architectures for Convolutional Neural Networks, and their corresponding novel loss functions, to learn from unlabeled videos, which jointly exploit the local temporal coherence between contiguous frames, and a global discriminative margin used to separate representations of different videos. An extensive experimental evaluation is presented, where we validate the proposed models on various tasks. First, we show how the learned features can be used to discover actions and scenes in video collections. Second, we show the benefits of such an unsupervised learning from just unlabeled videos, which can be directly used as a prior for the supervised recognition tasks of actions and objects in images, where our results further show that our features can even surpass a traditional and heavily supervised pre-training plus fine-tuning strategy.
Pipelined Neural Networks for Phrase-Level Sentiment Intensity Prediction

Linguistic modifiers such as negators (e.g., not), intensifiers (e.g., very) and modals (e.g., would) are commonly used in expressing opinions. These modifiers play an important role in recognizing the sentiment intensity of multi-word phrases because they may lead to an intensity shift and polarity reversal for the words they modify. Appropriately modeling the effect of such modifiers on the intensity shift can greatly improve the performance of phrase-level sentiment intensity prediction. To this end, this paper proposes two neural network (NN) models organized in a pipelined fashion to determine 1) the intensity of individual words and 2) the shift weights of modifiers representing the degrees of intensity change for the words they modify. The intensity of a phrase can then be determined by combining the intensity of the constituent word and the shift weight of the modifier within the phrase. When measuring the word intensity, the first NN model introduces a hidden layer as a filter to select appropriate similar seed words in the prediction process. Automatic word intensity prediction can address the unknown intensities of words not covered in sentiment lexicons. In learning the modifier weights, the second NN model considers both the weights of individual modifiers and groups of modifiers to capture various intensity shift effects caused by them. Experiments on a SemEval-2016 dataset showed that the proposed method yielded better prediction performance for both single words and multi-word phrases.
Distributed Private Randomness Distillation

We develop the resource theory of private randomness extraction in the distributed and device-dependent scenario. We begin by introducing the notion of independent random bits, which are bipartite states containing ideal private randomness for each party, and motivate the natural set of free operations. As a conceptual tool, we introduce Virtual Quantum State Merging, which is essentially the flip side of Quantum State Merging, without communication. We focus on the bipartite case and find the rate regions achievable in different settings. Surprisingly, it turns out that local noise can boost randomness extraction. As a consequence of our analysis, we resolve a long-standing problem by giving an operational interpretation for the reverse coherent information (up to a constant term $\log d$) as the number of private random bits obtained by sending quantum states from one honest party (server) to another one (client) via the eavesdropped quantum channel.
Some new bounds on LCD codes over finite fields

In this paper, we show that LCD codes are not equivalent to linear codes over small finite fields. The enumeration of binary optimal LCD codes is obtained. We also get the exact value of LD$(n,2)$ over $\mathbb{F}_3$ and $\mathbb{F}_4$. We study the bound of LCD codes over $\mathbb{F}_q$.
The Logical Essentials of Bayesian Reasoning

This chapter offers an accessible introduction to the channel-based approach to Bayesian probability theory. This framework rests on algebraic and logical foundations, inspired by the methodologies of programming language semantics. It offers a uniform, structured and expressive language for describing Bayesian phenomena in terms of familiar programming concepts, like channel, predicate transformation and state transformation. The introduction also covers inference in Bayesian networks, which will be modelled by a suitable calculus of string diagrams.