Miracle-dz/UniFAR-FTUs huggingface dataset:
KRADA: Known-region-aware Domain Alignment for Open World Semantic Segmentation

In semantic segmentation, we aim to train a pixel-level classifier to assign category labels to all pixels in an image, where labeled training images and unlabeled test images are from the same distribution and share the same label set. However, in an open world, the unlabeled test images probably contain unknown categories and have different distributions from the labeled images. Hence, in this paper, we consider a new, more realistic, and more challenging problem setting where the pixel-level classifier has to be trained with labeled images and unlabeled open-world images -- we name it open world semantic segmentation (OSS). In OSS, the trained classifier is expected to identify unknown-class pixels and classify known-class pixels well. To solve OSS, we first investigate which distribution that unknown-class pixels obey. Then, motivated by the goodness-of-fit test, we use statistical measurements to show how a pixel fits the distribution of an unknown class and select highly-fitted pixels to form the unknown region in each image. Eventually, we propose an end-to-end learning framework, known-region-aware domain alignment (KRADA), to distinguish unknown classes while aligning distributions of known classes in labeled and unlabeled open-world images. The effectiveness of KRADA has been verified on two synthetic tasks and one COVID-19 segmentation task.
DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learning

Games are abstractions of the real world, where artificial agents learn to compete and cooperate with other agents. While significant achievements have been made in various perfect- and imperfect-information games, DouDizhu (a.k.a. Fighting the Landlord), a three-player card game, is still unsolved. DouDizhu is a very challenging domain with competition, collaboration, imperfect information, large state space, and particularly a massive set of possible actions where the legal actions vary significantly from turn to turn. Unfortunately, modern reinforcement learning algorithms mainly focus on simple and small action spaces, and not surprisingly, are shown not to make satisfactory progress in DouDizhu. In this work, we propose a conceptually simple yet effective DouDizhu AI system, namely DouZero, which enhances traditional Monte-Carlo methods with deep neural networks, action encoding, and parallel actors. Starting from scratch in a single server with four GPUs, DouZero outperformed all the existing DouDizhu AI programs in days of training and was ranked the first in the Botzone leaderboard among 344 AI agents. Through building DouZero, we show that classic Monte-Carlo methods can be made to deliver strong results in a hard domain with a complex action space. The code and an online demo are released at this https URL with the hope that this insight could motivate future work.
Assessing the Effectiveness of Syntactic Structure to Learn Code Edit Representations

In recent times, it has been shown that one can use code as data to aid various applications such as automatic commit message generation, automatic generation of pull request descriptions and automatic program repair. Take for instance the problem of commit message generation. Treating source code as a sequence of tokens, state of the art techniques generate commit messages using neural machine translation models. However, they tend to ignore the syntactic structure of programming languages. Previous work, i.e., code2seq has used structural information from Abstract Syntax Tree (AST) to represent source code and they use it to automatically generate method names. In this paper, we elaborate upon this state of the art approach and modify it to represent source code edits. We determine the effect of using such syntactic structure for the problem of classifying code edits. Inspired by the code2seq approach, we evaluate how using structural information from AST, i.e., paths between AST leaf nodes can help with the task of code edit classification on two datasets of fine-grained syntactic edits. Our experiments shows that attempts of adding syntactic structure does not result in any improvements over less sophisticated methods. The results suggest that techniques such as code2seq, while promising, have a long way to go before they can be generically applied to learning code edit representations. We hope that these results will benefit other researchers and inspire them to work further on this problem.
Matching Patterns with Variables under Hamming Distance

A pattern $\alpha$ is a string of variables and terminal letters. We say that $\alpha$ matches a word $w$, consisting only of terminal letters, if $w$ can be obtained by replacing the variables of $\alpha$ by terminal words. The matching problem, i.e., deciding whether a given pattern matches a given word, was heavily investigated: it is NP-complete in general, but can be solved efficiently for classes of patterns with restricted structure. In this paper, we approach this problem in a generalized setting, by considering approximate pattern matching under Hamming distance. More precisely, we are interested in what is the minimum Hamming distance between $w$ and any word $u$ obtained by replacing the variables of $\alpha$ by terminal words. Firstly, we address the class of regular patterns (in which no variable occurs twice) and propose efficient algorithms for this problem, as well as matching conditional lower bounds. We show that the problem can still be solved efficiently if we allow repeated variables, but restrict the way the different variables can be interleaved according to a locality parameter. However, as soon as we allow a variable to occur more than once and its occurrences can be interleaved arbitrarily with those of other variables, even if none of them occurs more than once, the problem becomes intractable.
Small-space and streaming pattern matching with $k$ edits

In this work, we revisit the fundamental and well-studied problem of approximate pattern matching under edit distance. Given an integer <tex>$k$</tex>, a pattern <tex>$P$</tex> of length <tex>$m$</tex>, and a text <tex>$T$</tex> of length <tex>$n\geq m$</tex>, the task is to find substrings of <tex>$T$</tex> that are within edit distance <tex>$k$</tex> from <tex>$P$</tex>. Our main result is a streaming algorithm that solves the problem in <tex>$\tilde{\mathcal{O}}(k^{5})$</tex> space<sup>1</sup><sup>1</sup>Hereafter, <tex>$\tilde{\mathcal{O}}(\cdot)$</tex> hides a <tex>$\text{poly} (\log n)$</tex> factor. and <tex>$\tilde{\mathcal{O}}(k^{8})$</tex> amortized time per character of the text, providing answers correct with high probability. This answers a decade-old question: since the discovery of a poly (<tex>$k\ \text{log}\ n$</tex>) -space streaming algorithm for pattern matching under Hamming distance by Porat and Porat [FOCS 2009], the existence of an analogous result for edit distance remained open. Up to this work, no poly (<tex>$k\ \text{log}\ n$</tex>)-space algorithm was known even in the simpler semi-streaming model, where <tex>$T$</tex> comes as a stream but <tex>$P$</tex> is available for read-only access. In this model, we give a deterministic algorithm that achieves slightly better complexity. Our central technical contribution is a new space-efficient deterministic encoding of two strings, called the greedy encoding, which encodes a set of all alignments of cost at most <tex>$k$</tex> with a certain property (we call such alignments greedy). On strings of length at most <tex>$n$</tex>, the encoding occupies <tex>$\tilde{\mathcal{O}}(k^{2})$</tex> space. We use the encoding to compress substrings of the text that are close to the pattern. In order to do so, we compute the encoding for substrings of the text and of the pattern, which requires read-only access to the latter. In order to develop the fully streaming algorithm, we further introduce a new edit distance sketch parameterized by integers <tex>$n > k$</tex>. For any string of length at most <tex>$n$</tex>, the sketch is of size <tex>$\tilde{\mathcal{O}}\overline{(k}^{2})$</tex>, and it can be computed with an <tex>$\tilde{\mathcal{O}}(k^{2})$</tex>-space streaming algorithm. Given the sketches of two strings, in <tex>$\tilde{\mathcal{O}}(k^{3})$</tex> time we can compute their edit distance or certify that it is larger than <tex>$k$</tex>. This result improves upon <tex>$\tilde{\mathcal{O}}(k^{8})$</tex>-size sketches of Belazzougui and Zhang [FOCS 2016] and very recent <tex>$\tilde{\mathcal{O}}(k^{3})$</tex>-size sketches of Jin, Nelson, and Wu [STACS 2021].
The Effect of the Axial Velocity Density Ratio on the Aerodynamic Coefficients of Compressor Cascades

The aerodynamic coefficients of compressor blade sections in two-dimensional flow can easily and very accurately be determined by use of the well-known Lieblein correlations. Very often the flow across the compressor blade sections is quasi-two-dimensional with the axial velocity density ratio (AVDR) differing from unity. To establish simple correlations for this type of flow as well, the AVDR effect on the aerodynamic coefficients of compressor cascades is theoretically and experimentally investigated. This results in simple but accurate formulas for the calculation of the AVDR effect on the turning angle, the reference minimum-loss inlet angle, and the losses in terms of the wake momentum thickness and the diffusion ratio.Copyright © 1980 by ASME
Non-orthogonal Multiple Access for Multi-cell Indoor VLC

In this letter, we propose a non-orthogonal multiple access (NOMA) based scheme for multi-cell indoor visible light communications (VLC), where cell-edge user is jointly served by multiple cells. Unlike the typical designs in the existing literature, the proposed scheme doesn't involve complex domain and direct current bias addition, which makes the design simple and feasible for real time implementation in indoor VLC. The symbol error rate (SER) of the proposed NOMA scheme is analysed using analytical and simulation results, and is compared with orthogonal multiple access (OMA). It is observed that the average SER of the users with the proposed NOMA is marginally degraded as compared to OMA. However, the SER of cell-edge user with the proposed NOMA is significantly improved with joint maximum likelihood decoding, and outperforms successive interference cancellation based decoding and OMA, with trade-off on computational complexity.
Statistical modeling of on-street parking lot occupancy in smart cities

Many studies suggest that searching for parking is associated with significant direct and indirect costs. Therefore, it is appealing to reduce the time which car drivers spend on finding an available parking lot, especially in urban areas where the space for all road users is limited. The prediction of on-street parking lot occupancy can provide drivers a guidance where clear parking lots are likely to be found. This field of research has gained more and more attention in the last decade through the increasing availability of real-time parking lot occupancy data. In this paper, we pursue a statistical approach for the prediction of parking lot occupancy, where we make use of time to event models and semi-Markov process theory. The latter involves the employment of Laplace transformations as well as their inversion which is an ambitious numerical task. We apply our methodology to data from the City of Melbourne in Australia. Our main result is that the semi-Markov model outperforms a Markov model in terms of both true negative rate and true positive rate while this is essentially achieved by respecting the current duration which a parking lot already sojourns in its initial state.
Interpreting Expert Annotation Differences in Animal Behavior

Hand-annotated data can vary due to factors such as subjective differences, intra-rater variability, and differing annotator expertise. We study annotations from different experts who labelled the same behavior classes on a set of animal behavior videos, and observe a variation in annotation styles. We propose a new method using program synthesis to help interpret annotation differences for behavior analysis. Our model selects relevant trajectory features and learns a temporal filter as part of a program, which corresponds to estimated importance an annotator places on that feature at each timestamp. Our experiments on a dataset from behavioral neuroscience demonstrate that compared to baseline approaches, our method is more accurate at capturing annotator labels and learns interpretable temporal filters. We believe that our method can lead to greater reproducibility of behavior annotations used in scientific studies. We plan to release our code.
BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data

Maintaining a consistent persona is essential for dialogue agents. Although tremendous advancements have been brought, the limited-scale of annotated personalized dialogue datasets is still a barrier towards training robust and consistent persona-based dialogue models. This work shows how this challenge can be addressed by disentangling persona-based dialogue generation into two sub-tasks with a novel BERT-over-BERT (BoB) model. Specifically, the model consists of a BERT-based encoder and two BERT-based decoders, where one decoder is for response generation, and another is for consistency understanding. In particular, to learn the ability of consistency understanding from large-scale non-dialogue inference data, we train the second decoder in an unlikelihood manner. Under different limited data settings, both automatic and human evaluations demonstrate that the proposed model outperforms strong baselines in response quality and persona consistency.
One Sense per Translation

Word sense disambiguation (WSD) is the task of determining the sense of a word in context. Translations have been used in WSD as a source of knowledge, and even as a means of delimiting word senses. In this paper, we define three theoretical properties of the relationship between senses and translations, and argue that they constitute necessary conditions for using translations as sense inventories. The key property of One Sense per Translation (OSPT) provides a foundation for a translation-based WSD method. The results of an intrinsic evaluation experiment indicate that our method achieves a precision of approximately 93% compared to manual corpus annotations. Our extrinsic evaluation experiments demonstrate WSD improvements of up to 4.6% F1-score on difficult WSD datasets.
AugNet: End-to-End Unsupervised Visual Representation Learning with Image Augmentation

Most of the achievements in artificial intelligence so far were accomplished by supervised learning which requires numerous annotated training data and thus costs innumerable manpower for labeling. Unsupervised learning is one of the effective solutions to overcome such difficulties. In our work, we propose AugNet, a new deep learning training paradigm to learn image features from a collection of unlabeled pictures. We develop a method to construct the similarities between pictures as distance metrics in the embedding space by leveraging the inter-correlation between augmented versions of samples. Our experiments demonstrate that the method is able to represent the image in low dimensional space and performs competitively in downstream tasks such as image classification and image similarity comparison. Specifically, we achieved over 60% and 27% accuracy on the STL10 and CIFAR100 datasets with unsupervised clustering, respectively. Moreover, unlike many deep-learning-based image retrieval algorithms, our approach does not require access to external annotated datasets to train the feature extractor, but still shows comparable or even better feature representation ability and easy-to-use characteristics. In our evaluations, the method outperforms all the state-of-the-art image retrieval algorithms on some out-of-domain image datasets. The code for the model implementation is available at https://github.com/chenmingxiang110/AugNet.
Numerical Methods for Mean Field Games and Mean Field Type Control

Mean Field Games (MFG) have been introduced to tackle games with a large number of competing players. Considering the limit when the number of players is infinite, Nash equilibria are studied by considering the interaction of a typical player with the population's distribution. The situation in which the players cooperate corresponds to Mean Field Control (MFC) problems, which can also be viewed as optimal control problems driven by a McKean-Vlasov dynamics. These two types of problems have found a wide range of potential applications, for which numerical methods play a key role since most models do not have analytical solutions. In these notes, we review several aspects of numerical methods for MFG and MFC. We start by presenting some heuristics in a basic linear-quadratic setting. We then discuss numerical schemes for forward-backward systems of partial differential equations (PDEs), optimization techniques for variational problems driven by a Kolmogorov-Fokker-Planck PDE, an approach based on a monotone operator viewpoint, and stochastic methods relying on machine learning tools.
Optimal Model Selection in Contextual Bandits with Many Classes via Offline Oracles

We study the problem of model selection for contextual bandits, in which the al-gorithm must balance the bias-variance trade-off for model estimation while also balancing the exploration-exploitationtrade-off. In this paper, we propose the ﬁrst reduction of model selection in contextual bandits to ofﬂine model selection oracles, allowing for ﬂexible general purpose algorithms with computational requirements no worse than those for model selection for regression. Our main result is a new model selection guarantee for stochastic contextual bandits. When one of the classes in our set is realizable, up to a logarithmic dependency on the number of classes, our algorithm attains optimal realizability-based regret bounds for that class under one of two conditions: if the time-horizon is large enough, or if an assumption that helps with detecting misspeciﬁcation holds. Hence our algorithm adapts to the complexity of this unknown class. Even when this realizable class is known, we prove improved regret guarantees in early rounds by relying on simpler model classes for those rounds and hence further establish the importance of model selection in contextual bandits.
Census of Variable Stars toward Serpens Main

We monitored a 3 deg2 area toward Serpens Main in the Pan-STARRS1 r, i, and z bands from 2016 April to September. Light curves of more than 11,000 stars in each band were obtained, and 143 variables have been identified. Among those, 119 variables are new discoveries, while 24 variables were previously known. We present variability classes and periods of 99 stars. Of these, 81 are located in the upper giant branch, displaying long periods, while the remaining 18 variables are pre-main-sequence objects with short periods. We also identify eight eclipsing binary systems, including the known binary V0623 Ser, and derive their physical parameters. According to a clustering analysis of Gaia DR2 stars in the observed field, there are 10 variable members in Serpens Main, where six members have been classified as young stellar objects in previous studies. Here we provide color–magnitude and color–color diagrams for these variables. The color variability of most variables in the color–magnitude diagrams produces the expected displacements, while the movements of cluster members point in different directions; this behavior may be associated with accretion spots or circumstellar disks.
Locally Sparse Networks for Interpretable Predictions

Despite the enormous success of neural networks, they are still hard to interpret and often overﬁt when applied to low-sample-size (LSS) datasets. To tackle these obstacles, we propose a framework for training locally sparse neural networks where the local sparsity is learned via a sample-speciﬁc gating mechanism that identiﬁes the subset of most relevant features for each measurement. The sample-speciﬁc sparsity is predicted via a gating network, which is trained in tandem with the prediction network. By learning these subsets and weights of a prediction model, we obtain an interpretable neural network that can handle LSS data and can remove nuisance variables, which are irrelevant for the supervised learning task. Using both synthetic and real-world datasets, we demonstrate that our method outperforms state-of-the-art models when predicting the target function with far fewer features per instance.
Nonlinear evolution in the re-summed next-to-leading order of perturbative QCD: Confronting the experimental data

Carlos Contreras,1, ∗ Eugene Levin,1, 2, 3, † and Michael Sanhueza1, ‡ Departamento de Física, Universidad Técnica Federico Santa María, Avda. España 1680, Casilla 110-V, Valparaíso, Chile Centro CientíficoTecnológico de Valparaíso, Avda. España 1680, Casilla 110-V, Valparaíso, Chile Department of Particle Physics, School of Physics and Astronomy, Raymond and Beverly Sackler Faculty of Exact Science, Tel Aviv University, Tel Aviv, 69978, Israel (Dated: June 14, 2021)
Automated Configuration of Genetic Algorithms by Tuning for Anytime Performance

Finding the best configuration of algorithms’ hyperparameters for a given optimization problem is an important task in evolutionary computation. We compare in this work the results of four different hyperparameter optimization (HPO) approaches for a family of genetic algorithms (GAs) on 25 diverse pseudo-Boolean optimization (PBO) problems. More precisely, we compare previously obtained results from a grid search with those obtained from three automated configuration techniques: 1) iterated racing; 2) mixed-integer parallel-efficient global optimization (MIP-EGO); and 3) mixed-integer evolutionary strategies. Using two different cost metrics: 1) expected running time (ERT) and 2) the area under the empirical cumulative distribution function (ECDF) curve, we find that in several cases the best configurations with respect to ERT are obtained when using the area under the ECDF curve as the cost metric during the configuration process. Our results suggest that even when interested in ERT performance, it might be preferable to use anytime performance measures for the configuration task. We also observe that tuning for ERT is much more sensitive with respect to the budget that is allocated to the target algorithms.
Nested and Balanced Entity Recognition using Multi-Task Learning

Entity Recognition (ER) within a text is a fundamental exercise in Natural Language Processing, enabling further depending tasks such as Knowledge Extraction, Text Summarisation, or Keyphrase Extraction. An entity consists of single words or of a consecutive sequence of terms, constituting the basic building blocks for communication. Mainstream ER approaches are mainly limited to flat structures, concentrating on the outermost entities while ignoring the inner ones. This paper introduces a partly-layered network architecture that deals with the complexity of overlapping and nested cases. The proposed architecture consists of two parts: (1) a shared Sequence Layer and (2) a stacked component with multiple Tagging Layers. The adoption of such an architecture has the advantage of preventing overfit to a specific word-length, thus maintaining performance for longer entities despite their lower frequency. To verify the proposed architecture's effectiveness, we train and evaluate this architecture to recognise two kinds of entities - Concepts (CR) and Named Entities (NER). Our approach achieves state-of-the-art NER performances, while it outperforms previous CR approaches. Considering these promising results, we see the possibility to evolve the architecture for other cases such as the extraction of events or the detection of argumentative components.