Miracle-dz/UniFAR-FTUs huggingface dataset:
Masked Autoencoders As Spatiotemporal Learners

This paper studies a conceptually simple extension of Masked Autoencoders (MAE) to spatiotemporal representation learning from videos. We randomly mask out spacetime patches in videos and learn an autoencoder to reconstruct them in pixels. Interestingly, we show that our MAE method can learn strong representations with almost no inductive bias on spacetime (only except for patch and positional embeddings), and spacetime-agnostic random masking performs the best. We observe that the optimal masking ratio is as high as 90% (vs. 75% on images), supporting the hypothesis that this ratio is related to information redundancy of the data. A high masking ratio leads to a large speedup, e.g.,>4x in wall-clock time or even more. We report competitive results on several challenging video datasets using vanilla Vision Transformers. We observe that MAE can outperform supervised pre-training by large margins. We further report encouraging results of training on real-world, uncurated Instagram data. Our study suggests that the general framework of masked autoencoding (BERT, MAE, etc.) can be a unified methodology for representation learning with minimal domain knowledge.
AdaMCT: Adaptive Mixture of CNN-Transformer for Sequential Recommendation

Sequential recommendation (SR) aims to model users' dynamic preferences from a series of interactions. A pivotal challenge in user modeling for SR lies in the inherent variability of user preferences. An effective SR model is expected to capture both the long-term and short-term preferences exhibited by users, wherein the former can offer a comprehensive understanding of stable interests that impact the latter. To more effectively capture such information, we incorporate locality inductive bias into the Transformer by amalgamating its global attention mechanism with a local convolutional filter, and adaptively ascertain the mixing importance on a personalized basis through layer-aware adaptive mixture units, termed as AdaMCT. Moreover, as users may repeatedly browse potential purchases, it is expected to consider multiple relevant items concurrently in long-/short-term preferences modeling. Given that softmax-based attention may promote unimodal activation, we propose the Squeeze-Excitation Attention (with sigmoid activation) into SR models to capture multiple pertinent items (keys) simultaneously. Extensive experiments on three widely employed benchmarks substantiate the effectiveness and efficiency of our proposed approach. Source code is available at https://github.com/juyongjiang/AdaMCT.
Clustering through pair interactions in swimming zooplankton

This work focuses on the formation of mating aggregates in zooplankton. In particular, sexual encounters are behaviourally supported by males actively swimming in search for females, and approaching them for mating once they are found. While the random search leads to a diffusive flux of individuals, the approaching for encounter supports attraction. Thus, we ask whether these competing mechanisms of diffusion and attraction can support aggregation and lead to the formation of mating clusters. To answer our question we formulate a model in which particles performing random walks can briefly make contact with other particles if they are found within a particular distance from each other. Our analysis shows that this model supports clustering in a way analogous to the process of colloid aggregation. Following that, we analyze a dataset of 3D trajectories of swimming copepods and show that the results compare well with our model. These results support the hypothesis that pair-interactions promote mating aggregates in zooplankton and are sufficient to overcome the diffusive nature of their mate searching behavior. Our results are useful for understanding small-scale clustering of zooplankton, which is crucial for predicting encounter rates and reproduction rates in the ocean.
"Vi rigger til så godt vi kan" - konturer av praksisarkitekturer som muliggjør og hindrer undervisning i materialet tre i kunst- og håndverksfaget i norsk barneskole

”We rig up as well as we can” – contours of practice architectures that enable and constrain woodworking practices in the subject of arts and crafts in Norwegian primary school. Abstract: The study presented in this article is connected to the field of art and design didactic in Norwegian primary school and the use of wood as material. Woodwork has had varied focus and status in educational curriculums but it seems as if this status has gradually faded. The purpose of this article is to sketch the contours of woodwork practices throughout three counties, Nordland, Nord-Trondelag and Sor-Trondelag using a large survey and a micro ethnographic study in eight primary schools. The empirical material of the study is analysed using the Theory of practice architectures (Kemmis & Grootenboer, 2008) and feed-back loops (Ostern, 2016) as analytical tools. The aim of the analysis is to identify the building blocks that enable and constrain woodworking practices. Keywords: wooden material, micro ethnography, education. (Published: 13 January, 2017) Citation: Pauliina Maapalo. ««Vi rigger til sa godt vi kan» konturer av praksisarkitekturer som muliggjor og hindrer undervisning i materialet tre i kunst- og handverksfaget i norsk barneskole. » Journal for Research in Arts and Sports Education, Vol. 1, 2017, pp. 1-20. http://dx.doi. org/10.23865/jased.v1. 520
Symbolic-Numeric Factorization of Differential Operators

We present a symbolic-numeric Las Vegas algorithm for factoring Fuchsian ordinary differential operators with rational function coefficients. The new algorithm combines ideas of van Hoeij's "local-to-global" method and of the "analytic" approach proposed by van der Hoeven. It essentially reduces to the former in "easy" cases where the local-to-global method succeeds, and to an optimized variant of the latter in the "hardest" cases, while handling intermediate cases more efficiently than both.
Tight Differential Privacy Guarantees for the Shuffle Model with k-Randomized Response

Most differentially private (DP) algorithms assume a central model in which a reliable third party inserts noise to queries made on datasets, or a local model where the users locally perturb their data. However, the central model is vulnerable via a single point of failure, and in the local model, the utility of the data deteriorates significantly. The recently proposed shuffle model is an intermediate framework between the central and the local paradigms where the users send their locally privatized data to a server where messages are shuffled, effacing the link between a privatized message and the corresponding user, giving a better trade-off between privacy and utility than the local model, as its privacy gets amplified without adding more noise. In this paper, we theoretically derive the strictest known bound for DP guarantee for the shuffle models with $k$-Randomized Response local randomizers. There on, we focus on the utility of the shuffle model for histogram queries. Leveraging on the matrix inversion method, which is used to approximate the original distribution from the empirical one produced by the $k$-RR mechanism, we de-noise the histogram produced by the shuffle model to evaluate the total variation distance of the resulting histogram from the true one, which we regard as the measure of utility of the privacy mechanism. We perform experiments on both synthetic and real data to compare the privacy-utility trade-off of the shuffle model with that of the central one privatized by adding the state-of-the-art Gaussian noise to each bin. Although the experimental results stay consistent with the literature that favour the central model, we see that, the difference in statistical utilities between the central and the shuffle models is very small, showing that they are almost comparable under the same level of DP.
The local converse theorem for odd special orthogonal and symplectic groups in positive characteristic

Let F be a non-archimedean local ﬁeld of characteristic diﬀerent from 2 and G be either an odd special orthogonal group SO 2 r +1 ( F ) or a symplectic group Sp 2 r ( F ). In this paper, we establish the local converse theorem for G . Namely, for given two irreducible admissible generic representations of G with the same central character, if they have the same local gamma factors twisted by irreducible supercuspidal representations of GL n ( F ) for all 1 ≤ n ≤ r with the same additive character, these representations are isomorphic. Using the theory of Cogdell, Shahidi, and Tsai on partial Bessel functions and the classiﬁcation of irreducible generic representations, we break the barrier on the rank of twists 1 ≤ n ≤ 2 r − 1 in the work of Jiang and Soudry, and extend the result of Q. Zhang, which was achieved for all supercuspidal representations in characteristic 0.
Strategizing against Learners in Bayesian Games

We study repeated two-player games where one of the players, the learner, employs a no-regret learning strategy, while the other, the optimizer, is a rational utility maximizer. We consider general Bayesian games, where the payoffs of both the optimizer and the learner could depend on the type, which is drawn from a publicly known distribution, but revealed privately to the learner. We address the following questions: (a) what is the bare minimum that the optimizer can guarantee to obtain regardless of the no-regret learning algorithm employed by the learner? (b) are there learning algorithms that cap the optimizer payoff at this minimum? (c) can these algorithms be implemented efficiently? While building this theory of optimizer-learner interactions, we define a new combinatorial notion of regret called polytope swap regret, that could be of independent interest in other settings.
Sparse superposition codes with rotational invariant coding matrices for memoryless channels

We recently showed in [1] the superiority of certain structured coding matrix ensembles (such as partial row-orthogonal) for sparse superposition codes when compared with purely random matrices with i.i. d. entries, both information-theoretically and under practical vector approximate message-passing decoding. Here we generalize this result to binary input channels under generalized vector approximate message-passing decoding [2]. We focus on specific binary output channels for concreteness but our analysis based on the replica symmetric method from statistical physics applies to any memoryless channel. We confirm that the "spectral criterion" introduced in [1], a coding-matrix design principle which allows the code to be capacity-achieving in the "large section size" asymptotic limit, extends to generic memoryless channels. Moreover, we also show that the vanishing error floor property [3] of this coding scheme is universal for arbitrary spectrum of the coding matrix.
Data Augmentation to Address Out-of-Vocabulary Problem in Low-Resource Sinhala-English Neural Machine Translation

Out­of­Vocabulary (OOV) is a problem for Neural Machine Translation (NMT). OOV refers to words with a low occurrence in the training data, or to those that are absent from the training data. To alleviate this, word or phrase­based Data Augmentation (DA) tech­ niques have been used. However, existing DA techniques have addressed only one of these OOV types and limit to considering either syntactic constraints or semantic constraints. We present a word and phrase replacement­ based DA technique that consider both types of OOV, by augmenting (1) rare words in the ex­ isting parallel corpus, and (2) new words from a bilingual dictionary. During augmentation, we consider both syntactic and semantic prop­ erties of the words to guarantee fluency in the synthetic sentences. This technique was ex­ perimented with low resource Sinhala­English language pair. We observe with only semantic constraints in the DA, the results are compara­ ble with the scores obtained considering syn­ tactic constraints, and is favourable for low­ resourced languages that lacks linguistic tool support. Additionally, results can be further improved by considering both syntactic and semantic constraints.
Secure Coding via Gaussian Random Fields

Inverse probability problems whose generative models are given by strictly nonlinear Gaussian random fields show the all-or-nothing behavior: There exists a critical rate at which Bayesian inference exhibits a phase transition. Below this rate, the optimal Bayesian estimator recovers the data perfectly, and above it the recovered data becomes uncorrelated. This study uses the replica method from the theory of spin glasses to show that this critical rate is the channel capacity. This interesting finding has a particular application to the problem of secure transmission: A strictly nonlinear Gaussian random field along with random binning can be used to securely encode a confidential message in a wiretap channel. Our large-system characterization demonstrates that this secure coding scheme asymptotically achieves the secrecy capacity of the Gaussian wiretap channel.
GeoPointGAN: Synthetic Spatial Data with Local Label Differential Privacy

Synthetic data generation is a fundamental task for many data management and data science applications. Spatial data is of particular interest, and its sensitive nature often leads to privacy concerns. We introduce GeoPointGAN, a novel GAN-based solution for generating synthetic spatial point datasets with high utility and strong individual level privacy guarantees. GeoPointGAN’s architecture includes a novel point transformation generator that learns to project randomly generated point co-ordinates into meaningful synthetic co-ordinates that capture both microscopic (e.g., junctions, squares) and macroscopic (e.g., parks, lakes) geographic features. We provide our privacy guarantees through label local differential privacy, which is more practical than traditional local differential privacy. We seamlessly integrate this level of privacy into GeoPointGAN by augmenting the discriminator to the point level and implementing a randomized response-based mechanism that flips the labels associated with the ‘real’ and ‘fake’ points used in training. Extensive experiments show that GeoPointGAN significantly outperforms recent solutions, improving by up to 10 times compared to the most competitive baseline. We also evaluate GeoPointGAN using range, hotspot, and facility location queries, which confirm the practical effectiveness of GeoPointGAN for privacy-preserving querying. The results illustrate that a strong level of privacy is achieved with little-to-no adverse utility cost, which we explain through the generalization and regularization effects that are realized by flipping the labels of the data during training.
Seeing Sounds, Hearing Shapes: a gamified study to evaluate sound-sketches

Sound-shape associations, a subset of cross-modal associations between the auditory and visual domain, have been studied mainly in the context of matching a set of purposefully crafted shapes to sounds. Recent studies have explored how humans represent sound through free-form sketching and how a graphical sketch input could be used for sound production. In this paper, the potential of communicating sound characteristics through these free-form sketches is investigated in a gamified study that was conducted with eighty-two participants at two online exhibition events. The results show that participants managed to recognise sounds at a higher rate than the random baseline would suggest, however it appeared difficult to visually encode nuanced timbral differences.
Network structural perturbation against interlayer link prediction

Interlayer link prediction aims at matching the same entities across diﬀerent layers of the multiplex network. Existing studies attempt to predict more accurately, eﬃciently, or generically from the aspects of network structure, attribute characteristics, and their combination. Few of them analyze the eﬀects of intralayer links. Namely, few works study the backbone structures which can eﬀectively preserve the predictive accuracy while dealing with a smaller number of intralayer links. It can be used to investigate what types of intralayer links are most important for correct prediction. Are there any intralayer links whose presence leads to worse predictive performance than their absence, and how to attack the prediction algorithms at the minimum cost? To this end, two kinds of network structural perturbation methods are proposed. For the scenario where the structural information of the whole network is completely known, we oﬀer a global perturbation strategy that gives diﬀerent perturbation weights to diﬀerent types of intralayer links and then selects a predetermined proportion of intralayer links to remove according to the weights. In contrast, if these information cannot be obtained at one time, we design a biased random walk procedure, local perturbation strategy, to execute perturbation. Four kinds of interlayer link prediction algorithms are carried out on diﬀerent real-world and artiﬁcial perturbed multiplex networks. We ﬁnd out that the intralayer links connected with small degree nodes have the most signiﬁcant impact on the prediction accuracy. The intralayer links connected with large degree nodes may have side
Automatic Rule Induction for Efficient Semi-Supervised Learning

Semi-supervised learning has shown promise in allowing NLP models to generalize from small amounts of labeled data. Meanwhile, pretrained transformer models act as black-box correlation engines that are difficult to explain and sometimes behave unreliably. In this paper, we propose tackling both of these challenges via Automatic Rule Induction (ARI), a simple and general-purpose framework for the automatic discovery and integration of symbolic rules into pretrained transformer models. First, we extract weak symbolic rules from low-capacity machine learning models trained on small amounts of labeled data. Next, we use an attention mechanism to integrate these rules into high-capacity pretrained transformer models. Last, the rule-augmented system becomes part of a self-training framework to boost supervision signal on unlabeled data. These steps can be layered beneath a variety of existing weak supervision and semi-supervised NLP algorithms in order to improve performance and interpretability. Experiments across nine sequence classification and relation extraction tasks suggest that ARI can improve state-of-the-art methods with no manual effort and minimal computational overhead.
The Weyl double copy in maximally symmetric spacetimes

: Using the method we proposed previously, in which the spin-1 / 2 massless spinors (Dirac-Weyl ﬁelds) are regarded as basic units, we study the Weyl double copy for vacuum solutions with a cosmological constant. The result explicitly demonstrates that the single and zeroth copy satisfy conformally invariant ﬁeld equations on conformally ﬂat spacetime, based on the exact non-twisting vacuum type N and vacuum type D cases. Furthermore, irrespective of the presence of a cosmological constant, we show that the zeroth copy not only connects gravity ﬁelds with the single copy but also connects degenerate electromagnetic ﬁelds with Dirac-Weyl ﬁelds in the curved spacetime. Moreover, the study shows that the zeroth copy plays a vital role in time-dependent radiation spacetimes. In particular, for Robinson-Trautman (Λ) gravitational waves, as opposed to the single copy, we ﬁnd that the zeroth copy carries extra information to indicate whether the sources of associated gravitational waves are time-like, null, or space-like.
Stop the Spread: A Contextual Integrity Perspective on the Appropriateness of COVID-19 Vaccination Certificates

We present an empirical study exploring how privacy influences the acceptance of vaccination certificate (VC) deployments across different realistic usage scenarios. The study employed the privacy framework of Contextual Integrity, which has been shown to be particularly effective in capturing people’s privacy expectations across different contexts. We use a vignette methodology, where we selectively manipulate salient contextual parameters to learn whether and how they affect people’s attitudes towards VCs. We surveyed 890 participants from a demographically-stratified sample of the US population to gauge the acceptance and overall attitudes towards possible VC deployments to enforce vaccination mandates and the different information flows VCs might entail. Analysis of results collected as part of this study is used to derive general normative observations about different possible VC practices and to provide guidance for the possible deployments of VCs in different contexts.
Regularization for Wasserstein distributionally robust optimization
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Optimal transport has recently proved to be a useful tool in various machine learning applications, needing comparisons of probability measures. Among these, applications of distributionally robust optimization naturally involve Wasserstein distances in their models of uncertainty, capturing data shifts or worst-case scenarios. Inspired by the success of the regularization of Wasserstein distances in optimal transport, we study in this paper the regularization of Wassserstein distributionally robust optimization. First, we derive a general strong duality result of regularized Wasserstein distributionally robust problems. Second, we refine this duality result in the case of entropic regularization and provide an approximation result when the regularization parameters vanish.
Parton distribution functions and intrinsic charm at LHCb

At LHCb, proton parton distribution functions (PDFs) can be studied in a unique phase space complementary to that accessible by ATLAS and CMS, corresponding to low and high values of Bjorken- x . The measurements of vector boson production in the forward region, with and without an associated jet, are presented. These measurements can be used to constrain the proton PDFs, and in particular, the production of a Z boson in association with a c -jet can be studied to measure the intrinsic charm content of the proton.
T T Deformations of Supersymmetric Quantum Mechanics

We deﬁne a manifestly supersymmetric version of the T T deformation appropriate for a class of (0 + 1)-dimensional theories with N = 1 or N = 2 supersymmetry, including one presentation of the super-Schwarzian theory which is dual to JT supergravity. These deformations are written in terms of Noether currents associated with translations in superspace, so we refer to them collectively as f ( Q ) deformations. We provide evidence that the f ( Q ) deformations of N = 1 and N = 2 theories are on-shell equivalent to the dimensionally reduced supercurrent-squared deformations of 2 d theories with N = (0 , 1) and N = (1 , 1) supersymmetry, respectively. In the N = 1 case, we present two forms of the f ( Q ) deformation which drive the same ﬂow, and clarify their equivalence by studying the analogous equivalent deformations in the non-supersymmetric setting.