Miracle-dz/UniFAR-FTUs huggingface dataset:
Fluctuations, Bias, Variance & Ensemble of Learners: Exact Asymptotics for Convex Losses in High-Dimension

From the sampling of data to the initialisation of parameters, randomness is ubiquitous in modern Machine Learning practice. Understanding the statistical fluctuations engendered by the different sources of randomness in prediction is therefore key to understanding robust generalisation. In this manuscript we develop a quantitative and rigorous theory for the study of fluctuations in an ensemble of generalised linear models trained on different, but correlated, features in high-dimensions. In particular, we provide a complete description of the asymptotic joint distribution of the empirical risk minimiser for generic convex loss and regularisation in the high-dimensional limit. Our result encompasses a rich set of classification and regression tasks, such as the lazy regime of overparametrised neural networks, or equivalently the random features approximation of kernels. While allowing to study directly the mitigating effect of ensembling (or bagging) on the bias-variance decomposition of the test error, our analysis also helps disentangle the contribution of statistical fluctuations, and the singular role played by the interpolation threshold that are at the roots of the"double-descent"phenomenon.
Research on Integration Practice and Theoretics Curriculum Development Method Based on Working Process and Action-oriented

Since the beginning of 21 century,based on the two kinds of curriculun mode including "theory +experiment" and " theory +practice",the currilum mode of integration of practices and theoretics of higher vocational education is gradually formed in China.Its curriculum development mainly contains five steps which are the position analysis,working process analysisi,action refining,Learning refining and learning situation.Its teaching process includs six steps such as the information,the plan,the decision,the implementation and the evaluation,etc.
Any Variational Autoencoder Can Do Arbitrary Conditioning

Arbitrary conditioning is an important problem in unsupervised learning, where we seek to model the conditional densities p ( x u | x o ) that underly some data, for all possible non-intersecting sub-sets o, u ⊂ { 1 , . , d } . However, the vast majority of density estimation only focuses on modeling the joint distribution p ( x ) , in which important conditional dependencies between features are opaque. We propose a simple and general framework, coined Posterior Matching, that enables any Variational Autoencoder (VAE) to perform arbitrary conditioning, without modiﬁcation to the VAE itself. Posterior Matching applies to the numerous existing VAE-based approaches to joint density estimation, thereby circumventing the specialized models required by previous approaches to arbitrary conditioning. We ﬁnd that Posterior Matching achieves performance that is comparable or superior to current state-of-the-art methods for a variety of tasks.
PokeRRT: Poking as a Skill and Failure Recovery Tactic for Planar Non-Prehensile Manipulation

In this work, we introduce PokeRRT, a novel motion planning algorithm that demonstrates poking as an effective non-prehensile manipulation skill to enable fast manipulation of objects and increase the size of a robot’s reachable workspace. We showcase poking as a failure recovery tactic used synergistically with pick-and-place for resiliency in cases where pick-and-place initially fails or is unachievable. Our experiments demonstrate the efficiency of the proposed framework in planning object trajectories using poking manipulation in uncluttered and cluttered environments. In addition to quantitatively and qualitatively demonstrating the adaptability of PokeRRT to different scenarios in both simulation and real-world settings, our results show the advantages of poking over pushing and grasping in terms of success rate and task time.
Are Mutually Intelligible Languages Easier to Translate?

Two languages are considered mutually intelligible if their native speakers can communicate with each other, while using their own mother tongue. How does the fact that humans perceive a language pair as mutually intelligible affect the ability to learn a translation model between them? We hypothesize that the amount of data needed to train a neural ma-chine translation model is anti-proportional to the languages' mutual intelligibility. Experiments on the Romance language group reveal that there is indeed strong correlation between the area under a model's learning curve and mutual intelligibility scores obtained by studying human speakers.
e-Government maturity models

E-governance is considered as a new model of administration which is set to revolutionise the world of management and basically works in the government sector to extend different services to both public and private sector. Electronic Government reflects the ultimate visions for public administrations and governments to make change. However, e-Government is not a simple on-line information provision. It requires an evolutionary and comprehensive architecture to avoid unnecessary duplication of infrastructure and major components and to integrate disparate processes, services and activities located outside administrations. This paper will review e-government maturity models in order to compare and synthesize them.
Towards Fast and Accurate Federated Learning with non-IID Data for Cloud-Based IoT Applications

As a promising method of central model training on decentralized device data while securing user privacy, Federated Learning (FL)is becoming popular in Internet of Things (IoT) design. However, when the data collected by IoT devices are highly skewed in a non-independent and identically distributed (non-IID) manner, the accuracy of vanilla FL method cannot be guaranteed. Although there exist various solutions that try to address the bottleneck of FL with non-IID data, most of them suffer from extra intolerable communication overhead and low model accuracy. To enable fast and accurate FL, this paper proposes a novel data-based device grouping approach that can effectively reduce the disadvantages of weight divergence during the training of non-IID data. However, since our grouping method is based on the similarity of extracted feature maps from IoT devices, it may incur additional risks of privacy exposure. To solve this problem, we propose an improved version by exploiting similarity information using the Locality-Sensitive Hashing (LSH) algorithm without exposing extracted feature maps. Comprehensive experimental results on well-known benchmarks show that our approach can not only accelerate the convergence rate, but also improve the prediction accuracy for FL with non-IID data.
Filtering In Implicit Neural Networks

Implicit neural networks (INNs) are very effective for learning data representation. However, most INNs inevitably generate over-smoothed patches or obvious noisy artifacts in the results when the data has many scales of details or a wide range of frequencies, leading to signiﬁcant performance reduction. Adapting the result containing both noise and over-smoothed regions usually suffers from either over smoothing or noisy issues. To overcome this challenge, we propose a new framework, coined FINN, that integrated a ﬁltering module to the implicit neural network to perform data ﬁtting while ﬁltering artifacts. The ﬁltering module has a smoothing operator that acts on the intermediate results of the network and a recovering operator that brings distinct details from the input back to the regions overly smoothed. The proposed method signiﬁcantly alleviates over smoothing or noisy issues. We demonstrate the advantage of the FINN on the image regression task, considering both real and synthetic images, and showcases signiﬁcant improvement on both quantitative and qualitative results compared to state-of-the-art methods. Moreover, FINN yields better performance in both convergence speed and network stability. Source code is available at https://github.com/yixin26/FINN.
Fast Relative Entropy Coding with A* coding

Relative entropy coding (REC) algorithms encode a sample from a target distribution $Q$ using a proposal distribution $P$, such that the expected codelength is $\mathcal{O}(D_{KL}[Q \,||\, P])$. REC can be seamlessly integrated with existing learned compression models since, unlike entropy coding, it does not assume discrete $Q$ or $P$, and does not require quantisation. However, general REC algorithms require an intractable $\Omega(e^{D_{KL}[Q \,||\, P]})$ runtime. We introduce AS* and AD* coding, two REC algorithms based on A* sampling. We prove that, for continuous distributions over $\mathbb{R}$, if the density ratio is unimodal, AS* has $\mathcal{O}(D_{\infty}[Q \,||\, P])$ expected runtime, where $D_{\infty}[Q \,||\, P]$ is the R\'enyi $\infty$-divergence. We provide experimental evidence that AD* also has $\mathcal{O}(D_{\infty}[Q \,||\, P])$ expected runtime. We prove that AS* and AD* achieve an expected codelength of $\mathcal{O}(D_{KL}[Q \,||\, P])$. Further, we introduce DAD*, an approximate algorithm based on AD* which retains its favourable runtime and has bias similar to that of alternative methods. Focusing on VAEs, we propose the IsoKL VAE (IKVAE), which can be used with DAD* to further improve compression efficiency. We evaluate A* coding with (IK)VAEs on MNIST, showing that it can losslessly compress images near the theoretically optimal limit.
Detecting False Rumors from Retweet Dynamics on Social Media

False rumors are known to have detrimental effects on society. To prevent the spread of false rumors, social media platforms such as Twitter must detect them early. In this work, we develop a novel probabilistic mixture model that classifies true vs. false rumors based on the underlying spreading process. Specifically, our model is the first to formalize the self-exciting nature of true vs. false retweeting processes. This results in a novel mixture marked Hawkes model (MMHM). Owing to this, our model obviates the need for feature engineering; instead, it directly models the spreading process in order to make inferences of whether online rumors are incorrect. Our evaluation is based on 13,650 retweet cascades of both true. vs. false rumors from Twitter. Our model recognizes false rumors with a balanced accuracy of 64.97 % and an AUC of 69.46 %. It outperforms state-of-the-art baselines (both neural and feature engineering) by a considerable margin but while being fully interpretable. Our work has direct implications for practitioners: it leverages the spreading process as an implicit quality signal and, based on it, detects false content.
Optimizing Gradient-driven Criteria in Network Sparsity: Gradient is All You Need

Network sparsity receives popularity mostly due to its capability to reduce the network complexity. Extensive studies excavate gradient-driven sparsity. Typically, these methods are constructed upon premise of weight independence, which however, is contrary to the fact that weights are mutually inﬂuenced. Thus, their performance remains to be improved. In this paper, we propose to further optimize gradient-driven sparsity (OptG) by solving this independence paradox. Our motive comes from the recent advances on supermask training which shows that sparse subnetworks can be located in a randomly initialized network by simply updating mask values without modifying any weight. We prove that supermask training is to accumulate the weight gradients and can partly solve the independence paradox. Consequently, OptG integrates supermask training into gradient-driven sparsity, and a specialized mask optimizer is designed to solve the independence paradox. Experiments show that OptG can well surpass many existing state-of-the-art competitors. Our code is available at https://github.com/zyxxmu/OptG .
Robustness of Deep Recommendation Systems to Untargeted Interaction Perturbations

While deep learning-based sequential recommender systems are widely used in practice, their sensitivity to untargeted training data perturbations is unknown. Untargeted perturbations aim to modify ranked recommendation lists for all users at test time, by inserting imperceptible input perturbations during training time. Existing perturbation methods are mostly targeted attacks optimized to change ranks of target items, but not suitable for untargeted scenarios. In this paper, we develop a novel framework in which user-item training interactions are perturbed in unintentional and adversarial settings. First, through comprehensive experiments on four datasets, we show that four popular recommender models are unstable against even one random perturbation. Second, we establish a cascading effect in which minor manipulations of early training interactions can cause extensive changes to the model and the generated recommendations for all users. Leveraging this effect, we propose an adversarial perturbation method CASPER which identifies and perturbs an interaction that induces the maximal cascading effect. Experimentally, we demonstrate that CASPER reduces the stability of recommendation models the most, compared to several baselines and state-of-the-art methods. Finally, we show the runtime and success of CASPER scale near-linearly with the dataset size and the number of perturbations, respectively.
Leela Zero Score: a Study of a Score-based AlphaGo Zero

on a 9 × 9 board to predict score differences instead of winrates. We ﬁnd that the training produces a rational player, and we analyze its style against a strong amateur human player, to ﬁnd that it is prone to some mistakes when the outcome is close. We compare its strength against SAI, an AlphaGo Zero-like software working on the 9 × 9 board, and ﬁnd that the training of Leela Zero Score has reached a premature convergence to a player weaker than SAI.
Extracting Built Environment Features for Planning Research with Computer Vision: A Review and Discussion of State-of-the-Art Approaches

This is an extended abstract for a presentation at The 17th International Conference on CUPUM - Computational Urban Planning and Urban Management in June 2021. This study presents an interdisciplinary synthesis of the state-of-the-art approaches in computer vision technologies to extract built environment features that could improve the robustness of empirical research in planning. We discussed the findings from the review of studies in both planning and computer science.
Evidence for nodal superconductivity in infinite-layer nickelates

Shannon P. Harvey, 2, ∗ Bai Yang Wang, 3, ∗ Jennifer Fowlie, 2 Motoki Osada, 2 Kyuho Lee, 3 Yonghun Lee, 2 Danfeng Li, 2, † and Harold Y. Hwang 2 Stanford Institute for Materials and Energy Sciences, SLAC National Accelerator Laboratory, Menlo Park, CA USA Department of Applied Physics, Stanford University, Stanford, CA, USA Department of Physics, Stanford University, Stanford, CA, USA (Dated: February 1, 2022)
Privacy Limitations of Interest-based Advertising on The Web: A Post-mortem Empirical Analysis of Google's FLoC

In 2020, Google announced it would disable third-party cookies in the Chrome browser to improve user privacy. In order to continue to enable interest-based advertising while mitigating risks of individualized user tracking, Google proposed FLoC. The FLoC algorithm assigns users to "cohorts" that represent groups of users with similar browsing behaviors so that ads can be served to users based on their cohort. In 2022, after testing FLoC in a real world trial, Google canceled the proposal with little explanation in favor of another way to enable interest-based advertising. This work provides a post-mortem analysis of two critical privacy risks for FloC by applying an implementation of FLoC to a real-world browsing history dataset collected from over 90,000 U.S. devices over a one year period. First, we show how, contrary to its privacy goals, FLoC would have enabled individualized cross-site user tracking by providing a unique identifier for users available across sites, similar to the third-party cookies FLoC was meant to be an improvement over. We show how FLoC cohort ID sequences observed over time can provide this unique identifier to trackers, even with third-party cookies disabled. We estimate the number of users in our dataset that could be uniquely identified by FLoC IDs is more than 50% after 3 weeks and more than 95% after 4 weeks. We also show how these risks increase when cohort data are combined with browser fingerprinting, and how our results are conservative underestimates of the risks FLoC would have posed in a real-world deployment. Second, we examine the risk of FLoC leaking sensitive demographic information about users. Although we find statistically significant differences in browsing behaviors between demographic groups, we do not find that FLoC significantly risks exposing race or income information about users in our dataset. Our contributions provide insights and example analyses for future novel approaches that seek to protect user privacy while monetizing the web.
Single Object Tracking: A Survey of Methods, Datasets, and Evaluation Metrics

Object tracking is one of the foremost assignments in computer vision that has numerous commonsense applications such as traffic monitoring, robotics, autonomous vehicle tracking, and so on. Different researches have been tried later a long time, but since of diverse challenges such as occlusion, illumination variations, fast motion, etc. researches in this area continues. In this paper, different strategies of the following objects are inspected and a comprehensive classification is displayed that classified the following strategies into four fundamental categories of feature-based, segmentation-based, estimation-based, and learning-based methods that each of which has its claim sub-categories. The most center of this paper is on learning-based strategies, which are classified into three categories of generative strategies, discriminative strategies, and reinforcement learning. One of the sub-categories of the discriminative show is deep learning. Since of high-performance, deep learning has as of late been exceptionally much consider. Finally, the different datasets and the evaluation methods that are most commonly used will be introduced.
Experimental pressure sensing and technology of piezoelectric microwave/RF MEMS filters

In this work, we analyze the pressure sensing of a thin film molybdenum/aluminumnitride/molybdenum (Mo/AlN/Mo) microwave/RF MEMS filter fabricated by a simple technology. After an experimental characterization in a frequency range between 1 and 36 GHz, we focused on the piezoelectric effect due to the stress properties of the piezoelectric AlN layer by applying forces by means of weights. Variations in the bandpass region of the microwave/RF filter are observed by proving high sensitivity also for low applied weights. We check by a properly designed three-dimensional (3D) finite-element method (FEM) tool the pressure-sensing property of the proposed device. Finally, we analyze the bad gap property of a chip with central defect around 40 GHz.
Bidders' Responses to Auction Format Change in Internet Display Advertising Auctions

We study actual bidding behavior when a new auction format gets introduced into the marketplace. More specifically, we investigate this question using a novel dataset on internet display advertising auctions that exploits a staggered adoption by different publishers (sellers) of first-price auctions (FPAs), instead of the traditional second-price auctions (SPAs). Event study regression estimates indicate that, immediately after the auction format change, the revenue per sold impression (price) jumped considerably for the treated publishers relative to the control publishers, ranging from 35% to 75% of the pre-treatment price level of the treatment group. Further, we observe that in later auction format changes the increase in the price levels under FPAs relative to price levels under SPAs dissipates over time, reminiscent of the celebrated revenue equivalence theorem. A possible interpretation of these facts is initially insufficient bid shading after the format change rather than an immediate shift to a new Bayesian Nash equilibrium. The gradual decrease in prices can be interpreted as the result of bidders' learning to shade their bids. We also present suggestive evidence that bidders' sophistication may have impacted their response to the auction format change. Our work constitutes one of the first field studies on bidders' responses to auction format changes, providing an important complement to theoretical model predictions. As such, it provides valuable information to auction designers when considering the implementation of different formats. The full version of the paper is available at: https://arxiv.org/abs/2110.13814
Theoretically Motivated Data Augmentation and Regularization for Portfolio Construction

The task we consider is portfolio construction in a speculative market, a fundamental problem in modern finance. While various empirical works now exist to explore deep learning in finance, the theory side is almost non-existent. In this work, we focus on developing a theoretical framework for understanding the use of data augmentation for deep-learning-based approaches to quantitative finance. The proposed theory clarifies the role and necessity of data augmentation for finance; moreover, our theory implies that a simple algorithm of injecting a random noise of strength to the observed return rt is better than not injecting any noise and a few other financially irrelevant data augmentation techniques.
Steepest entropy ascent solution for a continuous-time quantum walker.

We consider the steepest entropy ascent (SEA) ansatz to describe the nonlinear thermodynamic evolution of a quantum system. Recently this principle has been dubbed the fourth law of thermodynamics [Beretta, Phil. Trans. R. Soc. A 378, 20190168 (2020)10.1098/rsta.2019. 0168]. A unique global equilibrium state exists in this context, and any other state is driven by the maximum entropy generation principle towards this equilibrium. We study the SEA evolution of a continuous-time quantum walker (CTQW) on a cycle graph with N nodes. SEA solutions are difficult to find analytically. We provide an approximate scheme to find a general single-particle evolution equation governed by the SEA principle, whose solution produces dissipation dynamics. We call this scheme the fixed Lagrange's multiplier (FLM) method. In the Bloch sphere representation, we find trajectories traced out by the Bloch vector within the sphere itself. We have discussed these trajectories under various initial conditions for the case of a qubit. A similar dissipative motion is also observed in the case of CTQW, where probability amplitudes have been used to characterize decoherence. Our FLM scheme shows good agreement with numerical results. As we report, in CTQW, a strong delocalization exists for low system relaxation time.