Miracle-dz/UniFAR-FTUs huggingface dataset:
Integral smoothed particle hydrodynamics with an improved partition of unit and a better track of contact discontinuities

The correct evaluation of gradients is at the cornerstone of the smoothed particle hydrodynamics (SPH) technique. Using an integral approach to estimate gradients has proven to enhance accuracy substantially. Such approach retains the Lagrangian structure of SPH equations and is fully conservative. In this paper we study, among other things, the connection between the choice of the volume elements (VEs), which enters in the SPH summations, and the accuracy in the gradient estimation within the integral approach scheme (ISPH). A new kind of VEs are proposed which improve the partition of unit and are fully compatible with the Lagrangian formulation of SPH, including the grad-h corrections. Using analytic considerations, simple static toy models in 1D, and a few full 3D test cases, we show that any improvement in the partition of unit also leads to a better calculation of gradients when the integral approach is used jointly. Additionally, we propose a simple-to-implement variant of the ISPH scheme which is more adequate to handle sharp density contrasts.
Metrics of constant negative scalar-Weyl curvature

Extending Aubin's construction of metrics with constant negative scalar curvature, we prove that every $n$-dimensional closed manifold admits a Riemannian metric with constant negative scalar-Weyl curvature, that is $R+t|W|, t\in\mathbb{R}$. In particular, there are no topological obstructions for metrics with $\varepsilon$-pinched Weyl curvature and negative scalar curvature.
High-Energy Neutrino Production in Clusters of Galaxies

Clusters of galaxies can potentially produce cosmic rays (CRs) up to very high energies via large-scale shocks and turbulent acceleration. Due to their unique magnetic-field configuration, CRs with energy ≤1017 eV can be trapped within these structures over cosmological time-scales, and generate secondary particles, including neutrinos and gamma rays, through interactions with the background gas and photons. In this work, we compute the contribution from clusters of galaxies to the diffuse neutrino background. We employ 3D cosmological magnetohydrodynamical simulations of structure formation to model the turbulent intergalactic medium. We use the distribution of clusters within this cosmological volume to extract the properties of this population, including mass, magnetic field, temperature, and density. We propagate CRs in this environment using multidimensional Monte Carlo simulations across different redshifts (from z ∼ 5 to z = 0), considering all relevant photohadronic, photonuclear, and hadronuclear interaction processes. We find that, for CRs injected with a spectral index α = 1.5–2.7 and cutoff energy Emax = 1016–5 × 1017 eV, clusters contribute to a sizeable fraction to the diffuse flux observed by the IceCube Neutrino Observatory, but most of the contribution comes from clusters with M ≳ 1014 M⊙ and redshift z ≲ 0.3. If we include the cosmological evolution of the CR sources, this flux can be even higher.
Predicting Pneumonia and Region Detection from X-Ray Images using Deep Neural Network

Biomedical images are increasing drastically. Along the way, many machine learning algorithms have been proposed to predict and identify various kinds of diseases. One such disease is Pneumonia which is an infection caused by both bacteria and viruses through the inflammation of a person's lung air sacs. In this paper, an algorithm was proposed that receives x-ray images as input and verifies whether this patient is infected by Pneumonia as well as specific region of the lungs that the inflammation has occurred at. The algorithm is based on the transfer learning mechanism where pre-trained ResNet-50 (Convolutional Neural Network) was used followed by some custom layer for making the prediction. The model has achieved an accuracy of 90.6 percent which confirms that the model is effective and can be implemented for the detection of Pneumonia in patients. Furthermore, a class activation map is used for the detection of the infected region in the lungs. Also, PneuNet was developed so that users can access more easily and use the services.
Momentum^2 Teacher: Momentum Teacher with Momentum Statistics for Self-Supervised Learning

In this paper, we present a novel approach, Momentum$^2$ Teacher, for student-teacher based self-supervised learning. The approach performs momentum update on both network weights and batch normalization (BN) statistics. The teacher's weight is a momentum update of the student, and the teacher's BN statistics is a momentum update of those in history. The Momentum$^2$ Teacher is simple and efficient. It can achieve the state of the art results (74.5\%) under ImageNet linear evaluation protocol using small-batch size(\eg, 128), without requiring large-batch training on special hardware like TPU or inefficient across GPU operation (\eg, shuffling BN, synced BN). Our implementation and pre-trained models will be given on GitHub\footnote{https://github.com/zengarden/momentum2-teacher}.
Creation and Evaluation of a Pretertiary Artificial Intelligence (AI) Curriculum

Contributions: The Chinese University of Hong Kong (CUHK)-Jockey Club AI for the Future Project (AI4Future) co-created the first pretertiary AI curriculum at the secondary school level for Hong Kong and evaluated its efficacy. This study added to the AI education community by introducing a new AI curriculum framework. The preposttest multifactors evaluation about students’ perceptions of AI learning confirmed that the curriculum is effective in promoting AI learning. The teachers also confirmed the co-creation process enhanced their capacity to implement AI education. Background: AI4Future is a cross-sector project that engages five major partners—CUHK’s Faculty of Engineering and Faculty of Education, secondary schools, Hong Kong government, and AI industry. A team of 14 professors collaborated with 17 principals and teachers from six secondary schools to co-create the curriculum. Research Questions: Would the curriculum significantly improve the student perceived competence, attitude, and motivation toward AI learning? How does the co-creation process benefit the implementation of the curriculum? Methodology: The participants were 335 students and eight teachers from the secondary schools. This study adopted a mix-method with quantitative data measures at pre- and post-questionnaires and qualitative data emphasizes teachers’ perspectives on the co-creation process. Paired ${t}$ -tests and ANCOVAs, and thematic analysis were used to analyze the data. Findings: 1) students perceived greater competence and developed a more positive attitude to learn AI and 2) the co-creation process enhanced teachers’ knowledge in AI, as well as fostered teachers’ autonomy in bringing the subject matter into their classrooms.
Learning over Families of Sets - Hypergraph Representation Learning for Higher Order Tasks

Graph representation learning has made major strides over the past decade. However, in many relational domains, the input data are not suited for simple graph representations as the relationships between entities go beyond pairwise interactions. In such cases, the relationships in the data are better represented as hyperedges (set of entities) of a non-uniform hypergraph. While there have been works on principled methods for learning representations of nodes of a hypergraph, these approaches are limited in their applicability to tasks on non-uniform hypergraphs (hyperedges with different cardinalities). In this work, we exploit the incidence structure to develop a hypergraph neural network to learn provably expressive representations of variable sized hyperedges which preserve local-isomorphism in the line graph of the hypergraph, while also being invariant to permutations of its constituent vertices. Specifically, for a given vertex set, we propose frameworks for (1) hyperedge classification and (2) variable sized expansion of partially observed hyperedges which captures the higher order interactions among vertices and hyperedges. We evaluate performance on multiple real-world hypergraph datasets and demonstrate consistent, significant improvement in accuracy, over state-of-the-art models.
RadioAstron reveals a spine-sheath jet structure in 3C 273

We present Space-VLBI RadioAstron observations at 1.6 GHz and 4.8 GHz of the flat spectrum radio quasar 3C 273, with detections on baselines up to 4.5 and 3.3 Earth Diameters, respectively. Achieving the best angular resolution at 1.6 GHz to date, we have imaged limb-brightening in the jet, not previously detected in this source. In contrast, at 4.8 GHz, we detected emission from a central stream of plasma, with a spatial distribution complementary to the limb-brightened emission, indicating an origin in the spine of the jet. While a stratification across the jet width in the flow density, internal energy, magnetic field, or bulk flow velocity are usually invoked to explain the limb-brightening, the different jet structure detected at the two frequencies probably requires a stratification in the emitting electron energy distribution. Future dedicated numerical simulations will allow the determination of which combination of physical parameters are needed to reproduce the spine/sheath structure observed by Space-VLBI with RadioAstron in 3C 273
ES-ENAS: Combining Evolution Strategies with Neural Architecture Search at No Extra Cost for Reinforcement Learning

We introduce ES-ENAS, a simple neural architecture search (NAS) algorithm for the purpose of reinforcement learning (RL) policy design, by combining Evolutionary Strategies (ES) [30, 40] and Efficient NAS (ENAS) [36, 54, 61] in a highly scalable and intuitive way. Our main insight is noticing that ES is already a distributed blackbox algorithm, and thus we may simply insert a model controller from ENAS into the central aggregator in ES and obtain weight sharing properties for free. By doing so, we bridge the gap from NAS research in supervised learning settings to the reinforcement learning scenario through this relatively simple marriage between two different lines of research, and are one of the first to apply controller-based NAS techniques to RL. We demonstrate the utility of our method by training combinatorial neural network architectures for RL problems in continuous control, via edge pruning and weight sharing. We also incorporate a wide variety of popular techniques from modern NAS literature, including multiobjective optimization and varying controller meth-ods, to showcase their promise in the RL field and discuss possible extensions. We achieve > 90% network compression for multiple tasks, which may be special interest in mobile robotics [15] with limited storage and computational resources.
Sensitivity of Indian summer monsoon rainfall forecast skill of CFSv2 model to initial conditions and the role of model biases

This study analyses Indian summer monsoon (ISM) seasonal reforecasts by CFSv2 model, initiated from January (4-month lead time, L4) through May (0-month lead time, L0) initial conditions (ICs), to examine the cause for highest all-India ISM rainfall (ISMR) forecast skill with February (3-month lead time, L3) ICs. The reported highest forecast skill for L3 ICs is based on correlation between observed and predicted interannual variation (IAV) of ISMR. Other skill scores such as mean error, bias, RMSE, mean, standard deviation and coeﬃcient of variation, indicate higher or comparable skill for April (L1)/May (L0) ICs. Although theory suggests that skill degrades with increase in lead-time, CFSv2 shows highest skill with L3 ICs, due to predicting ISMR excess of 1983 for which other ICs fail. But, this correct forecasting is caused by wrong forecast of La Ni ˜ na, cooling of the equatorial central Paciﬁc (NINO3.4) during the monsoon season, by L3 ICs. In observation, near-normal sea surface temperatures (SSTs) pre-vailed over NINO3.4 and ISMR excess was due to variation of convection over equatorial Indian Ocean or EQUINOO which CFSv2 failed to capture with all ICs. Major results are reaﬃrmed by analysing an optimum number of experimental reforecasts by current version of CFSv2 initiated from ﬁve late-April/early-May ICs having shorter yet useful forecast lead time. These experimental reforecasts are found to have least seasonal biases and highest correlation skill score if 1983 is excluded. Model deﬁciencies such as over-sensitivity of ISMR to SST variation over NINO3.4 (ENSO) and unrealistic inﬂuence of ENSO on the EQUINOO, contribute to errors in ISMR forecasting. Whereas, in observation, ISMR is inﬂuenced by both ENSO and EQUINOO. The forecast skill for Boreal summer ENSO is found to be deﬁcient in CFSv2 with the skill being the lowest for L3/L4 ICs, hinting the possible inﬂuence of dynamical drift induced by long forecast lead-time. Rainfall occurrence despite strong cold bias over NINO3.4 in CFSv2, is associated with a stronger ocean-atmosphere coupling, with a shift of the SST-rainfall relationship pattern to slightly colder SSTs than the observed. These results warrant the need for minimisation of biases in SST boundary forcing to achieve improved ISMR forecasts.
The Unreasonable Effectiveness of Patches in Deep Convolutional Kernels Methods

A recent line of work showed that various forms of convolutional kernel methods can be competitive with standard supervised deep convolutional networks on datasets like CIFAR-10, obtaining accuracies in the range of 87-90% while being more amenable to theoretical analysis. In this work, we highlight the importance of a data-dependent feature extraction step that is key to the obtain good performance in convolutional kernel methods. This step typically corresponds to a whitened dictionary of patches, and gives rise to a data-driven convolutional kernel methods. We extensively study its effect, demonstrating it is the key ingredient for high performance of these methods. Specifically, we show that one of the simplest instances of such kernel methods, based on a single layer of image patches followed by a linear classifier is already obtaining classification accuracies on CIFAR-10 in the same range as previous more sophisticated convolutional kernel methods. We scale this method to the challenging ImageNet dataset, showing such a simple approach can exceed all existing non-learned representation methods. This is a new baseline for object recognition without representation learning methods, that initiates the investigation of convolutional kernel models on ImageNet. We conduct experiments to analyze the dictionary that we used, our ablations showing they exhibit low-dimensional properties.
Moiré-Trapped Interlayer Trions in a Charge-Tunable 
WSe2/MoSe2
 Heterobilayer

Mauro Brotons-Gisbert,1, ∗ Hyeonjun Baek,1 Aidan Campbell,1 Kenji Watanabe,2 Takashi Taniguchi,3 and Brian D. Gerardot1, † Institute of Photonics and Quantum Sciences, SUPA, Heriot-Watt University, Edinburgh EH14 4AS, UK Research Center for Functional Materials, National Institute for Materials Science, 1-1 Namiki, Tsukuba 305-0044, Japan International Center for Materials Nanoarchitectonics, National Institute for Materials Science, 1-1 Namiki, Tsukuba 305-0044, Japan (Dated: August 16, 2021)
Training Learned Optimizers with Randomly Initialized Learned Optimizers

Learned optimizers are increasingly effective, with performance exceeding that of hand designed optimizers such as Adam~\citep{kingma2014adam} on specific tasks \citep{metz2019understanding}. Despite the potential gains available, in current work the meta-training (or `outer-training') of the learned optimizer is performed by a hand-designed optimizer, or by an optimizer trained by a hand-designed optimizer \citep{metz2020tasks}. We show that a population of randomly initialized learned optimizers can be used to train themselves from scratch in an online fashion, without resorting to a hand designed optimizer in any part of the process. A form of population based training is used to orchestrate this self-training. Although the randomly initialized optimizers initially make slow progress, as they improve they experience a positive feedback loop, and become rapidly more effective at training themselves. We believe feedback loops of this type, where an optimizer improves itself, will be important and powerful in the future of machine learning. These methods not only provide a path towards increased performance, but more importantly relieve research and engineering effort.
Generalized Alternating Projections on Manifolds and Convex Sets

In this paper, we extend the previous convergence results for the generalized alternating projection method applied to subspaces in [arXiv:1703.10547] to hold also for smooth manifolds. We show that the algorithm locally behaves similarly in the subspace and manifold settings and that the same rates are obtained. We also present convergence rate results for when the algorithm is applied to non-empty, closed, and convex sets. The results are based on a finite identification property that implies that the algorithm after an initial identification phase solves a smooth manifold feasibility problem. Therefore, the rates in this paper hold asymptotically for problems in which this identification property is satisfied. We present a few examples where this is the case and also a counter example for when this is not.
Star Formation Activity of Galaxies Undergoing Ram Pressure Stripping in the Virgo Cluster.

We study galaxies undergoing ram pressure stripping in the Virgo cluster to examine whether we can identify any discernible trend in their star formation activity. We first use 48 galaxies undergoing different stages of stripping based on HI morphology, HI deficiency, and relative extent to the stellar disk, from the VIVA survey. We then employ a new scheme for galaxy classification which combines HI mass fractions and locations in projected phase space, resulting in a new sample of 365 galaxies. We utilize a variety of star formation tracers, which include g - r, WISE [3.4] - [12] colors, and starburstiness that are defined by stellar mass and star formation rates to compare the star formation activity of galaxies at different stripping stages. We find no clear evidence for enhancement in the integrated star formation activity of galaxies undergoing early to active stripping. We are instead able to capture the overall quenching of star formation activity with increasing degree of ram pressure stripping, in agreement with previous studies. Our results suggest that if there is any ram pressure stripping induced enhancement, it is at best locally modest, and galaxies undergoing enhancement make up a small fraction of the total sample. Our results also indicate that it is possible to trace galaxies at different stages of stripping with the combination of HI gas content and location in projected phase space, which can be extended to other galaxy clusters that lack high-resolution HI imaging.
Information Theoretic Secure Aggregation with User Dropouts

In the robust secure aggregation problem, a server wishes to learn and only learn the sum of the inputs of a number of users while some users may drop out (i.e., may not respond). The identity of the dropped users is not known a priori and the server needs to securely recover the sum of the remaining surviving users. We consider the following minimal two-round model of secure aggregation. Over the first round, any set of no fewer than $U$ users out of $K$ users respond to the server and the server wants to learn the sum of the inputs of all responding users. The remaining users are viewed as dropped. Over the second round, any set of no fewer than $U$ users of the surviving users respond (i.e., dropouts are still possible over the second round) and from the information obtained from the surviving users over the two rounds, the server can decode the desired sum. The security constraint is that even if the server colludes with any $T$ users and the messages from the dropped users are received by the server (e.g., delayed packets), the server is not able to infer any additional information beyond the sum in the information theoretic sense. For this information theoretic secure aggregation problem, we characterize the optimal communication cost. When $U\leq T$, secure aggregation is not feasible, and when $U > T$, to securely compute one symbol of the sum, the minimum number of symbols sent from each user to the server is 1 over the first round, and $1/(U-T)$ over the second round.
Does Continual Learning = Catastrophic Forgetting?

Continual learning is known for suffering from catastrophic forgetting, a phenomenon where earlier learned concepts are forgotten at the expense of more recent samples. In this work, we challenge the assumption that continual learning is inevitably associated with catastrophic forgetting by presenting a set of tasks that surprisingly do not suffer from catastrophic forgetting when learned continually. We provide evidence that these reconstruction-type tasks exhibit positive forward transfer and that single-view 3D shape reconstruction improves the performance on learned and novel categories over time. We provide the novel analysis of knowledge transfer ability by looking at the output distribution shift across sequential learning tasks. Finally, we show that the robustness of these tasks leads to the potential of having a proxy representation learning task for continual classiﬁcation. The code-base, dataset and pre-trained models released with this article can be found at https://github.com/rehg-lab/CLRec.
Challenges for Computational Lexical Semantic Change

The computational study of lexical semantic change (LSC) has taken off in the past few years and we are seeing increasing interest in the field, from both computational sciences and linguistics. Most of the research so far has focused on methods for modelling and detecting semantic change using large diachronic textual data, with the majority of the approaches employing neural embeddings. While methods that offer easy modelling of diachronic text are one of the main reasons for the spiking interest in LSC, neural models leave many aspects of the problem unsolved. The field has several open and complex challenges. In this chapter, we aim to describe the most important of these challenges and outline future directions.
Learnable Embedding Sizes for Recommender Systems

The embedding-based representation learning is commonly used in deep learning recommendation models to map the raw sparse features to dense vectors. The traditional embedding manner that assigns a uniform size to all features has two issues. First, the numerous features inevitably lead to a gigantic embedding table that causes a high memory usage cost. Second, it is likely to cause the over-fitting problem for those features that do not require too large representation capacity. Existing works that try to address the problem always cause a significant drop in recommendation performance or suffers from the limitation of unaffordable training time cost. In this paper, we proposed a novel approach, named PEP (short for Plug-in Embedding Pruning), to reduce the size of the embedding table while obviating a drop in accuracy and computational optimization. PEP prunes embedding parameter where the pruning threshold(s) can be adaptively learned from data. Therefore we can automatically obtain a mixed-dimension embedding-scheme by pruning redundant parameters for each feature. PEP is a general framework that can plug in various base recommendation models. Extensive experiments demonstrate it can efficiently cut down embedding parameters and boost the base model's performance. Specifically, it achieves strong recommendation performance while reducing 97-99% parameters. As for the computation cost, PEP only brings an additional 20-30% time cost compared with base models. Codes are available at https://github.com/ssui-liu/learnable-embed-sizes-for-RecSys.