Miracle-dz/UniFAR-FTUs huggingface dataset:
A Physics Informed Neural Network Approach to Solution and Identification of Biharmonic Equations of Elasticity

We explore an application of the Physics Informed Neural Networks (PINNs) in conjunction with Airy stress functions and Fourier series to find optimal solutions to a few reference biharmonic problems of elasticity and elastic plate theory. Biharmonic relations are fourth-order partial differential equations (PDEs) that are challenging to solve using classical numerical methods, and have not been addressed using PINNs. Our work highlights a novel application of classical analytical methods to guide the construction of efficient neural networks with the minimal number of parameters that are very accurate and fast to evaluate. In particular, we find that enriching feature space using Airy stress functions can significantly improve the accuracy of PINN solutions for biharmonic PDEs.
Weakly Supervised Continual Learning

—Continual Learning (CL) investigates how to train Deep Networks on a stream of tasks without incurring catastrophic forgetting . CL settings proposed in the literature assume that every incoming example is paired with ground-truth annotations. However, this clashes with many real-world applications: gathering labeled data, which is in itself tedious and expensive, becomes indeed infeasible when data ﬂow as a stream and must be consumed in real-time. This work explores Weakly Supervised Continual Learning (WSCL): here, only a small fraction of labeled input examples are shown to the learner. We assess how current CL methods ( e.g. : EWC, LwF, iCaRL, ER, GDumb, DER) perform in this novel and challenging scenario, in which overﬁtting entangles forgetting. Subsequently, we design two novel WSCL methods which exploit metric learning and consistency regularization to leverage unsupervised data while learning. In doing so, we show that not only our proposals exhibit higher ﬂexibility when supervised information is scarce, but also that less than 25% labels can be enough to reach or even outperform SOTA methods trained under full supervision.
Policy Evaluation and Temporal-Difference Learning in Continuous Time and Space: A Martingale Approach

We propose a unified framework to study policy evaluation (PE) and the associated temporal difference (TD) methods for reinforcement learning in continuous time and space. We show that PE is equivalent to maintaining the martingale condition of a process. From this perspective, we find that the mean--square TD error approximates the quadratic variation of the martingale and thus is not a suitable objective for PE. We present two methods to use the martingale characterization for designing PE algorithms. The first one minimizes a ``martingale loss function", whose solution is proved to be the best approximation of the true value function in the mean--square sense. This method interprets the classical gradient Monte-Carlo algorithm. The second method is based on a system of equations called the ``martingale orthogonality conditions" with ``test functions''. Solving these equations in different ways recovers various classical TD algorithms, such as TD($\lambda$), LSTD, and GTD. Different choices of test functions determine in what sense the resulting solutions approximate the true value function. Moreover, we prove that any convergent time-discretized algorithm converges to its continuous-time counterpart as the mesh size goes to zero. We demonstrate the theoretical results and corresponding algorithms with numerical experiments and applications.
An Investigation of Replay-based Approaches for Continual Learning

Continual learning (CL) is a major challenge of machine learning (ML) and describes the ability to learn several tasks sequentially without catastrophic forgetting (CF). Recent works indicate that CL is a complex topic, even more so when real-world scenarios with multiple constraints are involved. Several solution classes have been proposed [1], of which socalled replay-based approaches seem very promising due to their simplicity and robustness. Such approaches store a subset of past samples in a dedicated memory for later processing: while this does not solve all problems, good results have been obtained. In this article, we empirically investigate replay-based approaches of continual learning and assess their potential for applications. Selected recent approaches as well as own proposals are compared on a common set of benchmarks, with a particular focus on assessing the performance of different sample selection strategies. We find that the impact of sample selection increases when a smaller number of samples is stored. Nevertheless, performance varies strongly between different replay approaches. Surprisingly, we find that the most naive rehearsal-based approaches that we propose here can outperform recent state-of-the-art methods.
Connected and Disconnected Sea Partons from CT18 Parametrization of PDFs

The separation of the connected and disconnected sea partons, which were uncovered in the Euclidean path-integral formulation of the hadronic tensor, is accommodated with the CT18 parametrization of the global analysis of the parton distribution functions (PDFs). This is achieved with the help of the distinct small xx behaviors of these two sea parton components and the constraint from the lattice calculation of the ratio of the strange momentum fraction to that of the {\bar u}u‾ or {\bar d}d‾ in the disconnected insertion. This allows lattice calculations of separate flavors in both the connected and disconnected insertions to be directly compared with the global analysis results term by term.
On the Opportunities and Risks of Foundation Models

AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles(e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and environmental impact, legal and ethical considerations). Though foundation models are based on standard deep learning and transfer learning, their scale results in new emergent capabilities,and their effectiveness across so many tasks incentivizes homogenization. Homogenization provides powerful leverage but demands caution, as the defects of the foundation model are inherited by all the adapted models downstream. Despite the impending widespread deployment of foundation models, we currently lack a clear understanding of how they work, when they fail, and what they are even capable of due to their emergent properties. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration commensurate with their fundamentally sociotechnical nature.
Clustering Filipino Disaster-Related Tweets Using Incremental and Density-Based Spatiotemporal Algorithm with Support Vector Machines for Needs Assessment

Social media has played a huge part on how people get informed and communicate with one another. It has helped people express their needs due to distress especially during disasters. Because posts made through it are publicly accessible by default, Twitter is among the most helpful social media sites in times of disaster. With this, the study aims to assess the needs expressed during calamities by Filipinos on Twitter. Data were gathered and classified as either disaster-related or unrelated with the use of Na\"ive Bayes classifier. After this, the disaster-related tweets were clustered per disaster type using Incremental Clustering Algorithm, and then sub-clustered based on the location and time of the tweet using Density-based Spatiotemporal Clustering Algorithm. Lastly, using Support Vector Machines, the tweets were classified according to the expressed need, such as shelter, rescue, relief, cash, prayer, and others. After conducting the study, results showed that the Incremental Clustering Algorithm and Density-Based Spatiotemporal Clustering Algorithm were able to cluster the tweets with f-measure scores of 47.20% and 82.28% respectively. Also, the Na\"ive Bayes and Support Vector Machines were able to classify with an average f-measure score of 97% and an average accuracy of 77.57% respectively.
Spinning guest fields during inflation: leftover signatures

We consider the possibility of extra spinning particles during inflation, focussing on the spin-2 case. Our analysis relies on the well-known fully non-linear formulation of interacting spin-2 theories. We explore the parameter space of the corresponding inflationary Lagrangian and identify regions therein exhibiting signatures within reach of upcoming CMB probes. We provide a thorough study of the early and late-time dynamics ensuring that stability conditions are met throughout the cosmic evolution. We characterise in particular the gravitational wave spectrum and three-point function finding a local-type non-Gaussianity whose amplitude may be within the sensitivity range of both the LiteBIRD and CMB-S4 experiments.
On Support Recovery With Sparse CCA: Information Theoretic and Computational Limits

In this paper, we consider asymptotically exact support recovery in the context of high dimensional and sparse Canonical Correlation Analysis (CCA). Our main results describe four regimes of interest based on information theoretic and computational considerations. In regimes of “low” sparsity we describe a simple, general, and computationally easy method for support recovery, whereas in a regime of “high” sparsity, it turns out that support recovery is information theoretically impossible. For the sake of information theoretic lower bounds, our results also demonstrate a non-trivial requirement on the “minimal” size of the nonzero elements of the canonical vectors that is required for asymptotically consistent support recovery. Subsequently, the regime of “moderate” sparsity is further divided into two sub-regimes. In the lower of the two sparsity regimes, we show that polynomial time support recovery is possible by using a sharp analysis of a co-ordinate thresholding type method. In contrast, in the higher end of the moderate sparsity regime, appealing to the “Low Degree Polynomial” Conjecture, we provide evidence that polynomial time support recovery methods are inconsistent. Finally, we carry out numerical experiments to compare the efficacy of various methods discussed.
Rotational symmetry breaking potential for two-dimensional magnets

Here we present a new perspective to the breakdown of ferromagnetic order in two-dimensional spin-lattice models employing the rotation of the underlying lattice. Using an Ising spin system on a square lattice as a prototype, we demonstrate that an additional low-symmetry interaction may lead to the absence of the truly long-range order and forms aperiodic structure, such as magnetic stripes. Employing annealing and entropic Monte Carlo simulations, we show that our model allows tuning between different phases, magnetically ordered as well as more exotic nonmagnetic phases such as Ising-nematic by changing only one control parameter, which is responsible for the arising of magnetic frustration. In addition, our methodology of considering the coupling between the magnetic structure and the host material can be extended to the study of any type of spin-exchange model in two dimensions and has many potential interesting ramifications and applications. The last several years have witnessed a renewed interest in frustrated magnetism[1–5], which was propelled by the discovery of actual compounds presenting new magnetic exotic phases which can be described by two-dimensional spin-exchange models[6]. These novel excitations inherent to frustrated systems include the possible quantum spin-liquid state[7] and chiral ordered states[8]. Experimentally observable, materials containing layers of transition metal ions on a square lattice are considered quasi-two-dimensional (quasi-2D) for magnetism, as the interactions between layers are much smaller than those within the layers. At sufficiently low energies their magnetic degrees of freedom can be mapped to effective spin-exchange models into 2D Bravais lattices. Commonly, the expected long-range magnetic order is suppressed by magnetic frustration, which arises when there is no unique ground state which simultaneously minimizes the energy of all exchange bonds. Generally, spin-exchange frustration can be due to either the geometry of the lattice (as in pyrochlore[9, 10], spinel, or Kagome systems) or where the competing interactions between magnetic moments cannot be mutually satisfied (as the competing nearest-neighbor and next-nearest-neighbor interactions). In its turn, frustration enhances spin fluctuations, and novel complex phase diagrams, new emergent critical phenomena, and technical applications can emerge. Nonetheless, mechanical vibrations of the crystal lattice (phonons) modulate the crystal ∗ claudio.silva@ifg.edu. br
Event2Graph: Event-driven Bipartite Graph for Multivariate Time-series Anomaly Detection

Modeling inter-dependencies between time-series is the key to achieve high performance in anomaly detection for multivariate time-series data. The de-facto solution to model the dependencies is to feed the data into a recurrent neural network (RNN). However, the fully connected network structure underneath the RNN (either GRU or LSTM) assumes a static and complete dependency graph between time-series, which may not hold in many real-world applications. To alleviate this assumption, we propose a dynamic bipartite graph structure to encode the inter-dependencies between time-series. More concretely, we model time series as one type of nodes, and the time series segments (regarded as event) as another type of nodes, where the edge between two types of nodes describe a temporal pattern occurred on a specific time series at a certain time. Based on this design, relations between time series can be explicitly modelled via dynamic connections to event nodes, and the multivariate time-series anomaly detection problem can be formulated as a self-supervised, edge stream prediction problem in dynamic graphs. We conducted extensive experiments to demonstrate the effectiveness of the design.
Continuous-Time Sequential Recommendation with Temporal Graph Collaborative Transformer

In order to model the evolution of user preference, we should learn user/item embeddings based on time-ordered item purchasing sequences, which is defined as Sequential Recommendation~(SR) problem. Existing methods leverage sequential patterns to model item transitions. However, most of them ignore crucial temporal collaborative signals, which are latent in evolving user-item interactions and coexist with sequential patterns. Therefore, we propose to unify sequential patterns and temporal collaborative signals to improve the quality of recommendation, which is rather challenging. Firstly, it is hard to simultaneously encode sequential patterns and collaborative signals. Secondly, it is non-trivial to express the temporal effects of collaborative signals. Hence, we design a new framework Temporal Graph Sequential Recommender (TGSRec) upon our defined continuous-time bipartite graph. We propose a novel Temporal Collaborative Transformer TCT layer in TGSRec, which advances the self-attention mechanism by adopting a novel collaborative attention. TCT layer can simultaneously capture collaborative signals from both users and items, as well as considering temporal dynamics inside sequential patterns. We propagate the information learned from TCT layer over the temporal graph to unify sequential patterns and temporal collaborative signals. Empirical results on five datasets show that modelname significantly outperforms other baselines, in average up to 22.5% and 22.1% absolute improvements in Recall@10 and MRR, respectively.
ST3D++: Denoised Self-Training for Unsupervised Domain Adaptation on 3D Object Detection

In this paper, we present a self-training method, named ST3D++, with a holistic pseudo label denoising pipeline for unsupervised domain adaptation on 3D object detection. ST3D++ aims at reducing noise in pseudo label generation as well as alleviating the negative impacts of noisy pseudo labels on model training. First, ST3D++ pre-trains the 3D object detector on the labeled source domain with random object scaling (ROS) which is designed to reduce target domain pseudo label noise arising from object scale bias of the source domain. Then, the detector is progressively improved through alternating between generating pseudo labels and training the object detector with pseudo-labeled target domain data. Here, we equip the pseudo label generation process with a hybrid quality-aware triplet memory to improve the quality and stability of generated pseudo labels. Meanwhile, in the model training stage, we propose a source data assisted training strategy and a curriculum data augmentation policy to effectively rectify noisy gradient directions and avoid model over-fitting to noisy pseudo labeled data. These specific designs enable the detector to be trained on meticulously refined pseudo labeled target data with denoised training signals, and thus effectively facilitate adapting an object detector to a target domain without requiring annotations. Finally, our method is assessed on four 3D benchmark datasets (i.e., Waymo, KITTI, Lyft, and nuScenes) for three common categories (i.e., car, pedestrian and bicycle). ST3D++ achieves state-of-the-art performance on all evaluated settings, outperforming the corresponding baseline by a large margin (e.g., 9.6% <inline-formula><tex-math notation="LaTeX">$\sim$</tex-math><alternatives><mml:math><mml:mo>∼</mml:mo></mml:math><inline-graphic xlink:href="qi-ieq1-3216606.gif"/></alternatives></inline-formula> 38.16% on Waymo <inline-formula><tex-math notation="LaTeX">$\rightarrow$</tex-math><alternatives><mml:math><mml:mo>→</mml:mo></mml:math><inline-graphic xlink:href="qi-ieq2-3216606.gif"/></alternatives></inline-formula> KITTI in terms of AP<inline-formula><tex-math notation="LaTeX">$_{\text{3D}}$</tex-math><alternatives><mml:math><mml:msub><mml:mrow/><mml:mtext>3D</mml:mtext></mml:msub></mml:math><inline-graphic xlink:href="qi-ieq3-3216606.gif"/></alternatives></inline-formula>), and even surpasses the fully supervised oracle results on the KITTI 3D object detection benchmark with target prior. Code is available at <uri>https://github.com/CVMI-Lab/ST3D</uri>.
On $p$-adic Simpson and Riemann-Hilbert correspondences in the imperfect residue field case

Let K be a mixed characteristic complete discrete valuation field with residue field admitting a finite p-basis, and let GK be the Galois group. Inspired by Liu and Zhu’s construction of p-adic Simpson and Riemann-Hilbert correspondences over rigid analytic varieties, we construct such correspondences for representations of GK . As an application, we prove a Hodge-Tate (resp. de Rham) “rigidity” theorem for p-adic representations of GK , generalizing a result of Morita.
Complex Knowledge Base Question Answering: A Survey

Knowledge base question answering (KBQA) aims to answer a question over a knowledge base (KB). Early studies mainly focused on answering simple questions over KBs and achieved great success. However, their performances on complex questions are still far from satisfactory. Therefore, in recent years, researchers propose a large number of novel methods, which looked into the challenges of answering complex questions. In this survey, we review recent advances in KBQA with the focus on solving complex questions, which usually contain multiple subjects, express compound relations, or involve numerical operations. In detail, we begin with introducing the complex KBQA task and relevant background. Then, we present two mainstream categories of methods for complex KBQA, namely semantic parsing-based (SP-based) methods and information retrieval-based (IR-based) methods. Specifically, we illustrate their procedures with flow designs and discuss their difference and similarity. Next, we summarize the challenges that these two categories of methods encounter when answering complex questions, and explicate advanced solutions as well as techniques used in existing work. After that, we discuss the potential impact of pre-trained language models (PLMs) on complex KBQA. To help readers catch up with SOTA methods, we also provide a comprehensive evaluation and resource about complex KBQA task. Finally, we conclude and discuss several promising directions related to complex KBQA for future research.
Interpolation-Aware Padding for 3D Sparse Convolutional Neural Networks

Sparse voxel-based 3D convolutional neural networks (CNNs) are widely used for various 3D vision tasks. Sparse voxel-based 3D CNNs create sparse non-empty voxels from the 3D input and perform 3D convolution operations on them only. We propose a simple yet effective padding scheme — interpolation-aware padding to pad a few empty voxels adjacent to the non-empty voxels and involve them in the 3D CNN computation so that all neighboring voxels exist when computing point-wise features via the trilinear interpolation. For fine-grained 3D vision tasks where point-wise features are essential, like semantic segmentation and 3D detection, our network achieves higher prediction accuracy than the existing networks using the nearest neighbor interpolation or the normalized trilinear interpolation with the zero-padding or the octree-padding scheme. Through extensive compar-isons on various 3D segmentation and detection tasks, we demonstrate the superiority of 3D sparse CNNs with our padding scheme in conjunction with feature interpolation.
GeoCLR: Georeference Contrastive Learning for Efficient Seafloor Image Interpretation

This paper describes georeference contrastive learning of visual representation (GeoCLR) for efficient training of deep-learning convolutional neural networks (CNNs). The method leverages georeference information by generating a similar image pair using images taken of nearby locations, and contrasting these with an image pair that is far apart. The underlying assumption is that images gathered within a close distance are more likely to have similar visual appearance, where this can be reasonably satisfied in seafloor robotic imaging applications where image footprints are limited to edge lengths of a few meters and are taken so that they overlap along a vehicle’s trajectory, whereas seafloor substrates and habitats have patch sizes that are far larger. A key advantage of this method is that it is self-supervised and does not require any human input for CNN training. The method is computationally efficient, where results can be generated between dives during multi-day autonomous underwater vehicle (AUV) missions using computational resources that would be accessible during most oceanic field trials. We apply GeoCLR to habitat classification on a dataset that consists of ~86,000 images gathered using an AUV. We demonstrate how the latent representations generated by GeoCLR can be used to efficiently guide human annotation efforts, where the semi-supervised framework improves classification accuracy by an average of 10.2% compared to the state-of-the-art SimCLR using the same CNN and equivalent number of human annotations for training.