Miracle-dz/UniFAR-FTUs huggingface dataset:
Efficient Per-Shot Convex Hull Prediction By Recurrent Learning

Adaptive video streaming relies on the construction of efficient bitrate ladders to deliver the best possible visual quality to viewers under bandwidth constraints. The traditional method of content dependent bitrate ladder selection requires a video shot to be pre-encoded with multiple encoding parameters to find the optimal operating points given by the convex hull of the resulting rate-quality curves. However, this pre-encoding step is equivalent to an exhaustive search process over the space of possible encoding parameters, which causes significant overhead in terms of both computation and time expenditure. To reduce this overhead, we propose a deep learning based method of content aware convex hull prediction. We employ a recurrent convolutional network (RCN) to implicitly analyze the spatiotemporal complexity of video shots in order to predict their convex hulls. A two-step transfer learning scheme is adopted to train our proposed RCN-Hull model, which ensures sufficient content diversity to analyze scene complexity, while also making it possible capture the scene statistics of pristine source videos. Our experimental results reveal that our proposed model yields better approximations of the optimal convex hulls, and offers competitive time savings as compared to existing approaches. On average, the pre-encoding time was reduced by 58.0% by our method, while the average Bjontegaard delta bitrate (BD-rate) of the predicted convex hulls against ground truth was 0.08%, while the mean absolute deviation of the BD-rate distribution was 0.44%
An Open Framework to Model Diffraction by Dynamic Blockers in Millimeter Wave Simulations

The millimeter wave (mmWave) band will be exploited to address the growing demand for high data rates and low latency. The higher frequencies, however, are prone to limitations on the propagation of the signal in the environment. Thus, highly directional beamforming is needed to increase the antenna gain. Another crucial problem of the mmWave frequencies is their vulnerability to blockage by physical obstacles. To this aim, we studied the problem of modeling the impact of second-order effects on mmWave channels, specifically the susceptibility of the mmWave signals to physical blockers. With respect to existing works on this topic, our project focuses on scenarios where mmWaves interact with multiple, dynamic blockers. Our open source software includes diffraction-based blockage models and interfaces directly with an open source Radio Frequency (RF) ray-tracing software.
Quantum Policy Iteration via Amplitude Estimation and Grover Search - Towards Quantum Advantage for Reinforcement Learning

We present a full implementation and simulation of a novel quantum reinforcement learning method. Our work is a detailed and formal proof of concept for how quantum algorithms can be used to solve reinforcement learning problems and shows that, given access to error-free, efficient quantum realizations of the agent and environment, quantum methods can yield provable improvements over classical Monte-Carlo based methods in terms of sample complexity. Our approach shows in detail how to combine amplitude estimation and Grover search into a policy evaluation and improvement scheme. We first develop quantum policy evaluation (QPE) which is quadratically more efficient compared to an analogous classical Monte Carlo estimation and is based on a quantum mechanical realization of a finite Markov decision process (MDP). Building on QPE, we derive a quantum policy iteration that repeatedly improves an initial policy using Grover search until the optimum is reached. Finally, we present an implementation of our algorithm for a two-armed bandit MDP which we then simulate.
Neural Laplace: Learning diverse classes of differential equations in the Laplace domain

Neural Ordinary Differential Equations model dynamical systems with ODE s learned by neural networks. However, ODEs are fundamentally inadequate to model systems with long-range dependencies or discontinuities, which are common in engineering and biological systems. Broader classes of differential equations (DE) have been proposed as remedies, including delay differential equations and integro-differential equations. Furthermore, Neural ODE suffers from numerical instability when modelling stiff ODEs and ODEs with piecewise forcing functions. In this work, we propose Neural Laplace , a uniﬁed framework for learning diverse classes of DEs including all the aforementioned ones. Instead of modelling the dynamics in the time domain, we model it in the Laplace domain, where the history-dependencies and discontinuities in time can be represented as summations of complex exponentials. To make learning more efﬁcient, we use the geometrical stereographic map of a Riemann sphere to induce more smoothness in the Laplace domain. In the experiments, Neural Laplace shows superior performance in modelling and extrapolating the trajectories of diverse classes of DEs, including the ones with complex history dependency and abrupt changes.
Smallset Timelines: A Visual Representation of Data Preprocessing Decisions

Data preprocessing is a crucial stage in the data analysis pipeline, with both technical and social aspects to consider. Yet, the attention it receives is often lacking in research practice and dissemination. We present the Smallset Timeline, a visualisation to help reflect on and communicate data preprocessing decisions. A “Smallset” is a small selection of rows from the original dataset containing instances of dataset alterations. The Timeline is comprised of Smallset snapshots representing different points in the preprocessing stage and captions to describe the alterations visualised at each point. Edits, additions, and deletions to the dataset are highlighted with colour. We develop the R software package, smallsets, that can create Smallset Timelines from R and Python data preprocessing scripts. Constructing the figure asks practitioners to reflect on and revise decisions as necessary, while sharing it aims to make the process accessible to a diverse range of audiences. We present two case studies to illustrate use of the Smallset Timeline for visualising preprocessing decisions. Case studies include software defect data and income survey benchmark data, in which preprocessing affects levels of data loss and group fairness in prediction tasks, respectively. We envision Smallset Timelines as a go-to data provenance tool, enabling better documentation and communication of preprocessing tasks at large.
Stochastic formulation of incompressible fluid flows in wall bounded regions

In this paper we establish a mathematical framework which may be used to design Monte-Carlo simulations for a class of time irreversible dynamic systems, such as incompressible ﬂuid ﬂows, including turbulent ﬂows in wall-bounded regions, and some other (non-linear) dynamic systems. Path integral representations for solutions of forward parabolic equations are obtained, and, in combining with the vorticity transport equations, probabilistic formulations for solutions of the Navier-Stokes equations are therefore derived in terms of (forward) McKean-Vlasov stochastic differential equations (SDEs), which provides us with the mathematical framework for Monte-Carlo simulations of wall-bounded turbulent ﬂows.
COSTA: Covariance-Preserving Feature Augmentation for Graph Contrastive Learning

Graph contrastive learning (GCL) improves graph representation learning, leading to SOTA on various downstream tasks. The graph augmentation step is a vital but scarcely studied step of GCL. In this paper, we show that the node embedding obtained via the graph augmentations is highly biased, somewhat limiting contrastive models from learning discriminative features for downstream tasks.Thus, instead of investigating graph augmentation in the input space, we alternatively propose to perform augmentations on the hidden features (feature augmentation). Inspired by so-called matrix sketching, we propose COSTA, a novel Covariance-preServing feaTure space Augmentation framework for GCL, which generates augmented features by maintaining a "good sketch" of original features. To highlight the superiority of feature augmentation with COSTA, we investigate a single-view setting (in addition to multi-view one) which conserves memory and computations. We show that the feature augmentation with COSTA achieves comparable/better results than graph augmentation based models.
Adversarial Counterfactual Environment Model Learning

A good model for action-effect prediction, named environment model, is important to achieve sample-efficient decision-making policy learning in many domains like robot control, recommender systems, and patients’ treatment selection. We can take unlimited trials with such a model to identify the appropriate actions so that the costs of queries in the real world can be saved. It requires the model to correctly handle unseen data, also called counterfactual data. However, standard data fitting techniques do not automatically achieve such generalization ability and commonly result in unreliable models. In this work, we introduce counterfactual-query risk minimization (CQRM) in model learning for generalizing to a counterfactual dataset queried by a specific target policy. Since the target policies can be various and unknown in policy learning, we propose an adversarial CQRM objective in which the model learns on counterfactual data queried by adversarial policies, and finally derive a tractable solution GALILEO. We also discover that adversarial CQRM is closely related to the adversarial model learning, explaining the effectiveness of the latter. We apply GALILEO in synthetic tasks and a real-world application. The results show that GALILEO makes accurate predictions on counterfactual data and thus significantly improves policies in real-world testing.
A Fast Spectral Solver for the Heat Equation, with Applications to Navier-Stokes

We develop a spectral method to solve the heat equation in a closed cylinder, achieving a near-optimal O(N logN) complexity and high-order, spectral accuracy. The algorithm relies on a novel Chebyshev–Chebyshev–Fourier (CCF) discretization of the cylinder, which is easily implemented and decouples the heat equation into a collection of smaller, sparse Sylvester equations. In turn, each of these equations is solved using the alternating direction implicit (ADI) method, which improves the complexity of each solve from cubic in the matrix size (in more traditional methods) to log-linear; overall, this represents an improvement in the heat equation solver from O(N7/3) (in traditional methods) to O(N logN). Lastly, we provide numerical simulations demonstrating significant speed-ups over traditional spectral collocation methods and finite difference methods, and we provide a framework by which this heat equation solver could be applied to the incompressible Navier–Stokes equations. For the latter, we decompose the equations using a poloidal–toroidal (PT) decomposition, turning them into heat equations with nonlinear forcing from the advection term; by using implicit–explicit methods to integrate these, we can achieve the same O(N logN) complexity and spectral accuracy achieved here in the heat equation. ∗Massachusetts Institute of Technology, Department of Mathematics †Cornell University, Department of Mathematics ‡Boise State University, Department of Mathematics 1 ar X iv :2 20 6. 04 82 7v 2 [ m at h. N A ] 2 0 Ju n 20 22
Defending Compositionality in Emergent Languages

Compositionality has traditionally been understood as a major factor in productivity of language and, more broadly, human cognition. Yet, recently some research started to question its status showing that artificial neural networks are good at generalization even without noticeable compositional behavior. We argue some of these conclusions are too strong and/or incomplete. In the context of a two-agent communication game, we show that compositionality indeed seems essential for successful generalization when the evaluation is done on a suitable dataset.
Experimental evaluation of Visual‐Inertial Odometry systems for arable farming

The farming industry constantly seeks the automation of different processes involved in agricultural production, such as sowing, harvesting and weed control. The use of mobile autonomous robots to perform those tasks is of great interest. Arable lands present hard challenges for Simultaneous Localization and Mapping (SLAM) systems, key for mobile robotics, given the visual difficulty due to the highly repetitive scene and the crop leaves movement caused by the wind. In recent years, several Visual‐Inertial Odometry (VIO) and SLAM systems have been developed. They have proved to be robust and capable of achieving high accuracy in indoor and outdoor urban environments. However, they were not properly assessed in agricultural fields. In this study we assess the most relevant state‐of‐the‐art VIO systems in terms of accuracy and processing time on arable lands to better understand how they behave on these environments. In particular, the evaluation is carried out on a collection of sensor data recorded by our wheeled robot in a soybean field, which was publicly released as the Rosario data set. The evaluation shows that the highly repetitive appearance of the environment, the strong vibration produced by the rough terrain and the movement of the leaves caused by the wind, expose the limitations of the current state‐of‐the‐art VIO and SLAM systems. We analyze the systems failures and highlight the observed drawbacks, including initialization failures, tracking loss and sensitivity to Inertial Measurement Unit saturation. Finally, we conclude that even though certain systems like ORB‐SLAM3 and stereo Multi‐State Constraint Kalman Filter show good results with respect to others, more improvements should be done to make them reliable in agricultural fields for certain applications such as soil tillage of crop rows and pesticide spraying.
Boosting unstable particles

In relativity, there is no absolute notion of simultaneity, because two clocks that are in different places can always be desynchronized by a Lorentz boost. Here, we explore the implications of this effect for the quantum theory of unstable particles. We show that, when a wavefunction is boosted, its tails travel one to the past and the other to the future. As a consequence, in the new frame of reference, the particle is in a quantum superposition"decayed + non decayed", where the property"decayed-ness"is entangled with the position. Since a particle cannot be localised in a region smaller than the Compton wavelength, there is a non-zero lower bound on this effect, which is fundamental in nature. The surprising implication is that, in a quantum world, decay probabilities can never be Lorentz-invariant. We show that this insight was the missing ingredient to reconcile the seemingly conflicting views about time dilation in relativistic quantum mechanics and quantum field theory.
Optical Diffraction Tomography based on 3D Physics-Inspired Neural Network (PINN)

Abstract: Optical diffraction tomography (ODT) is an emerging 3D imaging technique that is used for the 3D reconstruction of the refractive index (RI) for semi-transparent samples. Various inverse models have been proposed to reconstruct the 3D RI based on the holographic detection of different samples such as the Born and the Rytov approximations. However, such approximations usually suffer from the so-called “missing-cone” problem that results in an elongation of the final reconstruction along the optical axis. Different iterative schemes have been proposed to solve the missing cone problem relying on physical forward models and an error function that aims at filling in the k-space and thus eliminating the “missingcone” problem and reaching better reconstruction accuracy. In this paper, we propose a different approach where a 3D neural network (NN) is employed. The NN is trained with a cost function derived from a physical model based on the physics of optical wave propagation. The 3D NN starts with an initial guess for the 3D RI reconstruction (i.e. Born, or Rytov) and aims at reconstructing better 3D reconstruction based on an error function. With this technique, the NN can be trained without any examples of the relation between the ill-posed reconstruction (Born or Rytov) and the ground truth (true shape).
Grain Boundary Development of Silicon during Directional Solidification: A Phase-Field Study

In order to control the grain structure of multi-crystalline (mc) silicon during directional solidification, the development process of grain boundaries (GBs) with respect to the temperature gradient should be understood. A phase-field model incorporated with anisotropic interface energy and anisotropic attachment kinetic coefficient has produced the faceted shape of a growing silicon crystal, which is in agreement with experimental observation. The growth of coupled silicon grains under various growth velocities has been simulated to see the morphology of the solid-liquid front and the development process of the GBs. It has been found that the direction of GB is governed by either the kinetic rule or the equilibrium rule at the grain groove, depending on the growth velocity and the orientation relationship between grains on two sides. The GB beneath a groove with facet-facet surfaces follows the bisector of the two surfaces, while the direction of a GB stays far from the bisector when the groove has a rough surface. This research provides a numerical approach to predicting grain boundary development and gaining insights from grain structure evolution in mc-silicon, which can be potentially applied for high-efficiency and low-cost solar cells.
A Novel Partitioned Approach for Reduced Order Model - Finite Element Model (ROM-FEM) and ROM-ROM Coupling

Partitioned methods allow one to build a simulation capability for coupled problems by reusing existing single-component codes. In so doing, partitioned methods can shorten code development and validation times for multiphysics and multiscale applications. In this work, we consider a scenario in which one or more of the"codes"being coupled are projection-based reduced order models (ROMs), introduced to lower the computational cost associated with a particular component. We simulate this scenario by considering a model interface problem that is discretized independently on two non-overlapping subdomains. We then formulate a partitioned scheme for this problem that allows the coupling between a ROM"code"for one of the subdomains with a finite element model (FEM) or ROM"code"for the other subdomain. The ROM"codes"are constructed by performing proper orthogonal decomposition (POD) on a snapshot ensemble to obtain a low-dimensional reduced order basis, followed by a Galerkin projection onto this basis. The ROM and/or FEM"codes"on each subdomain are then coupled using a Lagrange multiplier representing the interface flux. To partition the resulting monolithic problem, we first eliminate the flux through a dual Schur complement. Application of an explicit time integration scheme to the transformed monolithic problem decouples the subdomain equations, allowing their independent solution for the next time step. We show numerical results that demonstrate the proposed method's efficacy in achieving both ROM-FEM and ROM-ROM coupling.
Fitting Covariance Matrices Models to Simulations

Data analysis in cosmology requires reliable covariance matrices. Covariance matrices derived from numerical simulations often require a very large number of realizations to be accurate. When a theoretical model for the covariance matrix exists, the parameters of the model can often be fit with many fewer simulations. We establish a rigorous Bayesian method for performing such a fit, but show that using the maximum posterior point is often sufficient. We show how a model covariance matrix can be tested by examining the appropriate χ2 distributions from simulations. We demonstrate our method on two examples. First, we measure the two-point correlation function of halos from a large set of 10000 mock halo catalogs. We build a model covariance with 2 free parameters, which we fit using our procedure. The resulting best-fit model covariance obtained from just 100 simulation realizations proves to be as reliable as the numerical covariance matrix built from the full 10000 set. We also test our method on a setup where the covariance matrix is large by measuring the halo bispectrum for thousands of triangles for the same set of mocks. We build a block diagonal model covariance with 2 free parameters as an improvement over the diagonal Gaussian covariance. Our model covariance passes the χ2 test only partially in this case, signaling that the model is insufficient even using free parameters, but significantly improves over the Gaussian one.
Extremal Fitting Problems for Conjunctive Queries

The fitting problem for conjunctive queries (CQs) is the problem to construct a CQ that fits a given set of labeled data examples. When a fitting CQ exists, it is in general not unique. This leads us to proposing natural refinements of the notion of a fitting CQ, such as most-general fitting CQ, most-specific fitting CQ, and unique fitting CQ. We give structural characterizations of these notions in terms of (suitable refinements of) homomorphism dualities, frontiers, and direct products, which enable the construction of the refined fitting CQs when they exist. We also pinpoint the complexity of the associated existence and verification problems, and determine the size of fitting CQs. We study the same problems for UCQs and for the more restricted class of tree CQs.
Merak: An Efficient Distributed DNN Training Framework With Automated 3D Parallelism for Giant Foundation Models

Foundation models are in the process of becoming the dominant deep learning technology. Pretraining a foundation model is always time-consuming due to the large scale of both the model parameter and training dataset. Besides being computing-intensive, the pretraining process is extremely memory- and communication-intensive. These challenges make it necessary to apply 3D parallelism, which integrates data parallelism, pipeline model parallelism, and tensor model parallelism, to achieve high training efficiency. However, current 3D parallelism frameworks still encounter two issues: i) they are not transparent to model developers, requiring manual model modification to parallelize training, and ii) their utilization of computation resources, GPU memory, and network bandwidth is insufficient. We propose Merak, an automated 3D parallelism deep learning training framework with high resource utilization. Merak automatically deploys 3D parallelism with an automatic model partitioner, which includes a graph-sharding algorithm and proxy node-based model graph. Merak also offers a non-intrusive API to scale out foundation model training with minimal code modification. In addition, we design a high-performance 3D parallel runtime engine that employs several techniques to exploit available training resources, including a shifted critical path pipeline schedule that increases computation utilization, stage-aware recomputation that makes use of idle worker memory, and sub-pipelined tensor model parallelism that overlaps communication and computation. Experiments on 64 GPUs demonstrate Merak's capability to speed up training performance over state-of-the-art 3D parallelism frameworks of models with 1.5, 2.5, 8.3, and 20 billion parameters by up to 1.42, 1.39, 1.43, and 1.61×, respectively.