BALLAST: Bayesian Active Learning with Look-ahead Amendment for Sea-drifter Trajectories

Authors: rui-yang  zhang
henry b moss
lachlan c astfalck
edward j cripps
david s leslie
Venue: NeurIPS_cc_2024_Workshop_BDU
Published: 2024-08-29 13:39:24

Abstract: We introduce a formal experimental design methodology for guiding the placement of drifters to infer ocean currents. The majority of drifter placement campaigns either follow standard `space-filling' designs or relatively ad-hoc expert opinions, which could be significantly improved on by appealing to statistical experimental design. Drifter observations follow a Lagrangian structure as drifters are advected through the Eulerian vector field. It is, therefore, important to consider the likely future trajectories of placed drifters to compute the utility of candidate measurement locations. We present BALLAST: Bayesian Active Learning with Look-ahead Amendment for Sea-drifter Trajectories, a novel light-weight, nonmyopic policy amendment for drifter trajectory data that can be appended to any myopic policy. Our numerical studies suggest benefits in incorporating BALLAST into the sequential placement strategies of drifters.

Main paper: 
 introduction: 
 Understanding and predicting ocean currents are of vital importance to mapping the flow of heat, nutrients, pollutants and sediments in the ocean [3, 9] . Ocean currents are inferred from a plurality of measurement devices, such as fixed-location buoys, satellites and free-floating buoys, known as drifters [11] . Free-floating drifters are being increasingly used due to their ability to sample both spatial and temporal flow properties and remain relatively affordable as compared to other measurement devices [15] .Standard experiment designs consider measurement devices that operate at a single location at a single time. Drifters are different: advected by the background vector field, drifters proceed to move following the Lagrangian flow of the ocean currents and collect data at multiple locations at multiple times (see the orange observation trajectories and the extended red observations in Figure 1 ). Drifter measurements thus require nontrivial policies for experiment designs and sequential experiment designs (SED) where observations are made incrementally -allowing efficient dynamic experimentation by adapting to the results of previous measurements. However, the majority of existing drifter placement campaigns either follow standard 'space-filling' designs [19] or relatively ad-hoc expert opinions [21] . To the best of our knowledge, methodology for a formal statistical design of drifter allocation strategies does not exist.Standard (S)ED policies only consider the observation made immediately at the selected observation point, making them myopic [4, 16] and are likely insufficient to guide the temporally extended Lagrangian observations associated with drifter placement designs. Policies that take into account potential future observations, often called nonmyopic policies, exist in the literature [8, 7, 23] , but they focus on solving or approximating the Bellman equation considering the next several observations actively selected by the decision-maker (see [5] for a discussion). A practical sequential placement strategy for drifters should incorporate potential future trajectories while remaining computationally light-weight.In this work, we introduce an experimental design strategy for drifters -Bayesian Active Learning with Look-ahead Amendment for Sea-drifter Trajectories (BALLAST) -that offers a light-weight, nonmyopic policy amendment by leveraging a vector-output Gaussian process model and the drifters' Lagrangian trajectory structure. We make three primary contributions: (i) we introduce sequential experimental design concepts to the problem of drifter placement, (ii) we propose a novel light-weight, nonmyopic SED policy amendment that can be easily appended to most existing myopic SED policies like maximum variance and maximum integrated variance [6] , and (iii) we present the results of numerical simulations that showcase improved performance of our proposed policy over common alternatives. 
background: 
 Gaussian Processes A vector-output Gaussian process model is used here to model the 2dimensional, stationary vector field that represents the ocean currents over a region. A vector-output Gaussian process (GP) {y(x)} x∈X with input space X , output space R D with D > 1, R D -valued mean function µ and R D×D -valued covariance function k is a continuous stochastic process fully denoted as y(•) ∼ GP(µ(•), k(•, •)) such that every finite collection of random variables in {y(x)} x∈X has a multivariate normal distribution [22, 1] . Normally, the mean function µ is set to be a constantzero function, which will be the case here too. Details about regressions and predictions with a vector-output GP can be found in Appendix A.Experimental Design Experimental design considers the problem of choosing the optimal measurement x * given some existing data D and a model g according to some utility function U that calculates the expected quality of each possible measurement x within the measurement set R under the belief of our latest model g. Mathematically, this is given by x * = argmax x∈R E g [U (x, g, D)]. When we wish to make measurements sequentially, or adaptively, we will find an optimal measurement x * n+1 for each time n based on existing data D n and latest model g n according tox * n+1 = argmax x∈R E gn [U (x, g n , D n )].In this work, we will use a Gaussian process as our model due to its expressibility, conjugacy with data updates and inherent uncertainty quantification. The choice of utility functions is central to the effectiveness of an (S)ED. Two common choices of utility functions are the variance of our latest model g n at the candidate decision x (yielding the maximum variance policy), and the total variance of the posterior distribution of our model after being updated by the candidate decision x and its outcome y (yielding the maximum integrated variance policy). See [6, 18] for comprehensive introductions to these designs. 
sequential drifter placement and ballast: 
 The problem of sequential drifter placement can be formulated as follows. Consider a fixed region R ⊂ R 2 of the ocean that we wish to understand its currents. Let F denote the true stationary vector field of the region R. We will place drifters sequentially into the region and use the observations to infer the true field F , and the goal is to maximally reduce our uncertainty about F using a minimum number of drifters.Assuming n drifters are currently deployed, we have existing data D n and latest model g n . With a utility function choice U , the myopic allocation policy is to only consider the initial placement point from R, selected byEQUATIONRecall that drifters are free-floating and will keep collecting data at different locations once placed. A myopic policy does not take into account the future trajectory of a drifter placed at x and could favour locations that are apparently more promising yet uninformative overall -say the placed drifter leaves R immediately or is stuck in a convergent region and so is, in effect, stationary. With myopic policies, the Lagrangian structure of the drifter observations is completely ignored.We propose the BALLAST method, a nonmyopic policy amendment with look-aheads of drifter trajectories that is computationally expedient. While still considering only the initial placement point x ∈ R, we construct the look-ahead of a drifter placed at x via a projection of T steps with step size δ using a vector field F , denoted by P T δ (x, F ) ∈ R T +1 . Given n existing drifters with data D n and model g n , the BALLAST method determines the next placement location byEQUATIONWe will denote the conversion from Equation (1) to Equation (2) as the BALLAST amendment to the base myopic policy. Various design choices of this amendment are discussed below.The trajectory of a drifter at x inside the vector field F can be approximated using numerical ordinary differential equation solvers. The simplest choice is the Euler method, which iteratively updates each step by x (m+1)δ = x mδ +δ F (x mδ ) for m ∈ Z ≥0 where F (x) denotes the value of the field F at x. Other solvers such as the implicit Euler method can also be used [17] .Step Size and Number Caution is required when setting the step size δ and the step number T to balance the trade-off between practicality and accuracy. A tiny step size δ may induce unwanted computational costs, yet too large a step size will produce poor approximations of the drifter dynamics and yield inaccurate predictions. For step number T , a small value will make the policy more myopic, which we wish to avoid, yet a large value will incur additional computational costs. Furthermore, the look-ahead is based on an estimated vector field F , which could be erroneous, so too long of a look-ahead might be misleading.Estimated Vector Field Choice of the estimated vector field reflects our latest understanding of currents in the region R. As alluded to previously, the quality of the estimated vector field strongly influences the projection quality. Here, we propose two ways to construct the estimated field which induce two versions of BALLAST. The first way is to choose, naively, the predictive mean µ Dn of the latest GP model g n , that leads to the policyEQUATIONwhich we call BALLASTn. The second way is to draw K independent sample fields { Fk } K k=1 from the predictive distribution of the latest model g n and estimate the utility using the sample averages. Denote this second method by BALLASTs, with Fk ∼ g n for k = 1, 2, . . . , K, we haveEQUATIONOne should note that BALLASTs and BALLASTn are different. For BALLASTs, when we take the limit k → ∞, we get1 K K k=1 U (P T δ (x, Fk ), g n , D n ) → E F ∼gn U (P T δ (x, F ), g n , D n )whereas for BALLASTn we haveU (P T δ (x, E F ∼gn [ F ]), g n , D n )instead. Since U • P T δ is not a linear function in the field choice, the two quantities are not identical, showing that BALLASTs and BALLASTn are indeed different.Utility Function It is obvious from the formulation that the BALLAST methods are compatible with any utility function U . With the look-ahead amendment provided by BALLAST methods, we have observed noticeable improvements from utility choices such as maximum variance and maximum integrated variance, as supported by the simulations in the following section. 
simulation results: 
 We generate a ground truth vector field F from a vector-valued GP parameterised by the Helmholtz kernel of [2] with prespecified hyperparameters. A total of 8 drifters are sequentially placed with T × δ = 5 × 0.03 unit time apart, and the observations provided by the drifters follow the ground truth field F with small additive observation noise. When a drifter leaves the region R, we terminate its motion. Eight placement policies are considered: uniform sampling, Sobol sequence [10] , maximum variance, maximum variance with BALLASTn, maximum variance with BALLASTs using 100 sampled look-aheads, maximum integrated variance, maximum integrated variance with BALLASTn, and maximum integrated variance with BALLASTs using 100 sampled look-aheads. As demonstrated in Figure 1 , we update the model and compute the mean squared error (MSE) of the predictive mean with each placement. Further details of the experiment are found in Appendix B.Figure 2 shows experimental results over 100 random starting seeds and indicates improvements, regardless of the base utility functions, of the BALLAST amendments. The improvement is more significant as more drifters are placed. Furthermore, a simple policy like maximum variance after BALLAST amendments displays comparable performance to a more complicated myopic policy, the maximum integrated variance.We have not carried out a thorough computational cost comparison, but for indicative timings using a standard laptop, one full run of the simulation with the maximum variance policy takes 15 seconds, while that with the BALLASTn takes around 20 seconds and that with the BALLASTs using 100 sampled look-aheads under simple parallelization of the samples takes around 1 minute 15 seconds. For operational drifter placement, these computational costs are likely to be insignificant. 
conclusion: 
 In this paper, we introduce a formal experimental design of the sequential placement of drifters for ocean currents inference. Our numerical results showed that SED-based policies noticeably outperformed common ad-hoc or space-filling drifter allocation strategies, which are further outperformed by our proposed light-weight and nonmyopic BALLAST extensions. Potential further work includes considering time-varying ocean flows, for which more efficient sampling methods will be investigated [20] , establishing theoretical guarantees for the non-myopic benefits of BALLAST, and incorporating BALLAST into a wider risk-averse [14] framework or incorporating information-theoretic framework for batch [12] or multi-fidelity designs [13] . 
 
 Appendix: For a D-vector-output GP model y ∼ GP(µ, k β ) with kernel specified by hyperparameters β, N observations D = X ∪ Y = {x i } N i=1 ∪ {y i } N i=1 for input x i and noisy output y i = y(x i ) + ε, ε ∼ N D (0, S) with S = σ 2 I D , if we let θ = (β, σ) denote all the unknown parameters of y, then the log-likelihood is given byl(θ|D) = - 1 2 y T (K β (X, X) + Σ) -1 y - 1 2 log |K β (X, X) + Σ| - N D 2 log 2πwhere y ∈ R N D is the concatenation of observations Furthermore, the posterior predictive distribution y(x * )|D of a test point x * ∈ X after observing D is provided by{y i } N i=1 , K β (X, X) ∈ R N D×N D with K β (X, X) =     K 1,1 (X, X) K 1,2 (X, X) . . . K 1,D (X, X) K 2,1 (X, X) K 2,2 (X, X) . . . K 2,D (X, X) . . . . . . . . . . . . K D,1 (X, X) K D,2 (X, X) . . . K D,D (X, X)     and K d,d ′ (X, X) = {k β (x i , x j )} N i,y(x * )|y, x * , X, y, θ ∼ N (µ * (x * ), K * (x * , x * )) µ * (x * ) = K β (x * , X)[K β (X, X) + Σ] -1 y K * (x * , x * ) = K β (x * , x * ) -K β (x * , X)[K β (X, X) + Σ] -1 K β (x * , X) Twhere K β (x * , X) ∈ R D×N D is the concatenation of D × D matrix K β (x * , x j ) where j = 1, 2, . . . , N .The vector-output kernel of choice for this paper is the Helmholtz kernel of [2] , which is a recently proposed model that can better capture the vector field structure using the Helmholtz decomposition. Denoting the 2D vector field by F = [u, v] T with Helmholtz decomposition F = gradΦ + rotΨ for the potential function Φ and the stream function Ψ, the Helmholtz kernel imposes a GP prior to each of the two functions, i.e. Φ ∼ GP(0, k Φ ), Ψ ∼ GP(0, k Ψ ), which gives the overall kernel F ∼ GP(0, k Helm ) with the kernel functionk Helm (x, x ′ ) = ∂ 2 x1x ′ 1 k Φ (x, x ′ ) + ∂ 2 x2x ′ 2 k Ψ (x, x ′ ) ∂ 2 x1x ′ 2 k Φ (x, x ′ ) -∂ 2 x2x ′ 1 k Ψ (x, x ′ ) ∂ 2 x2x ′ 1 k Φ (x, x ′ ) -∂ 2 x1x ′ 2 k Ψ (x, x ′ ) ∂ 2 x2x ′ 2 k Φ (x, x ′ ) + ∂ 2 x1x ′ 1 k Ψ (x, x ′ ) .for x, x ′ ∈ R 2 if we assume Φ and Ψ are independent. The two kernels of Φ, Ψ are chosen to be squared exponential kernels with respective lengthscales l Φ , l Ψ .B Additional Details and Plots of the Experiment The ground truth vector field F is generated using the Helmholtz kernel of [2] , further explained in Appendix A.1, with known hyperparameters (lengthscales l Φ = 0.7, l Ψ = 0.5 for k Φ , k Ψ respectively) over the region R = [0, 2] 2 and the square 25 × 25 grid G. A total of 8 drifters are sequentially placed, T δ unit time apart, with T = 5 and δ = 0.03, and the observations provided by the drifters follow the ground truth field F via an Euler discretization of step size δ = 0.03 and additive centered Gaussian observation noise with variance 0.01. The velocity of the drifter is taken to be the value of the vector field at the nearest block of the current position. When a drifter leaves the region R, we will terminate its motion. Eight placement policies are considered: uniform sampling, Sobol sequence, maximum variance, maximum variance with BALLASTs, maximum variance with BALLASTn using 100 samples, integrated variance, integrated variance with BALLASTs, and integrated variance with BALLASTs using 100 samples. The first drifters under all policies, except for the case of the Sobol sequence, are placed at a uniformly drawn location of G.The model we use is the Helmholtz kernel with the true hyperparameters. With each placement, we update the model with all existing data and compute the mean squared error (MSE) of the predictive mean against the truth F as well as the average standard deviation of the predictive distribution. A total of 100 iterations with different initial seeds are done for all eight policies, and the results are summarised in Tables 1 and 2 . We have also produced the mean average standard deviation summary and rank plots in Figure 3 , similar to those in Figure 2 for MSE. We can notice very similar, positive results of the BALLAST methods. 


Decision: Reviews for this paper are borderline, but their actual text is more positive than the scores. Reviewers praise the application, but complain about comprehensiveness of experiments. Given this is a workshop paper, I think these complains, while valid, are not sufficient to warrant rejection. I therefore opt to place this work on the "accept" side of the borderline paper threshold.

Reviews: review: title: title: The authors propose a new Bayesian active learning algorithm to determine the optimal placement of drifters for measuring ocean currents. The effectiveness of the proposed methodology is demonstrated in a simulated experiment.
main_review: main_review: ## Strengths
- The use of Bayesian experimental design for the problem of drifter placement appears to be novel.
- The paper is well-written and easy to follow.

---
## Weaknesses
- A true non-myopic policy must consider the placement of future drifters and not just the trajectory followed by the next drifter. The authors address this difference in the following sentence in the introduction:
> A suitable policy for drifter placement should be non-myopic, in the sense that future observations from stochastic projections of drifter trajectories are considered, even if they do not go to the extent of solving the Bellman equation to consider future drifter placement.

The latter part of this sentence is unconvincing. Recent works (see, for e.g., [1]) have shown that fully non-myopic policies significantly outperform myopic ones, and the authors provide no discussion or reasoning behind their choice in this paper. I'm assuming it is for computational reasons, but this point has to be elucidated further.

- A comparison of BALLASTn and BALLASTs is missing in the text. What is the motivation behind introducing BALLASTs? The sample average is just an unbiased estimator of the predictive mean.
- I did not find that the claim
> Our numerical studies suggest substantial benefits in incorporating BALLAST into the sequential placement strategies of drifters.

in the abstract is sufficiently backed up by the simulation results. For example, in the plot on the left in Figure 2, using the maximum integrated variance utility function seems to yield more or less identical results regardless of whether or not BALLAST is used. The plot on the right only shows marginal improvement, and that too only for BALLASTs. Why would BALLASTs outperform BALLASTn? The authors do not provide adequate insights into the results of the experiments. In short, I feel that the numerical validation in the paper is insufficient to claim any substantial benefit for using BALLAST, and further simulations are necessary to make conclusions.

---
## Minor comments
- Figure 2 is difficult to read (too much going on). A table with the same information could help the reader compare methods more easily.
- The following sentence starting on line 26, which makes a claim about existing work, is lacking citations.
> However, the majority of existing drifter placement campaigns either follow standard ‘space-filling’ designs or relatively ad-hoc expert opinions.

---
## References
[1] Foster, A., Ivanova, D. R., Malik, I., and Rainforth, T. Deep adaptive design: Amortizing sequential Bayesian experimental design. In International Conference on Machine Learning, 2021.
title: title: Review on "BALLAST: Bayesian Active Learning with Look-ahead Amendment for Sea-drifter Trajectories"
main_review: main_review: The paper presents a formal experimental design to optimize the placement of sea drifters used for measuring and inferring ocean currents. The main contribution of this paper is a lightweight Bayesian Active Learning-based drifter experimental design strategy, and the authors claim they are the first to give a formal statistical design of drifter allocation strategies. The numerical evaluation demonstrates the effectiveness of their methods.

Strength:
[1] The authors claim they are the first ones to give a formal statistical design of drifter allocation strategies.

[2] The results demonstrate the advantage of their methods over baselines in terms o MSE.

Weakness:

[1] The related works are lacking in the Introduction part. The authors should provide enough evidence to prove the exclusiveness of their work. For example, in the second paragraph of the Introduction, the authors mention the "space-filling" and relatively ad-hoc expert opinions, but they don't cite any existing references.

[2] Will the change of the first deployment of the drifter affect the final utility? The authors didn't mention how to optimally determine the first drifter placement and its effect on BALLAST.

[3] The experiments should be expanded to explore more factors (e.g., size of areas) on different methods.