A finite-volume, incompressible Navier Stokes model for studies of the ocean on parallel computers

The numerical implementation of an ocean model based on the incompressible Navier Stokes equations which is designed for studies of the ocean circulation on horizontal scales less than the depth of the ocean right up to global scale is described. A "pressure correction" method is used which is solved as a Poisson equation for the pressure field with Neumann boundary conditions in a geometry as complicated as that of the ocean basins. A major objective of the study is to make this inversion, and hence nonhydrostatic ocean modeling, efficient on parallel computers. The pressure field is separated into surface, hydrostatic, and nonhydrostatic components. First, as in hydrostatic models, a two-dimensional problem is inverted for the surface pressure which is then made use of in the three-dimensional inversion for the nonhydrostatic pressure. Preconditioned conjugate-gradient iteration is used to invert symmetric elliptic operators in both two and three dimensions. Physically motivated preconditioners are designed which are efficient at reducing computation and minimizing communication between processors. Our method exploits the fact that as the horizontal scale of the motion becomes very much larger than the vertical scale, the motion becomes more and more hydrostatic and the three- dimensional Poisson operator becomes increasingly anisotropic and dominated by the vertical axis. Accordingly, a preconditioner is used which, in the hydrostatic limit, is an exact integral of the Poisson operator and so leads to a single algorithm that seamlessly moves from nonhydrostatic to hydrostatic limits. Thus in the hydrostatic limit the model is "fast," competitive with the fastest ocean climate models in use today based on the hydrostatic primitive equations. But as the resolution is increased, the model dynamics asymptote smoothly to the Navier Stokes equations and so can be used to address small- scale processes. A "finite-volume" approach is employed to discretize the model in space in which property fluxes are defined normal to faces that delineate the volumes. The method makes possible a novel treatment of the boundary in which cells abutting the bottom or coast may take on irregular shapes and be "shaved" to fit the boundary. The algorithm can conveniently exploit massively parallel computers and suggests a domain decomposition which allocates vertical columns of ocean to each processing unit. The resulting model, which can handle arbitrarily complex geometry, is efficient and scalable and has been mapped on to massively parallel multiprocessors such as the Connection Machine (CM5) using data-parallel FORTRAN and the Massachusetts Institute of Technology data-flow machine MONSOON using the implicitly parallel language Id. Details of the numerical implementation of a model which has been designed for the study of dynamical processes in the ocean from the convective, through the geostrophic eddy, up to global scale are set out. The "kernel" algorithm solves the incompressible Navier Stokes equations on the sphere, in a geometry as complicated as that of the ocean basins with ir- regular coastlines and islands. (Here we use the term "Navier Stokes" to signify that the full nonhydrostatic equations are being employed; it does not imply a particular constitutive relation. The relevant equations for modeling the full complex- ity of the ocean include, as here, active tracers such as tem- perature and salt.) It builds on ideas developed in the compu- tational fluid community. The numerical challenge is to ensure that the evolving velocity field remains nondivergent. Most