OCS Study BOEM 2018-007

Technical Manual for a Coupled Sea-Ice/Ocean

Circulation Model (Version 5)

Katherine S. Hedström

Arctic Region Supercomputing Center

University of Alaska Fairbanks

U.S. Department of the Interior

Bureau of Ocean Energy Management

Alaska OCS Region

Anchorage, Alaska

BOEM Cooperative Agreement M15AC00011
OCS Study BOEM 2018-007

Technical Manual for a Coupled Sea-Ice/Ocean

Circulation Model (Version 5)

Katherine S. Hedström

Arctic Region Supercomputing Center

University of Alaska Fairbanks

June 2018

This study was funded by the Alaska Outer Continental Shelf Region of the Bureau of Ocean

Energy Management, U.S. Department of the Interior, Herndon, Virginia, through Cooperative

Agreement M15AC00011 with Rutgers University.
DISCLAIMER
 

This report was prepared under agreement between the Bureau of Ocean Energy Management

(BOEM) and Rutgers University. This report has been technically reviewed by BOEM, and it has

been approved for publication. Approval does not signify that the contents necessarily reflect the

views and policies of BOEM, nor does mention of trade names or commercial products constitute

endorsement or recommendation for use. It is, however, exempt from review and compliance with

BOEM editorial standards.

REPORT AVAILABILITY

The report may be downloaded from the boem.gov website at: http://www.boem.gov/ESPIS/ or

http://www.boem.gov/akpubs/.

CITATION

K. Hedstrom. 2018. Technical Manual for a Coupled Sea-Ice/Ocean Circulation Model (Version

5). U.S. Dept. of the Interior, Bureau of Ocean Energy Management, Alaska OCS Region. OCS

Study BOEM 2018-007. 182 pp.
This document was prepared with LATEX, xfig, and inkscape.

Acknowledgments

The ROMS model is descended from the SPEM and SCRUM models, but has been entirely

rewritten by Sasha Shchepetkin, Hernan Arango and John Warner, with many, many other contributors. I am indebted to every one of them for their hard work.

Bill Hibler first came up with the viscous-plastic rheology we are using. Paul Budgell has

rewritten the dynamic sea-ice model, improving the solution procedure and making the waterstress term implicit in time, then changing it again to use the elastic-viscous-plastic rheology of

Hunke and Dukowicz. I am very grateful that he is allowing us to use his version of the code. The

sea-ice thermodynamics is derived from Sirpa Häkkinen’s implementation of the Mellor-Kantha

scheme. She was kind enough to allow Paul and I to start with her code.

Thanks to the internet community for providing great tools like Perl, Python, cpp, svn, git,

and gmake to aid in software development (and to make it more fun).

This work was supported in part by a grant of HPC resources from the Arctic Region Supercomputing Center.

Development and testing of the ROMS model has been funded by many, including the USGS

Coastal and Marine Program, the Office of Naval Research, and the National Ocean Partnership

Program.

UNIX is a registered trademark of the Open Group.
 

Cygwin is a registered trademark of Red Hat, Inc.
 
Abstract

The Regional Ocean Modeling System (ROMS), authored by many, most notably Sasha

Shchepetkin, is one approach to regional and basin-scale ocean modeling. This user’s manual for

ROMS describes the model equations and algorithms, as well as additional user configurations

necessary for specific applications. ROMS itself has now branched out as well—the version

described here is that available through the myroms.org svn site with modifications to include

sea ice and other changes. This sea ice branch is available at github.com.
Contents

1 Introduction 1
 

2 Getting started 3
 

2.1 myroms.org . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
 

2.2 Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
 

2.3 Acquiring the ROMS code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
 

2.4 Compiling ROMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
 

2.4.1 Environment Variables for make . . . . . . . . . . . . . . . . . . . . . . . . . 4
 

2.4.2 Providing the Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
 

2.4.3 Build scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
 

2.5 Running ROMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
 

2.6 Warnings and bugs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
 

3 Ocean Model Formulation 9
 

3.1 Equations of motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
 

3.2 Vertical boundary conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
 

3.3 Horizontal boundary conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
 

3.4 Terrain-following coordinate system . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
 

3.5 Horizontal curvilinear coordinates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
 

4 Numerical Solution Technique 14
 

4.1 Vertical and horizontal discretization . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
 

4.1.1 Horizontal grid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
 

4.1.2 Vertical grid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
 

4.2 Masking of land areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
 

4.2.1 Velocity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
 

4.2.2 Temperature, salinity and surface elevation . . . . . . . . . . . . . . . . . . . 15
 

4.2.3 Wetting and drying . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
 

4.3 Time-stepping overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
 

4.4 Conservation properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
 

4.5 Depth-integrated equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
 

4.6 Density in the mode coupling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
 

4.7 Time stepping: internal velocity modes and tracers . . . . . . . . . . . . . . . . . . . 24
 

4.8 Advection schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
 

4.8.1 Second-order Centered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
 

4.8.2 Fourth-order Centered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
 

4.8.3 Fourth-order Akima . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
 

4.8.4 Third-order Upwind . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
 

4.9 Determination of the vertical velocity and density fields . . . . . . . . . . . . . . . . 26
 

4.10 Horizontal mixing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
 

4.10.1 Deviatory stress tensor (viscosity) . . . . . . . . . . . . . . . . . . . . . . . . 27
 

4.10.2 Transverse stress tensor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
 

4.10.3 Rotated Transverse Stress Tensor . . . . . . . . . . . . . . . . . . . . . . . . . 29
 

4.10.4 Horizontal diffusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
 

4.10.5 Laplacian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
 

4.10.6 Biharmonic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
 

4.10.7 Rotated mixing tensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
 

4.11 Vertical mixing schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
 

4.11.1 The Large, McWilliams and Doney parameterization . . . . . . . . . . . . . . 31
 

i
 
4.11.2 Mellor-Yamada . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
 

4.11.3 Generic length scale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
 

4.12 Timestepping vertical viscosity and diffusion . . . . . . . . . . . . . . . . . . . . . . . 37
 

4.13 Lateral Boundary Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
 

4.13.1 Gradient boundary condition . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
 

4.13.2 Wall boundary condition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
 

4.13.3 Clamped boundary condition . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
 

4.13.4 Flather boundary condition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
 

4.13.5 Shchepetkin boundary condition . . . . . . . . . . . . . . . . . . . . . . . . . 39
 

4.13.6 Chapman boundary condition . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
 

4.13.7 Radiation boundary condition . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
 

4.13.8 Mixed radiation-nudging boundary condition . . . . . . . . . . . . . . . . . . 40
 

5 Ice Model Formulation 41
 

5.1 Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
 

5.1.1 Landfast ice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
 

5.2 Thermodynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
 

5.2.1 Ocean surface boundary conditions . . . . . . . . . . . . . . . . . . . . . . . . 48
 

5.2.2 Frazil ice formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
 

5.2.3 Differences from Mellor and Kantha . . . . . . . . . . . . . . . . . . . . . . . 50
 

6 Details of the Code 51
 

6.1 Directory structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
 

6.2 Main subroutines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
 

6.2.1 master.F . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
 

6.2.2 ocean_control.F . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
 

6.2.3 ROMS_initialize . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
 

6.2.4 ROMS_run . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
 

6.2.5 ROMS_finalize . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
 

6.3 Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
 

6.3.1 main3d . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
 

6.4 Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
 

6.5 Functionals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
 

6.6 Other subroutines and functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
 

6.7 C preprocessor variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
 

6.8 Important parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
 

6.9 Domain decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
 

6.9.1 ROMS internal numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
 

6.9.2 MPI exchange . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
 

6.9.3 Code syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
 

6.9.4 Input/output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
 

7 Configuring ROMS for a Specific Application 83
 

7.1 Configuring ROMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
 

7.1.1 Case Name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
 

7.1.2 Case-specific Include File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
 

7.1.3 Functionals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
 

7.1.4 checkdefs.F . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
 

7.1.5 Model domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
 

7.1.6 x, y grid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
 

7.1.7 ξ, η grid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
 

ii
 
7.1.8 Initial conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
 

7.1.9 Equation of state . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
 

7.1.10 Boundary conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
 

7.1.11 Model forcing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
 

7.1.12 ocean.in . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
 

7.1.13 User variables and subroutines . . . . . . . . . . . . . . . . . . . . . . . . . . 94
 

7.2 Upwelling/Downwelling Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
 

7.2.1 cppdefs.h . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
 

7.2.2 Model domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
 

7.2.3 ana_grid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
 

7.2.4 Initial conditions and the equation of state . . . . . . . . . . . . . . . . . . . 97
 

7.2.5 Boundary conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
 

7.2.6 Model forcing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
 

7.2.7 ocean.in . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
 

7.2.8 Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
 

7.3 Arctic example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
 

7.3.1 arctic.h . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
 

7.3.2 Arctic code chunks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
 

7.3.3 Model domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
 

7.3.4 Initial and boundary conditions . . . . . . . . . . . . . . . . . . . . . . . . . . 119
 

7.3.5 Forcing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
 

7.3.6 ocean.in . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
 

7.3.7 Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
 

7.4 Northwest Gulf of Alaska example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
 

7.4.1 nwgoa.h . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
 

7.4.2 NWGOA code chunks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
 

7.4.3 Model domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
 

7.4.4 Initial and boundary conditions . . . . . . . . . . . . . . . . . . . . . . . . . . 124
 

7.4.5 Forcing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
 

7.4.6 ocean.in . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
 

7.4.7 Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
 

7.5 Beaufort Sea example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
 

7.5.1 beaufort.h . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
 

7.5.2 BEAUFORT code chunks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
 

7.5.3 Model domains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
 

7.5.4 Initial and boundary conditions . . . . . . . . . . . . . . . . . . . . . . . . . . 128
 

7.5.5 Forcing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
 

7.5.6 ocean.in . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
 

7.5.7 Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
 

A Model Time-stepping Schemes 131
 

A.1 Euler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
 

A.2 Leapfrog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
 

A.3 Third-order Adams-Bashforth (AB3) . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
 

A.4 Forward-Backward . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
 

A.5 Forward-Backward Feedback (RK2-FB) . . . . . . . . . . . . . . . . . . . . . . . . . 132
 

A.6 LF-TR and LF-AM3 with FB Feedback . . . . . . . . . . . . . . . . . . . . . . . . . 133
 

A.7 Generalized FB with an AB3-AM4 Step . . . . . . . . . . . . . . . . . . . . . . . . . 133
 

iii
 
B The vertical σ-coordinate 134

B.1 Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
 

B.2 Stretching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
 

C Horizontal curvilinear coordinates 136
 

D Viscosity and Diffusion 137
 

D.1 Horizontal viscosity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
 

D.2 Horizontal Diffusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
 

D.3 Vertical Viscosity and Diffusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
 

E Radiant heat fluxes 138
 

E.1 Shortwave radiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
 

E.2 Longwave radiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
 

E.3 Sensible heat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
 

E.4 Latent heat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
 

F The C preprocessor 140
 

F.1 File inclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
 

F.2 Macro substitution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
 

F.3 Conditional inclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
 

F.4 C comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
 

F.5 A note on style . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
 

F.6 Potential problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
 

F.7 Modern Fortran . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
 

G Makefiles 144
 

G.1 Introduction to Portable make . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
 

G.1.1 Macros . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
 

G.1.2 Implicit Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
 

G.1.3 Dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
 

G.2 gnu make . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
 

G.2.1 Make rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
 

G.2.2 Assignments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
 

G.2.3 Include and a Few Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
 

G.2.4 Conditionals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
 

G.3 Multiple Source Directories the ROMS Way . . . . . . . . . . . . . . . . . . . . . . . 150
 

G.3.1 Directory Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
 

G.3.2 Conditionally Including Components . . . . . . . . . . . . . . . . . . . . . . . 150
 

G.3.3 User-defined make Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
 

G.3.4 Library Module.mk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
 

G.3.5 Main Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
 

G.3.6 Top Level Makefile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
 

G.4 Final warnings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
 

H sfmakedepend 158
 

I Subversion 160
 

I.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
 

I.2 Checking out the code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
 

I.3 Updates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
 

I.4 Code changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
 

iv
 
I.5 Conflicts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
 

I.5.1 Merging conflicts by hand . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
 

I.5.2 Copying a file onto your working file . . . . . . . . . . . . . . . . . . . . . . . 163
 

I.5.3 Punting: Using svn revert . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
 

v
 
List of Figures
 

1 Placement of variables on an Arakawa C grid . . . . . . . . . . . . . . . . . . . . . . 14
 

2 Placement of variables on staggered vertical grid . . . . . . . . . . . . . . . . . . . . 15
 

3 Masked region within the domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
 

4

1 function has weights to produce an estimate at time n + 2

Diagrams of the time stepping and mode coupling used in various ROMS versions. (a)
 

Rutgers University ROMS (from myroms.org), (b) older ROMS AGRIF, (c) UCLA
 

ROMS, described in Shchepetkin and McWilliams [2005], (d) non-hydrostatic ROMS
 

(Kanarska et al. [2007]). In all, the curved arrows update the 3-D fields; those with
 

“pillars” are leapfrog in nature with the pillar representing the r.h.s. terms. Straight
 

arrows indicate exchange between the barotropic and baroclinic modes. The shape
 

functions for the fast time steps show just one option out of many possibilities. The
 

grey function has weights to produce an estimate at time n + 1, while the light red

. . . . . . . . . . . . . . . 18
 

5 The split time stepping used in the model. . . . . . . . . . . . . . . . . . . . . . . . . 21
 

6 Weights for the barotropic time stepping. The upper panel shows the primary
 

weights, centered at time n + 1, while the lower panel shows the secondary weights
 1 weights, 2 centered at time n + . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
 

7 Cartoon of an ice floe with a deep keel. . . . . . . . . . . . . . . . . . . . . . . . . . . 44
 

8 Diagram of the different locations where ice melting and freezing can occur. . . . . . 46
 

9 Diagram of internal ice temperatures and fluxes. The hashed layer is the snow. . . . 46
 

10 ROMS directory structure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
 

11 ROMS main structure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
 

12 Flow chart of the model main program in forward mode. Calls to nesting for nested
 

grid applications have been left out. . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
 

13 The whole grid. Note that there are Lm by Mm interior computational points. The
 

points on the thick outer line and those outside it are provided by the boundary

conditions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
 

14 A tiled grid with some ROMS tile variables. . . . . . . . . . . . . . . . . . . . . . . . 76
 

15 A choice of numbering schemes: (a) each tile is numbered the same, and (b) each
 

tile retains the numbering of the parent domain. . . . . . . . . . . . . . . . . . . . . 77
 

16 Some ROMS variables for tiles, for both a periodic and non-periodic case. Shown
 

are the variables in the i-direction, the j-direction is similar. . . . . . . . . . . . . . . 78
 

17 A tiled grid with out-of-date halo regions shown in grey and the interior points
 

color-coded by tile: (a) before an exchange and (b) after an exchange. . . . . . . . . 79
 

18 The upwelling/downwelling bathymetry. . . . . . . . . . . . . . . . . . . . . . . . . . 108
 

19 Surface velocities after one day, showing the flow to the left of the wind (southern
 

hemisphere). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
 

24 Ice concentration averaged over the month of September, 1982. . . . . . . . . . . . . 122
 

25 Ice thickness averaged over the month of September, 1982. . . . . . . . . . . . . . . . 123
 

28 Surface temperature and velocities for 10 August, 2008. . . . . . . . . . . . . . . . . 126
 

20 Constant ξ slices of the u, v, T and w fields at day 1. . . . . . . . . . . . . . . . . . . 110
 

21 Constant ξ slices of the u, v, T, and w fields at day 5. . . . . . . . . . . . . . . . . . . 111
 

22 Bathymetry of the Arctic domain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
 

23 Grid spacing of the Arctic domain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
 

26 Bathymetry of the NWGOA domain. . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
 

27 Surface salinity for 10 August, 2008. . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
 

29 Bathymetry of the larger Beaufort Sea domain. . . . . . . . . . . . . . . . . . . . . . 126
 

30 Bathymetry of the smaller Beaufort Sea domain. . . . . . . . . . . . . . . . . . . . . 127
 

vi
 
31 Ice speed contour at 0.0005 for the April climatology in the smaller Beaufort domain.

This contour was chosen as an estimate of the land-fast ice extent. . . . . . . . . . . 129

32 Sea-ice concentration for the June climatology in the smaller Beaufort domain. Note

the location of a few warm rivers between 148 and 151 W longitude. . . . . . . . . . 129

33	 Vertical slices for the August climatology in the smaller Beaufort domain, (a) temperature, (b) salinity, (c) longshore velocity and (d) cross-shore velocity. Note the

warm, fresh water right up on the shelf, trapped by a barrier island. . . . . . . . . . 130

List of Tables

1 The variables used in the description of the ocean model . . . . . . . . . . . . . . . . 10

2 The variables used in the vertical boundary conditions for the ocean model . . . . . 10

3 The time stepping schemes used in the various ROMS versions. α ≡ ωδt is the

Courant number and ω = ck is the frequency for a wave component with wavenumber

k. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

4 Generic length scale parameters. Note that Mellor-Yamada 2.5 is an example of a

k − kl scheme. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

5 Variables used in the ice momentum equations. . . . . . . . . . . . . . . . . . . . . . 43

6 Variables used in the landfast ice parameterization. . . . . . . . . . . . . . . . . . . . 43

7 Variables used in the ice thermodynamics . . . . . . . . . . . . . . . . . . . . . . . . 45

8 Ocean surface variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

9 Frazil ice variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

10 Variables used in computing the incoming radiation and latent and sensible heat . . 139

1
 
1 Introduction

This user’s manual for the Regional Ocean Modeling System (ROMS) describes the model equations

and algorithms, as well as additional user configurations necessary for specific applications. This

manual also describes the sea-ice model that we are using [Budgell, 2005].

The principle attributes of the model are as follows:

General

•	 Primitive equations with potential temperature, salinity, and an equation of state.

•	 Hydrostatic and Boussinesq approximations.

•	 Optional third-order upwind advection scheme.

•	 Optional Smolarkiewicz advection scheme for tracers (potential temperature, salinity, etc.).

•	 Optional Lagrangian floats.

•	 Option for point sources and sinks.

Horizontal

•	 Orthogonal-curvilinear coordinates.

•	 Arakawa C grid.

•	 Closed basin, periodic, prescribed, radiation, and gradient open boundary conditions.

•	 Masking of land areas.

•	 Early version of nesting.

Vertical

•	 σ (terrain-following) coordinate.

•	 Free surface.

•	 Tridiagonal solve with implicit treatment of vertical viscosity and diffusivity.

Ice

•	 Hunke and Dukowicz elastic-viscous-plastic dynamics.

•	 Mellor-Kantha thermodynamics.

•	 Orthogonal-curvilinear coordinates.

•	 Arakawa C grid.

•	 Smolarkiewicz advection of tracers.

•	 Lemieux landfast ice parameterization.

Mixing options

•	 Horizontal Laplacian and biharmonic diffusion along constant s, z or density surfaces.

•	 Horizontal Laplacian and biharmonic viscosity along constant s or z surfaces.

•	 Optional Smagorinsky horizontal viscosity and diffusion (but not recommended for

diffusion).

•	 Horizontal free-slip or no-slip boundaries.

•	 Vertical harmonic viscosity and diffusion with a spatially variable coefficient, with

options to compute the coefficients via Large et al. [1994], Mellor and Yamada [1974],

or generic length scale (GLS) Umlauf and Burchard [2003] mixing schemes.

1
 
Implementation
 

• Dimensional in meter, kilogram, second (MKS) units.
 

• Fortran 90.
 

•	 Runs under UNIX, requires the C preprocessor, gnu make, and Perl.

•	 All input and output is done in NetCDF Rew et al. [1996] (Network Common Data

Format), requires the NetCDF and NetCDF-fortran libraries.

•	 Options include serial, parallel with MPI, and parallel with OpenMP.

The above list hasn’t changed so very much in the past ten to fifteen years, but many of the

numerical details have changed a great deal. Examples include consistent temporal averaging of

the barotropic mode to guarantee both exact conservation and constancy preservation properties for

tracers; redefined barotropic pressure-gradient terms to account for local variations in the density

field; vertical interpolation performed using conservative parabolic splines; and higher-order, quasimonotone advection algorithms.

New with this version of the manual is the introduction of nesting capabilities which requires

Matlab for preprocessing.

ROMS also comes with a full suite of advanced data assimilation routines; these options are

beyond the scope of this document.

In addition to the sea ice model described here, there are ongoing efforts to couple ROMS with

the Community sea ice model (CICE) [Hunke et al., 2013]. Please ask for the latest status of these

efforts, which include direct coupling, using the model coupling toolkit (MCT) and using the new

NUOPC layer to the Earth System Modeling Framework (ESMF). As of this writing, the first

two of these are available on github while the third is in private alpha testing/debugging and will

ultimately include atmospheric and wave models as well.

Chapter 2 has some information on getting started with ROMS. Chapters 3 and 4 describe the

model physics and numerical techniques and contain information from Shchepetkin and McWilliams

[2009b], Haidvogel et al. [2007]. Chapter 5 describes the ice equations and Chapter 6 lists the

model subroutines and functions. As distributed, ROMS is ready to run with a number of example

problems. The process of configuring ROMS for a particular application and running it is described

in Chapter 7, including a discussion of a few example applications.

2
 
2 Getting started

2.1 myroms.org

Starting off with ROMS is not the easiest thing to do, and it just seems to be getting more complex

as time goes by. There are some resources, however, beginning with the electronic home for ROMS

users at www.myroms.org. Go to register, which gives you access to the subversion server for the

code and to the discussion forum for all things ROMS. There is also a wiki, a bug tracking system,

and even a developer blog.

The wiki contains parts of this manual, but the nature of wikis is that they can be more fluid,

with more authors, than a document such as this. Dave Robertson (robertson@marine.rutgers.edu)

is the one to talk to if you would like to contribute to the wiki. On the other hand, this manual is

now at github, with all that implies (not that I’ve gotten pull requests there yet).

2.2 Prerequisites

As mentioned in Chapter 1, ROMS has some external requirements. These are:

•	 UNIX or UNIX-like environment, such as Cygwin.

•	 A Fortran 90 compiler.

•	 The NetCDF library and the netcdf-fortran library compiled with the above compiler (and a

C compiler).

•	 svn, the subversion revision control software. See Appendix I and the ROMS wiki.

•	 git is optional, but can be used instead of svn, especially for downloading the ice branch of

roms from github.

•	 Gnu make version 3.81 or higher. Appendix G contains more than you ever wanted to know

about this software.

•	 A C preprocessor—the one from gnu with the -traditional flag works well. See Appendix F.

•	 The Perl scripting language.

•	 Matlab is optional, but it is a common tool for pre- and post-processing of ROMS files.

•	 Python is also optional, but might be a good thing to learn if you want an alternative to

Matlab.

Make sure you’ve got the right environment before attempting to download or compile ROMS. In

particular, it’s best to download ROMS using the same environment as will be used when compiling

and running it.

2.3 Acquiring the ROMS code

The main ROMS code is available for download via svn at https://www.myroms.org/svn/src/.

The version of the model described in this document is a merger between ROMS 3.7 and a sea-ice

model and is available at github. I am no longer maintaining the svn branch on the myroms.org

site.

ROMS comes with several cases all ready to go at the flip of a switch. Try these out first and

learn how they are set up.

•	 §2.4 describes how to pick the cases and set up the build environment.

3
 
•	 §6.7 lists all the ROMS options that can be added to your case.

•	 §6.5 lists the fields which can be provided to ROMS via analytic expressions.

•	 §7.1.12 lists the input parameters ROMS reads from a text file at run time.

•	 Chapters 6 and 7 are meant to be informative for the simple and not-so-simple cases. If that

isn’t the case, please let me know.

In addition to this manual, there are some other ROMS resources:

•	 You may be best served by going to the ROMS wiki which includes sections called Getting

Started and Tutorials, as well as a Frequently Asked Questions (FAQ) section.

•	 Don’t be afraid to use the forum. It has everything from employment opportunities to debugging help. Posting there can get you help from one of several people, improving your odds of

success over private emails. Registered users get an email once a day about new postings, so

you might have to wait a day (or more) for a reply.

•	 There have been ROMS meetings and classes in which a tutorial session is included as part

of the program.

•	 There are various resources online—please let me know of any I should list here.

2.4 Compiling ROMS

2.4.1 Environment Variables for make

ROMS has a growing list of choices the user must make before starting the compile process. These

are user-defined variables, some about the model, some about your computer. This section is on the

computer/compiler pairing that is your build system. Since we now use gnu make, it is possible

to set the value of these variables in the Unix environment in a build script, rather than necessarily

inside the Makefile (see §G). The user-definable variables understood by the ROMS makefile are:

ROMS_APPLICATION Set the cpp option defining the particular application. This is used

for setting up options inside the code specific to this application and also determines

the name of the .h header file for it. This can be either a predefined case, such as

BENCHMARK, or one of your own, such as NEP5. It should be in all upper case by

convention.

MY_HEADER_DIR Sets the path to the user’s header file, if any. It can be left empty for the

standard cases, since benchmark.h and the like are found in ROMS/Include, which

is already in the search path. In the case of NEP5, this is set to Apps/NEP where

nep5.h resides.

MY_ANALYTICAL_DIR Sets the path to the user’s analytic files described in §6.5, if any.

This can be User/Functionals or some other location. I tend to place both the header

file and the functionals in the same directory, one directory per application.

MY_CPP_FLAGS Set tunable cpp options. Sometimes it is desirable to activate one or more

cpp options to run different variants of the same application without modifying its header

file. If this is the case, specify each option here using the -D syntax. Notice that you need

to use the shell’s quoting syntax (either single or double quotes) to enclose the definition

if you are using one of the build scripts below.

Compiler-specific Options These flags are used by the files inside the Compilers directory.

4
 
USE_DEBUG Set this to on to turn off optimization and turn on the -g flag for

debugging.

USE_MPI Set this if running an MPI parallel job.

USE_OpenMP Set this if running an OpenMP parallel job.

USE_MPIF90 This flag changes the name of the Fortran compiler to mpif90, while

USE_MPI changes the cpp flags for changing the source code (you might need

both). OpenMPI and others provide a shell script called mpif90 for finding

the required MPI files.

USE_LARGE Some systems support both 32-bit and 64-bit options. Select this to get

64-bit addressing, usually used for programs needing more than 2 GB of memory

on 32-bit systems. Does anyone still need this? I run on machines which are

64-bit by default.

USE_NETCDF4 Set this if linking against the NetCDF4 library. Use this if you have

a working nc-config or nf-config—highly recommended! If you set this, you can

skip the next two.

NETCDF_INCDIR The location of the netcdf.mod and typesizes.mod files.

NETCDF_LIBDIR The location of the NetCDF and netcdf-fortran libraries.

FORT	 A shorthand name for the compiler to be used when selecting which system-compiler file

is to be included from the Compilers directory. See section §G.2.3 and §2.4.2.

Local File Options:

BINDIR Directory in which to place the binary executable. The default is “.”, the

current directory.

SCRATCH_DIR Put the .f90 and the temporary binary files in a build directory

to avoid clutter. The default is Build under the current directory. Set it to

something else if you want to keep these files for multiple projects at the same

time, each in their own directory or if they need to be outside your source tree

to comply with system quotas.

2.4.2 Providing the Environment

Before compiling, you will need to find out some background information:

• What is the name of your compiler?

• What is returned by uname -s on your system?

• Is there a working NetCDF-fortran library?

• Where is it?

• Was it built with the above compiler?

• Did it come with a working nc-config? How about nf-config?

• Do you have access to MPI or OpenMP?

As described more fully in §G.2.3, the makefile will be looking for a file in the Compilers directory

with the combination of your operating system and your compiler. For instance, using Linux and

the Gnu compiler, the file would be called Linux-gfortran.mk. Is the corresponding file for your

5
 
system and compiler in the Compilers directory? If not, you will have to create it following the

existing examples there.

Next, there are several ways to provide the location for the NetCDF files (and optional HDF5

library). The recommended way is to take advantage of the nc-config program that comes

with NetCDF 4.0.1 and newer. The files in the Compilers directory will attempt to do this if

USE_NETCDF4 is defined. Specifically, it will invoke “nc-config –flibs” and “nc-config –prefix”

which you can run on the command line yourself for testing purposes:

pacman3 201% nc-config --flibs
 

-L/usr/local/pkg/netcdf/netcdf-4.3.0.gnu-4.7.3/lib -lnetcdff
 

-L/usr/local/pkg/hdf5/hdf5-1.8.10-p1.gnu-4.7.3/lib
 

-L/usr/local/pkg/szip/szip-2.1.gnu/lib
 

-L/usr/local/pkg/udunits/udunits-2.1.24.gnu-4.7.3/lib
 

-L/usr/local/pkg/netcdf/netcdf-4.3.0.gnu-4.7.3/lib -lnetcdf -lnetcdf
 

pacman3 202% nc-config --prefix
 

/usr/local/pkg/netcdf/netcdf-4.3.0.gnu-4.7.3
 

As you can see, if it needs to link to the hdf5 library, it does this automatically. Another example:

kate@ThinkPad-W520$ nc-config --flibs

kate@ThinkPad-W520$ nf-config --flibs
 

-L/usr/local/lib -lnetcdff -lnetcdf -lnetcdf
 

kate@ThinkPad-W520$ nf-config --prefix
 

/usr/local
 

This is an example of a system where nc-config is broken but nf-config works. Edit this line of

the corresponding file in the Compilers directory:

NC_CONFIG ?= nf-config

If you insist on using the old method, it requires setting the location of the NetCDF files in one of

three ways:

1. Hardcoding lines in the Compilers/xxx-yyy.mk file:

NETCDF_INCDIR ?= /usr/local/include
 

NETCDF_LIBDIR ?= /usr/local/lib
 

2. Setting them in the build.bash (if you can figure out where in all those if cases).

3. Edit your .profile or .login files to globally set them. Here is an example for tcsh:

setenv NETCDF_INCDIR /usr/local/netcdf4/include
 

setenv NETCDF_LIBDIR /usr/local/netcdf4/lib
 

setenv HDF5_LIBDIR /usr/local/hdf5/lib
 

The bash equivalent is:

export NETCDF_INCDIR=/usr/local/netcdf4/include
 

export NETCDF_LIBDIR=/usr/local/netcdf4/lib
 

export HDF5_LIBDIR=/usr/local/hdf5/lib
 

Perhaps you can see why we recommend using nc-config!

6
 
2.4.3 Build scripts

If you have more than one application (or more than one compiler), you will get tired of editing

the makefile. One option is to have a makefile for each configuration, then type:

make -f makefile.circle_pgi

for instance. Another option for keeping track of the user-defined choices is in a build script. The

advantage is that updates to the build scripts are less frequent than updates to the makefile.

There are now two of these scripts in the ROMS/Bin directory: build.sh (which is surprisingly

a csh script) and build.bash. The build scripts use environment variables to provide values for

the list above, overwriting those found in the ROMS makefile. Just as in the multiple makefile

option, you will need as many copies of the build script as you have applications. The scope of

these variables is local to the build script, allowing you to compile different applications at the

same time from the same sources as long as each $(SCRATCH_DIR) is unique.

Both scripts have the same options:

-j [N] Compile in parallel using N cpus, omit argument for all available CPUs.

-noclean Do not clean already compiled objects.

Note that the default is to compile serially and to issue a “make clean” before compiling. It is

left as an exercise for the user if you prefer a different default behavior.

There are also a few variables which are not recognized by the ROMS makefile, but are used

locally inside the build script. These are:

MY_PROJECT_DIR This is used in setting $(SCRATCH_DIR) and $(BINDIR).

MY_ROMS_SRC Set the path to the user’s local current ROMS source code. This is used so

that the script can be run from any directory, not necessarily only from the top ROMS

directory.

2.5 Running ROMS

ROMS expects to read a number of variables from an ASCII file (details of the file are in §7.1.12).

For serial or OpenMP execution, the syntax is:

oceanS (or oceanO) < ocean.in > roms.out &

while MPI execution requires:

oceanM ocean.in > roms.out &

so that each process can read the ocean.in file.

Realistically, you would only want to run relatively small applications such as UPWELLING

interactively on the command line as shown here. Also, for either of the parallel options, you will

have to provide some information to ROMS and to the operating system about how many threads

or processes to use. Parallel computers may also have some sort of batch queuing system in place

in which you would submit a job script. I have had easy access to a number of systems over the

years, each with its own style of job script. Some MPI environments require that you run ROMS

with:

cd $PBS_O_WORKDIR
 

mpirun -np 32 ./oceanM ocean_benchmark3.in
 

while others need:

7
 
aprun -np 32 ./oceanM ocean_benchmark3.in

You just have to find out from the locals, for this as well as the syntax of the batch queue scripts.

If all goes according to plan, ROMS will create both a collection of NetCDF files and a verbose

text file on standard out. There is a large and ever-changing list of ways to view NetCDF files,

but I like ncview for a first view of what’s going on. Other tools include Matlab, Python, NCL,

Fortran and ferret.

If things don’t go according to plan, the text output file is your friend. Examine it carefully. If

it fails on the UPWELLING problem, you can compare your output to that in §7.2.8. If it fails

on your application, make sure simple domains like UPWELLING work on your system.

2.6 Warnings and bugs

ROMS is not a large program by some standards, but it is still complex enough to require some

effort to use effectively. Some specific things to be wary of include:

•	 It is recommended that you use 64 bits of precision rather than 32 bits.

•	 The code must be run through the C preprocessor before it is compiled. This can occasionally

be dangerous, especially with the newer ANSI C versions of cpp. Potential problems are listed

in Appendix F. The gnu cpp with the -traditional flag is known to work well. On the other

hand, the output of cpp is saved in the $(SCRATCH_DIR), where it can be examined.

•	 The vertical σ-coordinate was chosen as being a sensible way to handle variations in the

water depth as seen in the coastal oceans. Changes to the code have allowed us to expand the

well-behaved range of depths and the range of values for THETA_S, plus there are some

new vertical coordinate options.

•	 σ-coordinates have long had a bad reputation because errors in the pressure gradient terms

can lead to spurious currents. These errors are much less troublesome than in the past due to

code improvements and there are more improvements possible through an alternate equation

of state [Shchepetkin and McWilliams, 2011], though they are not yet in the myroms.org code.

The pressure gradient errors can also be controlled with some smoothing of the bathymetry.

This in turn changes the shape of the basin and leads to its own set of problems, such as

altered sill depths. Also, the currents will react to the change in shelf slope—you are now

solving a different problem.

•	 I occasionally get the model to run for multiple days/months, then suddenly blow up middepth. Just restart with a smaller timestep and keep on going.

•	 There remain bugs in ROMS. If you find any, please report them on the forum and/or the

bug tracking system at myroms.org.

8
 
3 Ocean Model Formulation

3.1 Equations of motion

ROMS is a member of a general class of three-dimensional, free-surface, terrain-following numerical models that solve the Reynolds-averaged Navier-Stokes equations using the hydrostatic and

Boussinesq assumptions. The governing equations in Cartesian coordinates can be written:

 ∂u ∂φ ∂ ∂u

+ -v · 'u − fv = − − u'

w' − ν + Fu + Du (1) ∂t ∂x ∂z ∂z

 ∂v ∂φ ∂ ∂v

+ -v · 'v + fu = − v − ' − ν + Dv '

w + Fv (2) ∂t ∂y ∂z ∂z

∂φ −ρg = (3) ∂z ρo

with the continuity equation:

∂u ∂v ∂w

+ + = 0. (4) ∂x ∂y ∂z

and scalar transport given by:

 ∂C ∂ ∂C

+ -v · 'C = − C'

w' − νθ + FC + DC. (5) ∂t ∂z ∂z

An equation of state is also required:

ρ = ρ(T, S, P) (6)

The variables are shown in Table 3.1. An overbar represents a time average and a prime represents

a fluctuation about the mean. These equations are closed by parameterizing the Reynolds stresses

and turbulent tracer fluxes as:

∂u ∂v ∂C

u'

w' = −KM ; v'w' = −KM ; C'w' = −KC . (7) ∂z ∂z ∂z

Equations (1) and (2) express the momentum balance in the x- and y-directions, respectively.

The time evolution of all scalar concentration fields, including those for T(x, y, z,t) and S(x, y, z,t),

are governed by the advective-diffusive equation (5). The equation of state is given by equation

(6). In the Boussinesq approximation, density variations are neglected in the momentum equations

except for their contribution to the buoyancy force in the vertical momentum equation (3). Under

the hydrostatic approximation, it is further assumed that the vertical pressure gradient balances

the buoyancy force. Lastly, equation (4) expresses the continuity equation for an incompressible

fluid. For the moment, the effects of forcing and horizontal dissipation will be represented by the

schematic terms F and D, respectively. The horizontal and vertical mixing will be described more

fully in §4.10.

9
 
Variable Description

C(x, y, z,t)

Du, Dv, DC

Fu, Fv, FC

f(x, y)

g

h(x, y)

Hz(x, y, z)

ν, νθ

KM, KC

P

φ(x, y, z,t)

ρo + ρ(x, y, z,t)

S(x, y, z,t)

t

T(x, y, z,t)

u, v, w

x, y

z

ζ(x, y,t)

scalar quantity, i.e. temperature, salinity, nutrient concentration

optional horizontal diffusive terms

forcing/source terms

Coriolis parameter

acceleration of gravity

depth of sea floor below mean sea level

vertical grid spacing

molecular viscosity and diffusivity

vertical eddy viscosity and diffusivity

total pressure P ≈ −ρogz

dynamic pressure φ = (P/ρo)

total in situ density

salinity

time

potential temperature

the (x, y, z) components of vector velocity -v

horizontal coordinates

vertical coordinate

the surface elevation

Table 1: The variables used in the description of the ocean model

3.2 Vertical boundary conditions

The vertical boundary conditions can be prescribed as follows:

∂u top (z = ζ(x, y,t)) Km = τ x(x, y,t) ∂z s

∂v = τ y Km (x, y,t) ∂z s

K ∂C QC C ∂z = ρocP

∂ζ w = ∂t

∂u and bottom (z = −h(x, y)) Km = τ x(x, y,t) ∂z b

K ∂v m = τ y(x, y,t) ∂z b

K ∂C

C = 0 ∂z

−w + -v · 'h = 0.

Variable Description

QC

τ x, τ y s s

τ x, τ y

b b

surface concentration flux

surface wind stress

bottom stress

Table 2: The variables used in the vertical boundary conditions for the ocean model

The surface boundary condition variables are defined in Table 3.2. Since QT is a strong function

of the surface temperature, we usually choose to compute QT using the surface temperature and

the atmospheric fields in an atmospheric bulk flux parameterization. This bulk flux routine also

computes the wind stress from the winds.

On the variable bottom, z = −h(x, y), the horizontal velocity has a prescribed bottom stress

which is a choice between linear, quadratic, or logarithmic terms. The vertical concentration flux

10
 
 

 

 

 

may also be prescribed at the bottom, although it is usually set to zero.

3.3 Horizontal boundary conditions

As distributed, the model can easily be configured for a periodic channel, a doubly periodic domain,

or a closed basin. Code is also included for open boundaries which may or may not work for your

particular application. Horizontal boundary conditions are described more fully in §4.13 and are

provided for u, v, T, S, and ζ.

The model domain is logically rectangular, but it is possible to mask out land areas on the

boundary and in the interior. Boundary conditions on these masked regions are straightforward,

with a choice of no-slip or free-slip walls.

If biharmonic friction is used, a higher order boundary condition must also be provided. The

model currently has this built into the code where the biharmonic terms are calculated. The high

 h ∂ ν ∂2u order boundary conditions used for u are = 0 on the eastern and western boundaries ∂x ∂x2 h ∂ ν ∂2u and = 0 on the northern and southern boundaries. The boundary conditions for v and ∂y ∂y2

C are similar. These boundary conditions were chosen because they preserve the property of no

gain or loss of volume-integrated momentum or scalar concentration.

3.4 Terrain-following coordinate system

From the point of view of the computational model, it is highly convenient to introduce a stretched

vertical coordinate system which essentially “flattens out” the variable bottom at z = −h(x, y).

Such “σ” coordinate systems have long been used, with slight appropriate modification, in both

meteorology and oceanography [e.g., Phillips, 1957, Freeman et al., 1972]. To proceed, we make

the coordinate transformation:

xˆ = x
 

yˆ = y
 

σ = σ(x, y, z)
 

z = z(x, y, σ)
 

and

t

ˆ= t.

See Appendix B for the form of σ used here. Also, see Shchepetkin and McWilliams [2005] for a

discussion about the nature of this form of σ and how it differs from that used in SCRUM.

In the stretched system, the vertical coordinate σ spans the range −1 ≤ σ ≤ 0; we are therefore

left with level upper (σ = 0) and lower (σ = −1) bounding surfaces. The chain rules for this

transformation are:

∂ ∂ 1 ∂z ∂ = − ∂x z ∂x σ Hz ∂x σ ∂σ

∂ ∂ 1 ∂z ∂ = − ∂y z ∂y σ Hz ∂y σ ∂σ

∂ ∂s ∂ 1 ∂ = = ∂z ∂z ∂σ Hz ∂σ

where

∂z Hz ≡

∂σ

As a trade-off for this geometric simplification, the dynamic equations become somewhat more

complicated. The resulting dynamic equations are, after dropping the carats:

 ∂u ∂φ gρ ∂z ∂ζ 1 ∂ Km ∂u − fv + -v · 'u = − − − g + + Fu + Du (8) ∂t ∂x ρo ∂x ∂x Hz ∂σ Hz ∂σ

11
 
 

 

 

 

 

 

 

 

 

 

∂v ∂φ gρ ∂z ∂ζ 1 ∂ Km ∂v

+ fu + -v · 'v = − − − g + + Fv + Dv (9) ∂t ∂y ρo ∂y ∂y Hz ∂σ Hz ∂σ

∂C 1 ∂ ∂C

+ -v · 'C = KC + FT + DT (10) ∂t Hz ∂σ Hz ∂σ

ρ = ρ(T, S, P) (11)

∂φ −gHzρ = (12) ∂σ ρo

∂Hz ∂(Hzu) ∂(Hzv) ∂(HzΩ) + + + = 0 (13) ∂t ∂x ∂y ∂σ

where

-v = (u, v, Ω)

∂ ∂ -v · ' = u + v + Ω ∂ . ∂x ∂y ∂σ

The vertical velocity in σ coordinates is

1 z + h ∂ζ ∂z ∂z Ω(x, y, σ,t) = w − − u − v Hz ζ + h ∂t ∂x ∂y

and

∂z ∂z ∂z

w = + u + v + ΩHz. ∂t ∂x ∂y

In the stretched coordinate system, the vertical boundary conditions become:

h Km ∂u top (σ= 0) = τ x(x, y,t) Hz ∂σ s h Km ∂v = τ y(x, y,t) Hz ∂σ s h KC ∂C = QC

Hz ∂σ ρocP

Ω = 0

h Km ∂u and bottom (σ = −1) = τ x(x, y,t) Hz ∂σ b

h Km ∂v = τ y

(x, y,t) Hz ∂σ b

h KC ∂C = 0 Hz ∂s

Ω = 0.

Note the simplification of the boundary conditions on vertical velocity that arises from the σ

coordinate transformation.

3.5 Horizontal curvilinear coordinates

In many applications of interest (e.g., flow adjacent to a coastal boundary), the fluid may be confined

horizontally within an irregular region. In such problems, a horizontal coordinate system which

conforms to the irregular lateral boundaries is advantageous. It is often also true in many geophysical problems that the simulated flow fields have regions of enhanced structure (e.g., boundary

currents or fronts) which occupy a relatively small fraction of the physical/computational domain.

In these problems, added efficiency can be gained by placing more computational resolution in such

regions.

The requirement for a boundary-following coordinate system and for a laterally variable grid

resolution can both be met, for suitably smooth domains, by introducing an appropriate orthogonal

12
 
 

 

 

 

 

 

 

 

 

 

 

 

� �

� �

coordinate transformation in the horizontal. Let the new coordinates be ξ(x, y) and η(x, y), where

the relationship of horizontal arc length to the differential distance is given by:

1 (ds)ξ = dξ (14) m

1 (ds)η = dη (15) n

Here, m(ξ, η) and n(ξ, η) are the scale factors which relate the differential distances (Δξ, Δη) to

the actual (physical) arc lengths. Appendix C contains the curvilinear version of several common

vector quantities.

Denoting the velocity components in the new coordinate system by

-v · ξ

ˆ = u (16)

and

-v · ηˆ = v (17)

the equations of motion (8)-(13) can be re-written [see, e.g., Arakawa and Lamb, 1977] as:

∂ Hzu ∂ Hzu2 ∂ Hzuv ∂ HzuΩ

+ + + ∂t mn ∂ξ n ∂η m ∂σ mn

 f ∂ 1 ∂ 1 − + v − u Hzv = mn ∂ξ n ∂η m

Hz ∂φ gρ ∂z ∂ζ 1 ∂ Km ∂u Hz − + + g + + (Fu + Du) (18) n ∂ξ ρo ∂ξ ∂ξ mn ∂σ Hz ∂σ mn

∂ Hzv ∂ Hzuv ∂ Hzv2 ∂ HzvΩ

+ + + ∂t mn ∂ξ n ∂η m ∂σ mn

 f ∂ 1 ∂ 1

+ + v − u Hzu = mn ∂ξ n ∂η m

Hz ∂φ gρ ∂z ∂ζ 1 ∂ Km Hz − + + g + ∂v∂σ + (Fv + Dv) (19) m ∂η ρo ∂η ∂η mn ∂σ Hz mn

∂ HzC ∂ HzuC ∂ HzvC ∂ HzΩC

+ + + = ∂t mn ∂ξ n ∂η m ∂σ mn

1 ∂ KC ∂C Hz + (FC + DC) (20) mn ∂s Hz ∂σ mn

ρ = ρ(T, S, P) (21)

∂φ gHzρ = − (22) ∂σ ρo

∂ Hz ∂ Hzu ∂ Hzv ∂ HzΩ

+ + + = 0. (23) ∂t mn ∂ξ n ∂η m ∂σ mn

All boundary conditions remain unchanged.

13
 
4 Numerical Solution Technique

4.1 Vertical and horizontal discretization

4.1.1 Horizontal grid

In the horizontal (ξ, η), a traditional, centered, second-order finite-difference approximation is

adopted. In particular, the horizontal arrangement of variables is as shown in Fig. 1. This is

equivalent to the well known Arakawa “C” grid, which is well suited for problems with horizontal

resolution that is fine compared to the first radius of deformation [Arakawa and Lamb, 1977].

_ Δξ

t vi,j+1

_

t

ui,j _

(ρ, h, f, Ω)i,j

f

ui+1,j _ Δη

t vi,j

_

Figure 1: Placement of variables on an Arakawa C grid

Note that the relative placement of the velocity points which match the indices of the central

tracer point is known as the Southwest convention while some other well-known models use a

Northeast convention.

4.1.2 Vertical grid

The vertical discretization also uses a second-order finite-difference approximation. Just as we use

a staggered horizontal grid, the model was found to be more well-behaved with a staggered vertical

grid. The vertical grid is shown in Fig. 2.

4.2 Masking of land areas

ROMS has the ability to work with interior land areas, although the computations occur over the

entire model domain. One grid cell is shown in Fig. 1 while several cells are shown in Fig. 3,

including two land cells. The process of defining which areas are to be masked is external to ROMS

and is usually accomplished in Matlab or Python; this section describes how the masking affects

the computation of the various terms in the equations of motion.

4.2.1 Velocity

At the end of every time step, the values of many variables within the masked region are set to zero

by multiplying by the mask for either the u, v or ρ points. This is appropriate for the v points E

and L in Fig. 3, since the flow in and out of the land should be zero. It is likewise appropriate for

the u point at I, but is not necessarily correct for point G. The only term in the u equation that

14
 
�

�

�

wN

� ρN

wN−1

� ρN−1

w2

� ρ2

w1

ρ1

w0

Figure 2: Placement of variables on staggered vertical grid

∂ ν ∂u requires the u value at point G is the horizontal viscosity, which has a term of the form . ∂η ∂η

Since point G is used in this term by both points A and M, it is not sufficient to replace its value

with that of the image point for A. Instead, the term ∂u

∂η is computed and the values at points D

and K are replaced with the values appropriate for either free-slip or no-slip boundary conditions.

∂ ν ∂v Likewise, the term in the v equation must be corrected at the mask boundaries. ∂ξ ∂ξ

This is accomplished by having a fourth mask array defined at the ψ points, in which the values

are set to be no-slip in metrics. For no-slip boundaries, we count on the values inside the land

(point G) having been zeroed out. For point D, the image point at G should contain minus the

∂u value of u at point A. The desired value of is therefore 2uA while instead we have simply uA. ∂η

In order to achieve the correct result, we multiply by a mask which contains the value 2 at point

D ∂u . It also contains a 2 at point K so that there will acquire the desired value of −2uM. The ∂η

corner point F is set to have a value of 1.

4.2.2 Temperature, salinity and surface elevation

The handling of masks by the temperature, salinity and surface elevation equations is similar to

that in the momentum equations, and is in fact simpler. Values of T, S and ζ inside the land

masks, such as point H in Fig. 3, are set to zero after every time step. This point would be used by

the horizontal diffusion term for points B, J, and N. This is corrected by setting the first derivative

terms at points E, I, and L to zero, to be consistent with a no-flux boundary condition. Note that

the equation of state must be able to handle T = S = 0 since these are the values inside masked

regions.

4.2.3 Wetting and drying

There is now an option to have wetting and drying in the model, in which a cell can switch between

being wet or being dry as the tides come in and go out, for instance. However, cells which are

masked out as in Fig. 3 are never allowed to be wet.

•	 In the case of wetting and drying, a critical depth, Dcrit, is supplied by the user.

•	 The total water depth (D = h + ζ) is compared to Dcrit. If the water level is less than this

depth, no flux is allowed out of that cell. Water can always flow in and resubmerge the cell.

•	 The wetting and drying only happens during the 2-D computations; the 3-D computations

see a depth of Dcrit in the “dry” areas, though surface fluxes are turned off in the dry cells,

as are some other processes.

15
 
CBA

D E

G H I J

K L

M N

F

– u points

– v points

– ρ points

– ψ points

Figure 3: Masked region within the domain

•	 The ice component now checks for dry cells when computing the ice rheology and ice thermodynamics.

4.3 Time-stepping overview

While time stepping the model, we have a stored history of the model fields at time n − 1, an

estimate of the fields at the current time n, and we need to come up with an estimate for time

n + 1. For reasons of efficiency, we choose to use a split-explicit time step, integrating the depthintegrated equations with a shorter time step than the full 3-D equations. There is an integer ratio

M between the time steps. The exact details of how the time stepping is done vary from one version

of ROMS to the next, with the east coast ROMS described here being older than other branches.

Still, all versions have these steps:

1. Take a predictor step for at least the 3-D tracers to time n + 1

2 .
 

1

2

2. Compute ρ and ρ∗

n or time n +

for use in the depth-integrated time steps, from the density either at time
 

.
 

3. Depth integrate the 3-D momentum right-hand side terms at time n+ 1

2 for use in the depthintegrated time steps (or extrapolate to obtain an estimate of those terms).
 

1

2

4. Take all the depth-integrated steps. Store weighted time-means of the u, v fields centered at

both time n + and time n + 1 (plus ζ at time n + 1). The latter requires this time stepping
 

to extend past time n + 1, using M∗ steps rather than just M.

5. Use the weighted time-means from depth-integrated fields to complete the corrector step for

the 3-D fields to time n + 1.

Great care is taken to avoid the introduction of a mode-splitting instability due to the use of shorter

time steps for the depth-integrated computations.

The mode coupling has evolved through the various ROMS versions, as shown in Fig. 4 [from

Shchepetkin and McWilliams, 2009a]. The time stepping schemes are also listed in Table 4.3 and

16
 
 � � � � 

 

 � � � � 

described in detail in Shchepetkin and McWilliams [2005] and Shchepetkin and McWilliams [2009b];

the relevant ones are described in Appendix A.

4.4 Conservation properties

From Shchepetkin and McWilliams [2005], we have a tracer concentration equation in advective

form:

∂C + (u · ')C = 0 (24) ∂t

and also a tracer concentration equation in conservation form:

∂C

+ ' · (uC) = 0. (25) ∂t

The continuity equation:

(' · u) = 0 (26)

can be used to get from one tracer equation to the other. As a consequence of eq. (24), if the tracer is

spatially uniform, it will remain so regardless of the velocity field (constancy preservation). On the

other hand, as a consequence of (25), the volume integral of the tracer concentration is conserved

in the absence of internal sources and fluxes through the boundary. Both properties are valuable

and should be retained when constructing numerical ocean models.

The semi-discrete form of the tracer equation (20) is:

ξ ξ η η ∂ HzC uHz C vHz C σ HzΩ 1 ∂ Km ∂C

+δξ +δη +δσ C = +DC +FC (27) ξ η ∂t mn n m mn mn ∂σ Δz ∂σ

Here δξ, δη and δσ denote simple centered finite-difference approximations to ∂/∂ξ, ∂/∂η and ∂/∂σ

with the differences taken over the distances Δξ, Δη and Δσ, respectively. Δz is the vertical

distance from one ρ point to another. ( )

ξ

, ( )

η and ( )σ represent averages taken over the

distances Δξ, Δη and Δσ.

The finite volume version of the same equation is no different, except that a quantity C is

defined as the volume-averaged concentration over the grid box ΔV :

 mn HzC C = δξ δη δσ (28) Hz ΔV mn

uHz C The quantity

ξ ξ

is the flux through an interface between adjacent grid boxes. ξ n

This method of averaging was chosen because it internally conserves first moments in the model

domain, although it is still possible to exchange mass and energy through the open boundaries. The

method is similar to that used in Arakawa and Lamb [1977], though their scheme also conserves

enstrophy. Instead, we will focus on (nearly) retaining constancy preservation while coupling the

barotropic (depth-integrated) equations and the baroclinic equations.

The time step in eq. (27) is assumed to be from time n to time n+ 1, with the other terms being

evaluated at time n + 1

2 for second-order accuracy. Setting C to 1 everywhere reduces eq. (27) to:

ξ η ∂ Hz uHz vHz HzΩ

+ δξ + δη + δσ = 0 (29) ξ η ∂t mn n m mn

If this equation holds true for the step from time n to time n + 1, then our constancy preservation

will hold.

In a hydrostatic model such as ROMS, the discrete continuity equation is needed to compute

Hz vertical velocity rather than grid-box volume (the latter is controlled by changes in ζ in the mn

17
 
Figure 4: Diagrams of the time stepping and mode coupling used in various ROMS versions.

(a) Rutgers University ROMS (from myroms.org), (b) older ROMS AGRIF, (c) UCLA ROMS, described in Shchepetkin and McWilliams [2005], (d) non-hydrostatic ROMS (Kanarska et al. [2007]).

In all, the curved arrows update the 3-D fields; those with “pillars” are leapfrog in nature with the

pillar representing the r.h.s. terms. Straight arrows indicate exchange between the barotropic and

baroclinic modes. The shape functions for the fast time steps show just one option out of many

possibilities. The grey function has weights to produce an estimate at time n + 1, while the light

red function has weights to produce an estimate at time n + 1

2 .

18
 
19
 

SCRUM 3.0 Rutgers AGRIF UCLA Non-hydrostatic

Reference Hedstrom [2000] Haidvogel et al. [2000] Penven et al. [2006] Shchepetkin and McWilliams [2005] Kanarska et al. [2007]Barotropicmode

LF-TR

√

LF-AM3 with

FB feedback

LF-AM3 with

FB feedbacka

Gen. FB

(AB3-AM4)

Gen. FB

(AB3-AM4)

2-D αmax, iter. 2, (2)b 1.85, (2) 1.85, (2) 1.78, (1) 1.78, (1)

3-D momenta AB3 AB3 LF-AM3 LF-AM3 AB3 (mod)

Tracers AB3 LF-TR LF-AM3 LF-AM3 AB3 (mod)

Internalwaves

AB3 Gen. FB

(AB3-TR)

LF-AM3,

FB feedback

LF-AM3,

FB feedback

Gen. FB

(AB3-AM4)

αmax, advect. 0.72 0.72 1.587 1.587 0.78

αmax, Cor. 0.72 0.72 1.587 1.587 0.78αmax, int. w. 0.72, (1) 1.14, (1,2) 1.85, (2) 1.85, (2) 1.78, (1)

Table 3: The time stepping schemes used in the various ROMS versions. α ≡ ωδt is the Courant number and ω = ck is the frequency for a wave

component with wavenumber k.

aThe generalized FB barotropic mode was ported into the newest AGRIF code at the end of 2007.

bThe number in parentheses (e.g., 2) indicates the number of r.h.s. computations per time step. If there are two parenthesized number, the first one is for momenta,

the second for tracers.
 � � � �

 

 

 � �

 

 

 � �

 

barotropic mode computations). Here, HzΩ is the finite-volume flux across the moving grid-box mn

interface, vertically on the w grid.

The vertical integral of the continuity eq. (23), using the vertical boundary conditions on Ω, is:

ξ η ∂ ζ uD vD

+ δξ + δη = 0	 (30) ξ η ∂t mn n m

where ζ is the surface elevation, D = h + ζ is the total depth, and u, v are the depth-integrated

horizontal velocities. This equation and the corresponding 2-D momentum equations are time

stepped on a shorter time step than eq. (27) and the other 3-D equations. Due to the details in

the mode coupling, it is only possible to maintain constancy preservation to the accuracy of the

barotropic time steps.

4.5 Depth-integrated equations

The depth average of a quantity A is given by:

0 1 A = HzAdσ	 (31) D −1

where the overbar indicates a vertically averaged quantity and

D ≡ ζ(ξ, η,t) + h(ξ, η)	 (32)

is the total depth of the water column. The vertical integral of equation (18) is:

∂ Du ∂ Duu ∂ Duv Dfv + + − ∂t mn ∂ξ n ∂η m mn
 

∂ 1 ∂ 1 D ∂φ2 ∂ζ
 − vv − uv D = − + g ∂ξ n ∂η m	 n ∂ξ ∂ξ

D h 1 h

ξ + + τ ξ Fu + Dh − τ (33) mn u s b 	 mn

∂z where φ2 includes the term, Dhu is the horizontal viscosity, and the vertical viscosity only ∂ξ

contributes through the upper and lower boundary conditions. The corresponding vertical integral

of equation (19) is:

∂ Dv ∂ Duv ∂ Dvv Dfu + + + ∂t mn ∂ξ n ∂η m mn

∂ 1 ∂ 1	 D ∂φ2 ∂ζ + uv − uu D = − + g ∂ξ n ∂η m	 m ∂η ∂η

D h 1 η + Fv + Dhv + τs

η − τb . (34) mn	 mn

We also need the vertical integral of equation (23), shown above as eq. (30).

√ The presence of a free surface introduces waves which propagate at a speed of gh. These

waves usually impose a more severe time-step limit than any of the internal processes. We have

therefore chosen to solve the full equations by means of a split time step. In other words, the depth

integrated equations (33), (34), and (30) are integrated using a short time step and the values of

u and v are used to replace those found by integrating the full equations on a longer time step. A

diagram of the barotropic time stepping is shown in Fig. 5.

20
 
m=M

Barotropic steps

m=0 m=M

n n+1

*

 

 

 

 

 

Figure 5: The split time stepping used in the model.

Some of the terms in equations (33) and (34) are updated on the short time step while others

are not. The contributions from the slow terms are computed once per long time step and stored.

If we call these terms Ruslow and Rvslow , equations (33) and (34) become:

∂ Du ∂ Du u	 ∂ Du v Dfv + + − ∂t mn ∂ξ n	 ∂η m mn
 

∂ 1 ∂ 1 gD ∂ζ D 1
 ξ − v v − u v D = Ruslow − + Du − τb (35) ∂ξ n ∂η m	 n ∂ξ mn mn

∂ Dv ∂ Du v	 ∂ Dv v Dfu + + + ∂t mn ∂ξ n ∂η m mn
 

∂ 1 ∂ 1 gD ∂ζ D 1
 η + u v − u u D = Rvslow − + Dv − τb . (36) ∂ξ n ∂η m	 m ∂η mn mn

When time stepping the model, we compute the right-hand-sides for equations (18) and (19) as

well as the right-hand-sides for equations (35) and (36). The vertical integral of the 3-D righthand-sides are obtained and then the 2-D right-hand-sides are subtracted. The resulting fields are

the slow forcings Ruslow and Rvslow . This was found to be the easiest way to retain the baroclinic

contributions of the non-linear terms such as uu − u u.

The model is time stepped from time n to time n+1 by using short time steps on equations (35),

(36) and (30). Equation (30) is time stepped first, so that an estimate of the new D is available for

the time rate of change terms in equations (35) and (36). A third-order predictor-corrector time

stepping is used. In practice, we actually time step all the way to time (n + dtfast × M*), while

maintaining weighted averages of the values of u, v and ζ. The averages are used to replace the

values at time n + 1 in both the baroclinic and barotropic modes, and for recomputing the vertical

grid spacing Hz. Fig. 6 shows one option for how these weights might look.

 M:

The primary weights, am, are used to compute (ζ)n+1 ≡ m=1 amζm. There is a related set of

 M:

secondary weights bm, used as ((u))n+ 2

1

≡ m=1 bmum. In order to maintain constancy preservation,

this relation must hold:

 �� �� n+ 1 �� ��n+ 1 �� ��n+ 1 �� ��n+ 1

(ζ)

n+1	 Du 2 Du 2 Dv 2 Dv 2

= (ζ)i,j

n − (mn)i,jΔt − + −	 (37) i,j	 n i+ m 2

1 ,j n i− 1

2 ,j i,j+ 1 m i,j− 12 2

Shchepetkin and McWilliams [2005] introduce a range of possible weights, but the ones used here

have a shape function:

p q τ τ τ A(τ ) = A0 1 − − r	 (38)

τ0 τ0 τ0

21
 
Figure 6: Weights for the barotropic time stepping. The upper panel shows the primary weights,

centered at time n+ 1, while the lower panel shows the secondary weights weights, centered at time

n + 1

2 .

22
 
 

 

 

 

 

� � 

where p, q are parameters and A0, τ0, and r are chosen to satisfy normalization, consistency, and

second-order accuracy conditions,

τ:

In = τnA(τ )dτ = 1, n = 0, 1, 2 (39)

0

using Newton iterations. τ * is the upper limit of τ with A(τ ) ≥ 0. In practice we initially set

(p + 2)(p + q + 2) A0 = 1, r = 0 and τ = (p + 1)(p + q + 1),

compute A(τ ) using eq. (38), normalize using:

M

M: MM:

m

am ≡ 1, am ≡ 1, (40)

m=1 m=1 M

and adjust r iteratively to satisfy the n = 2 condition of (39). We are using values of p = 2, q = 4,

and r = 0.284. This form allows some negative weights for small m, allowing M* to be less than

1.5M.

ROMS also supports an older cosine weighting option, which isn’t recommended since it is only

first-order accurate.

4.6 Density in the mode coupling

Equation (35) contains the term Ruslow , computed as the difference between the 3-D right-hand-side

and the 2-D right-hand-side. The pressure gradient therefore has the form:

gD ∂ζ gD ∂ζ − + + F (41) n ∂ξ n ∂ξ

where the term in square brackets is the mode coupling term and is held fixed over all the barotropic

steps and

ζ 1 ∂P F = − dz (42) ρ0n −h ∂ξ

is the vertically integrated pressure gradient. The latter is a function of the bathymetry, free surface

gradient, and the free surface itself, as well as the vertical distribution of density.

The disadvantage of this approach is that after the barotropic time stepping is complete and

the new free surface is substituted into the full baroclinic pressure gradient, its vertical integral will

no longer be equal to the sum of the new surface slope term and the original coupling term based

on the old free surface. This is one form of mode-splitting error which can lead to trouble because

the vertically integrated pressure gradient is not in balance with the barotropic mass flux.

Instead, let us define the following:

ρ = 1

D

ζ

ρdz, −hρ* = 1

1

2D2

 ζ

−h

ζ

z

 

ρdz' dz (43)

Changing the vertical coordinate to σ yields:

0 0 0

ρ = ρdσ, −1ρ* = 2 −1 σρdσ' dσ (44)

which implies that ρ and ρ* are actually independent of ζ as long as the density profile ρ = ρ(σ)

does not change. The vertically integrated pressure gradient becomes:

 1 g ∂ ρ*D2 ∂h 1 g

ρ* ∂ζ D ∂ρ*

− − ρD = − D + + (ρ* − ρ)

∂h (45) ρ0 n ∂ξ 2 ∂ξ ρ0 n ∂ξ 2 ∂ξ ∂ξ

In the case of uniform density ρ0, we obtain ρ* ≡ ρ ≡ ρ0, but we otherwise have two new terms.

The accuracy of these terms depends on an accurate vertical integration of the density, as described

in Shchepetkin and McWilliams [2005].

23
 
4.7 Time stepping: internal velocity modes and tracers

The momentum equations (18) and(19) are advanced before the tracer equation, by computing all

the terms except the vertical viscosity and then using the implicit scheme described in §4.12 to

find the new values for u and v. The depth-averaged component is then removed and replaced by

the (u) and (v) computed as in §4.5. A third-order Adams-Bashforth (AB3) time stepping is used,

requiring multiple right-hand-side time levels (see Appendix A). These stored up r.h.s. values can

1

2 be used to extrapolate to a value at time n + for use in the barotropic steps as shown in Fig. 4.
 

The tracer concentration equation (27) is advanced in a predictor-corrector leapfrog-trapezoidal

step, with great care taken to optimize both the conservation and constancy-preserving properties

1

2

of the continuous equations. The corrector step can maintain both, as long as it uses velocities

and column depths which satisfy eq. (37). This also requires tracer values centered at time n + ,
 

obtained from the predictor step. The vertical diffusion is computed as in §4.12.

The predictor step cannot be both constancy-preserving and conservative; it was therefore

decided to make it constancy-preserving. Also, since it is only being used to compute the advection

for the corrector step, the expensive diffusion operations are not carried out during the predictor

step.

The preceeding notes on tracer advection refer to all but the MPDATA option. The MPDATA

algorithm has its own predictor-corrector with emphasis on not allowing values to exceed their

original range; it therefore gives up the constancy-preservation. This is most noticeable in shallow

areas with large tides.

4.8 Advection schemes

The advection of a tracer C has an equation of the form

∂ HzC ∂ ∂ ∂ = − Fξ − Fη − Fσ, (46) ∂t mn ∂ξ ∂η ∂σ

where we have introduced the advective fluxes:

HzuC Fξ = n (47)

Fη = HzvC

m (48)

Fσ = HzΩC . mn (49)

4.8.1 Second-order Centered

The simplest form of the advective fluxes is the centered second-order:
 

Fξ =

Fη =

Fσ =

Hz

ξ

uCξ

nξ

Hz

η

vCη

mη

Hz

σ

ΩCσ

mn .
 

(50)
 

(51)
 

(52)
 

This scheme is known to have some unfortunate properties in the presence of strong gradients,

such as large over- and under-shoots of tracers, leading to the need for large amounts of horizontal

smoothing. ROMS provides alternative advection schemes with better behavior in many situations,

but retains this one for comparison purposes.

24
 
 

 

 

� �

 

 

� �

� �

 

� �

� �

�

�

�

4.8.2 Fourth-order Centered

The barotropic advection is centered fourth-order unless you specifically pick centered second-order

as your horizontal advection scheme. To get fourth-order, create gradient terms:

ξ

Gξ = ∂C (53) ∂ξ

η

Gη = ∂C (54) ∂η σ

Gσ = ∂C . (55) ∂σ

The fluxes now become:

ξ

Fξ Hz ξ 1 ∂Gξ= u C − (56) ξ n 3 ∂ξ

η Hz η 1 ∂Gη

Fη = v C − (57) mη 3 ∂η σ Hz σ 1 ∂Gσ

Fσ = Ω C − . (58) mn 3 ∂σ

4.8.3 Fourth-order Akima

An alternate fourth-order algorithm is that by Akima:

∂C ∂C ∂C Gξ = 2∂C

+ (59) ∂ξ i ∂ξ i+1 ∂ξ i ∂ξ i+1

∂C ∂C ∂C Gη = 2∂C

+ (60) ∂η j ∂η j+1 ∂η j ∂η j+1

∂C ∂C ∂C Gσ = 2∂C

+ (61) ∂σ k ∂σ k−1 ∂σ k ∂σ k−1

(62)

With the fluxes as in 56–58.

4.8.4 Third-order Upwind

There is a class of third-order upwind advection schemes, both one-dimensional [Leonard, 1979]

and two-dimensional [Rasch, 1994, Shchepetkin and McWilliams, 1998]. This scheme is known

as UTOPIA (Uniformly Third-Order Polynomial Interpolation Algorithm). Applying flux limiters

to UTOPIA is explored in Thuburn [1996], although it is not implemented in ROMS. The twodimensional formulation in Rasch contains terms of order u2C and u3C, including cross terms

(uvC). The terms which are nonlinear in velocity have been dropped in ROMS, leaving one extra

upwind term in the computation of the advective fluxes:

Hzu ∂2C Fξ = C − γ (63) n ∂ξ2

Hzv ∂2C Fη = C − γ (64) m ∂η2

25
 
 

� � 

� � 

 

� � 

� � 

 

 

 � � � � � � � � 

The second derivative terms are centered on a ρ point in the grid, but are needed at a u or v point

in the flux. The upstream value is used:

ξ

Fξ = Hz [max(0, ui,j,k)Ci−1,j,k + min(0, ui,j,k)Ci,j,k] . (65) i,j,k ξ n

The value of γ in the model is 1

8 while that in Rasch [1994] is 16 .

Because the third-order upwind scheme is designed to be two-dimensional, it is not used in the

vertical (though one might argue that we are simply performing one-dimensional operations here).

Instead, we use a centered fourth-order scheme in the vertical when the third-order upwind option

is turned on:

Hzw 1 9 9 1 Fs = −16Ci,j,k−1 + 16Ci,j,k + 16Ci,j,k+1 − 16Ci,j,k+2 (66) mn

One advantage of UTOPIA over MPDATA is that it can be used on variables having both

negative and positive values. Therefore, it can be used on velocity as well as scalars. For the

u-velocity, we have:

∂2u Hzu ∂2 Hzu Fξ = u − γ − γ (67) ∂ξ2 n ∂ξ2 n

∂2 Hzv ∂ u 2 Hzv Fη = u − γ − γ (68) ∂η2 m ∂ξ2 m

Hzw 1 9 9 1 Fσ = −16

ui,j,k−1 + 16ui,j,k + 16ui,j,k+1 − 16ui,j,k+2 (69) mn

while for the v-velocity we have:

∂2 Hzu ∂ v 2 Hzu Fξ = v − γ − γ (70) ∂ξ2 n ∂η2 n

∂2v Hzv ∂2 Hzv Fη = v − γ − γ (71) ∂η2 m ∂η2 m

Hzw 1 9 9 1 Fσ = −16

vi,j,k−1 + 16vi,j,k + 16vi,j,k+1 − 16vi,j,k+2 (72) mn

In all these terms, the second derivatives are evaluated at an upstream location.

4.9 Determination of the vertical velocity and density fields

Having obtained a complete specification of the u, v, T, and S fields at the next time level by the

methods outlined above, the vertical velocity and density fields can be calculated. The vertical

velocity is obtained by combining equations (23) and (30) to obtain:

∂ Hzu ∂ Hzv ∂ Ω ∂ Du ∂ Dv

+ +

Hz − − = 0. (73) ∂ξ n ∂η m ∂σ mn ∂ξ n ∂η m

Solving for HzΩ/mn and using the semi-discrete notation of §4.4 we obtain:

ξ η ξ η HzΩ uD vD uHz vHz = δξ + δη − δξ − δη dσ. (74) mn ξ η ξ η n m n m

The integral is actually computed as a sum from the bottom upwards and also as a sum from the

top downwards. The value used is a linear combination of the two, weighted so that the surface

down value is used near the surface while the other is used near the bottom.

26
 
 � � � � � �

 

 � � � � � �

 

The density is obtained from temperature and salinity via an equation of state. ROMS provides

a choice of a nonlinear equation of state ρ = ρ(T, S, z) or a linear equation of state ρ = ρ(T). The

nonlinear equation of state has been modified and now corresponds to the UNESCO equation of

state as derived by Jackett and McDougall [1995]. It computes in situ density as a function of

potential temperature, salinity and pressure.

Warning: although we have used it quite extensively, McDougall (personal communication)

claims that the single-variable (ρ = ρ(T)) equation of state is not dynamically appropriate as is.

He has worked out the extra source and sink terms required, arising from vertical motions and the

compressibility of water. They are quite complicated and we have not implemented them to see if

they alter the flow.

4.10 Horizontal mixing

In Chapter 3, the diffusive terms were written simply as Du, Dv, DT , and DS. The vertical component of these terms is described in §4.12. Here we describe the ROMS options for representing the

horizontal component of these terms, first the viscosity then the diffusion.

4.10.1 Deviatory stress tensor (viscosity)

Note: this material was copied from the wiki, where it was contributed by Hernan Arango. He uses

“s” where we have been using “σ” while here σ is the stress tensor.

The horizontal components of the divergence of the stress tensor [Wajsowicz, 1993] in nondimesional, orthogonal curvilinear coordinates (ξ, η, s) with dimensional, spatially-varying metric factors

1 ( 1 , , Hz) and velocity components (u, v, ωHz) are given by: m n

Fu ≡ fξ · (' · -σ) = mn

Hz

∂

∂ξ

Hzσξξ

n

+

∂

∂η

Hzσξη

m

+

∂

∂s

σξs

mn

+ (75)

∂ Hzσξη ∂η

1

m

∂ − Hzσηη ∂ξ1

n

1 ∂Hz − σss n ∂ξ (76)

Fv ≡ ηf · (' · -σ) = mn

Hz

∂

∂ξ

Hzσηξ

n

+

∂

∂η

Hzσηη

m

+

∂

∂s

σηs

mn

+ (77)

∂ Hzσηξ ∂ξ

1

n

∂ − Hzσξξ ∂η1

m − 1 ∂Hz σss m ∂η (78)

where

σξξ = (AM + ν) eξξ + (ν − AM) eηη, (79)

σηη = (ν − AM) eξξ + (AM + ν) eηη, (80)

σss = 2 ν ess, (81)

σξη = σηξ = 2 AM eξη, (82)

σξs = 2 KM eξs, (83)

σηs = 2 KM eηs, (84)

and the strain field is:

27
 
 

 

� � 

� � 

 

 

 

 

∂u ∂ 1

eξξ = m + mnv ,	 (85) ∂ξ ∂η m

∂v ∂ 1

eηη = n + mnu ,	 (86) ∂η ∂ξ n

1 ∂ (ωHz) m ∂Hz n ∂Hz ess = + u + v ,	 (87) Hz ∂s Hz ∂ξ Hz ∂η

m ∂ (nv) n ∂ (mu) 2 eξη = + ,	 (88) n ∂ξ m ∂η

1 ∂ (mu) ∂ω 2 eξs = + mHz ,	 (89) mHz ∂s ∂ξ

1 ∂ (nv) ∂ω 2 eηs = + n Hz .	 (90) nHz ∂s ∂η

Here, AM(ξ, η) and KM(ξ, η, s) are the spatially varying horizontal and vertical viscosity coefficients, respectively, and ν is another (very small, often neglected) horizontal viscosity coefficient.

Notice that because of the generalized terrain-following vertical coordinates of ROMS, we need

to transform the horizontal partial derivatives from constant ”z-”surfaces to constant ”s-”surfaces.

∂z And the vertical metric or level thickness is the Jacobian of the transformation, Hz = . Also in ∂s

these models, the ”vertical” velocity is computed as ωHz and has units of m3/s. mn

4.10.2 Transverse stress tensor

Assuming transverse isotropy, as in Sadourny and Maynard [1997] and Griffies and Hallberg [2000],

the deviatoric stress tensor can be split into vertical and horizontal sub-tensors. The horizontal

(or transverse) sub-tensor is symmetric, it has a null trace, and it possesses axial symmetry in the

local vertical direction. Then, the transverse stress tensor can be derived from eq. (76) and (78),

yielding

Fuξ	 Fuη ∂ Hz ∂ Hz HzFu = n2m + m2n	 (91) ∂ξ	 n ∂η m

Fvξ Fvη ∂ Hz ∂ Hz HzFv = n2m + m2n	 (92) ∂ξ n ∂η m

where

Fuξ 1 m ∂ (nu) n ∂ (mv) = AM − ,	 (93) n n ∂ξ m ∂η

Fuη 1 n ∂ (mu) m ∂ (nv) = AM + ,	 (94) m m ∂η n ∂ξ

Fvξ 1 m ∂ (nv) n ∂ (mu) = AM + ,	 (95) n n ∂ξ m ∂η

Fvη 1 n ∂ (mv) m ∂ (nu) = AM − .	 (96) m m ∂η n ∂ξ

Notice the flux form of eq. (91) and (92) and the symmetry between the Fuξ and Fvη terms

which are defined at density points on a C-grid. Similarly, the Fuη and Fvξ terms are symmetric

28
 
� � � � � �

� � � � � �

 

 

 

 

 

 

 

 

and defined at vorticity points. These staggering positions are optimal for the discretization of the

tensor; it has no computational modes and satisfies first-moment conservation.

The biharmonic friction operator can be computed by applying the tensor operator eq. (91) and

(92) twice, but with the squared root of the biharmonic viscosity coefficient [Griffies and Hallberg,

2000]. For simplicity and momentum balance, the thickness Hz appears only when computing the

second harmonic operator as in Griffies and Hallberg [2000].

4.10.3 Rotated Transverse Stress Tensor

In some applications with tall and steep topography, it will be advantageous to substantially reduce

the contribution of the stress tensor eq. (91) and (92) to the vertical mixing when operating along

constant s-surfaces. The transverse stress tensor rotated along geopotentials (constant depth) is

then given by

Ruξ Ruη ∂ Hz ∂ Hz ∂ Ru 2 2 R H us z = n m + m n + (97) ∂ξ n ∂η m ∂s

Rvξ Rvη

Rv 2 ∂ Hz 2 ∂ Hz ∂ R H vs z = n m + m n + (98) ∂ξ n ∂η m ∂s

where

Ruξ 1 =n

AM

1

n

∂ (nu) m ∂ξ∂z − m∂ξ1

Hz

∂ (nu)

∂s − 1m

∂ (mv) n ∂η∂z − n∂η1

Hz

∂ (mv)

∂s , (99)

Ruη 1 = AM

m

1

m

∂ (mu) n ∂η∂z − n∂η1

Hz

∂ (mu)

∂s +

1

n

∂ (nv) m ∂ξ∂z − m∂ξ1

Hz

∂ (nv)

∂s , (100)

Rus ∂z =m AM ∂ξ

1

n

∂z

m∂ξ

1

Hz

∂ (nu)

∂s

∂ (nu) − m ∂ξ − 1

m

∂z

n∂η

1

Hz

∂ (mv)

∂s

∂ (mv) − n ∂η + (101)

n

∂z AM ∂η

1

m

∂z

n∂η

1

Hz

∂ (mu)

∂s

∂ (mu) − n ∂η +1

n

∂z

m∂ξ

1

Hz

∂ (nv)

∂s

∂ (nv) − m ∂ξ , (102)

Rvξ 1 =n

AM

1

n

∂ (nv) m ∂ξ∂z − m∂ξ1

Hz

∂ (nv)

∂s +

1

m

∂ (mu) n ∂η∂z − n∂η1

Hz

∂ (mu)

∂s , (103)

Rvη 1 = AM

m

1

m

∂ (mv) n ∂η∂z − n∂η1

Hz

∂ (mv)

∂s − 1n

∂ (nu) m ∂ξ∂z − m∂ξ1

Hz

∂ (nu)

∂s , (104)

Rvs ∂z =m AM ∂ξ

1

n

∂z

m∂ξ

1

Hz

∂ (nv)

∂s

∂ (nv) − m ∂ξ +1

m

∂z

n∂η

1

Hz

∂ (mu)

∂s

∂ (mu) − n ∂η + (105)

n

∂z AM ∂η

1

m

∂z

n∂η

1

Hz

∂ (mv)

∂s

∂ (mv) − n ∂η − 1

n

∂z

m∂ξ

1

Hz

∂ (nu)

∂s

∂ (nu) − m ∂ξ . (106)

Notice that the transverse stress tensor remains invariant under coordinate transformation. The

rotated tensor (97) and (98) retains the same properties as the unrotated tensor (91) and (92). The

additional terms that arise from the slopes of s-surfaces along geopotentials are discretized using a

modified version of the triad approach of Griffies et al. [1998].

4.10.4 Horizontal diffusion

In Chapter 3, the diffusive terms were written simply as DT and DS. The vertical component

of these terms is described in §4.12. Here we describe the various options for representing the

horizontal component of these terms.

29
 
 

 

 

 

4.10.5 Laplacian

The Laplacian of a scalar C in curvilinear coordinates is (see Appendix C):

∂ m ∂C ∂ n ∂C '2C = ' · 'C = mn + (107) ∂ξ n ∂ξ ∂η m ∂η

In ROMS, this term is multiplied by ν2Hz and becomes mn

∂ ν2Hzm ∂C ∂ ν2Hzn ∂C

+ (108) ∂ξ n ∂ξ ∂η m ∂η

where C is any tracer. This form guarantees that the term does not contribute to the volumeintegrated equations.

4.10.6 Biharmonic

'4 The biharmonic operator is = '2'2; the corresponding term is computed using a temporary

variable Y :

Y = mn

Hz

∂ ν4Hzm ∂C

∂ξ n ∂ξ +

∂ ν4Hzn ∂C

∂η m ∂η (109)

and is

∂ ν4Hzm ∂Y ∂ ν4Hzn ∂Y − + (110) ∂ξ n ∂ξ ∂η m ∂η

where C is once again any tracer and ν4 is the square root of the input value so that it can be

applied twice.

4.10.7 Rotated mixing tensors

Both the Laplacian and biharmonic terms above operate on surfaces of constant s and can contribute

substantially to the vertical mixing. However, the oceans are thought to mix along constant density

surfaces so this is not entirely satisfactory. Therefore, the option of using rotated mixing tensors

for the Laplacian and biharmonic operators has been added. Options exist to diffuse on constant

z surfaces (MIX_GEO_TS) and constant potential density surfaces (MIX_ISO_TS).

The horizontal Laplacian diffusion operator is computed by finding the three components of

the flux of the quantity C. The ξ and η components are locally horizontal, rather than along the

s surface. The diffusive fluxes are:

⎡ ⎤

⎢

⎢

⎢

⎢

⎢

⎢

⎢

⎣

m
 

⎥

⎥

⎥

⎥

⎥

⎥

⎥

⎦

⎛ ⎞

∂C
 ⎜⎝

m
 

∂z
 ∂C
 

O ∂z
 +Sx 

MIX_ISO 

MIX_GEO

⎟ F

⎠ (111) ξ = ν2 − ∂ξ ∂ξ
 

O 

⎡ ⎤

⎢

⎢

⎢

⎢

⎢

⎢

⎢

⎢

⎣

n
 ∂C
 

∂z
 

 

⎥

⎥

⎥

⎥

⎥

⎥

⎥

⎥

⎦

⎡ ⎞

⎢

⎢

⎣

n
 

∂z
 

∂η
 O

+Sy

 

MIX_ISO 

MIX_GEO

⎟⎟⎠

∂C
 

∂η
 F −
 (112) η = ν2

O

⎛ ⎞ ⎛ ⎞

1

⎜⎝

∂z
 ⎟⎠

Fξ 1 ⎜⎜⎝

∂z
 ⎟⎟⎠

F (113) η Fs = − O+Sx 

MIX_ISO

 

n
 ∂η
 O+Sy

 

MIX_ISO

 

m
 − Hz ∂ξ Hz

O O MIX_GEO MIX_GEO

30
� � 

 � � 

where
 

 ∂ρ m ∂z ∂ρ ∂ρ m∂ξ − Hz ∂ξ ∂s Sx = ∂x

∂ρ = 1 ∂ρ

∂z Hz ∂s

 ∂ρ ∂ρ n ∂z ∂ρ n − ∂y ∂η Hz ∂η ∂s Sy = = ∂ρ 1 ∂ρ

∂z Hz ∂s

and there is some trickery whereby the computational details depend on the sign of ∂z and of ∂z . ∂ξ ∂η

No flux boundary conditions are easily imposed by setting

Fξ = 0 at ξ walls

Fη = 0 at η walls

Fs = 0 at s = −1, 0

Finally, the flux divergence is calculated and is added to the right-hand-side term for the field

being computed:

∂ HzFξ ∂ HzFη ∂ HzFs

+ + (114) ∂ξ n ∂η m ∂s mn

The biharmonic rotated mixing tensors are computed much as the non-rotated biharmonic

mixing. We define a temporary variable Y based on equation (114):

mn ∂ ν4HzFξ ∂ ν4HzFη ∂ ν4HzFs

Y = + + . (115) Hz ∂ξ n ∂η m ∂s mn

We then build up fluxes of Y as in equations (111)–(113). We then apply equation (114) to these

Y fluxes to obtain the biharmonic mixing tensors. Again, the value of ν4 is the square root of that

read in so that it can be applied twice.

4.11 Vertical mixing schemes

ROMS contains a variety of methods for setting the vertical viscous and diffusive coefficients. The

choices range from simply choosing fixed values to the KPP, generic lengthscale (GLS) and MellorYamada turbulence closure schemes. See Large [1998] for a review of surface ocean mixing schemes.

Many schemes have a background molecular value which is used when the turbulent processes are

assumed to be small (such as in the interior). All assume that there is some Km(ζ, η, s) such that

the vertical turbulent mixing can be applied as:

∂u ∂v ' ' ' ' u w = −Km and v w = −Km (116) ∂z ∂z

with a similar Ks for temperature, salinity and other tracers. The primed quantities represent

perturbations of smaller scale than those resolved by the model while the unprimed u and v are

those in the model.

4.11.1 The Large, McWilliams and Doney parameterization

The vertical mixing parameterization introduced by Large et al. [1994] is a versatile first order

scheme which has been shown to perform well in open ocean settings. Its design facilitates experimentation with additional or modified representations of specific turbulent processes.

31
 
- -

Surface boundary layer The Large, McWilliams and Doney scheme (LMD) matches separate

parameterizations for vertical mixing of the surface boundary layer and the ocean interior. A

formulation based on boundary layer similarity theory is applied in the water column above a

calculated boundary layer depth hsbl. This parameterization is then matched at the interior with

schemes to account for local shear, internal wave and double diffusive mixing effects.

Viscosity and diffusivities at model levels above a calculated surface boundary layer depth

(hsbl ) are expressed as the product of the length scale hsbl, a turbulent velocity scale wx and a

non-dimensional shape function.

νx = hsblwx(σ)Gx(σ) (117)

where σ is a non-dimensional coordinate ranging from 0 to 1 indicating depth within the surface

boundary layer. The x subscript stands for one of momentum, temperature and salinity.

Surface Boundary layer depth The boundary layer depth hsbl is calculated as the minimum

of the Ekman depth, estimated as,

he = 0.7u∗/f (118)

 

(where u∗ is the friction velocity u∗ = τ 2 + τ 2/ρ ), the Monin-Obukhov depth: x y

L = u∗

3/(κBf ) (119)

(where κ = 0.4 is von Karman’s contant and Bf is the surface buoyancy flux), and the shallowest

depth at which a critical bulk Richardson number is reached. The critical bulk Richardson number

(Ric) is typically in the range 0.25–0.5. The bulk Richardson number (Rib) is calculated as:

(Br − B(d))d Rib(z) = - − - (120) |Vr V (d)|

2 + Vt

2(d)

where d is distance down from the surface, B is the buoyancy, Br is the buoyancy at a near surface

reference depth, V is the mean horizontal velocity, Vr the velocity at the near surface reference

depth and Vt is an estimate of the turbulent velocity contribution to velocity shear.

The turbulent velocity shear term in this equation is given by LMD as,

(−βT )1/2

V 2(d) = Cv (csE)

−1/2dNws (121) t Ricκ

where Cv is the ratio of interior N to N at the entrainment depth, βT is ratio of entrainment flux

to surface buoyancy flux, cs and E are constants, and ws is the turbulent velocity scale for scalars.

LMD derive (121) based on the expected behavior in the pure convective limit. The empirical rule

of convection states that the ratio of the surface buoyancy flux to that at the entrainment depth be

a constant. Thus the entrainment flux at the bottom of the boundary layer under such conditions

should be independent of the stratification at that depth. Without a turbulent shear term in the

denominator of the bulk Richardson number calculation, the estimated boundary layer depth is too

shallow and the diffusivity at the entrainment depth is too low to obtain the necessary entrainment

flux. Thus by adding a turbulent shear term proportional to the stratification in the denominator,

the calculated boundary layer depth will be deeper and will lead to a high enough diffusivity to

satisfy the empirical rule of convection.

Turbulent velocity scale To estimate wx (where x is m - momentum or s - any scalar)

throughout the boundary layer, surface layer similarity theory is utilized. Following an argument

by Troen and Mahrt [1986], Large et al. [1994] estimate the velocity scale as

wx = κu∗ (122) φx(ζ)

32
 
where ζ is the surface layer stability parameter defined as z/L. φx is a non-dimensional flux profile

which varies based on the stability of the boundary layer forcing. The stability parameter used in

this equation is assumed to vary over the entire depth of the boundary layer in stable and neutral

conditions. In unstable conditions it is assumed only to vary through the surface layer which is

defined as Ehsbl (where E is set at 0.10) . Beyond this depth ζ is set equal to its value at Ehsbl.

The flux profiles are expressed as analytical fits to atmospheric surface boundary layer data. In

stable conditions they vary linearly with the stability parameter ζ as

φx = 1 + 5ζ (123)

In near-neutral unstable conditions common Businger-Dyer forms are used which match with the

formulation for stable conditions at ζ = 0. Near neutral conditions are defined as

− 0.2 ≤ ζ < 0 (124)

for momentum and,

− 1.0 ≤ ζ < 0 (125)

for scalars. The non dimensional flux profiles in this regime are,

φm = (1 − 16ζ)

1/4 (126)

φs = (1 − 16ζ)

1/2 (127)

In more unstable conditions φx is chosen to match the Businger-Dyer forms and with the free

convective limit. Here the flux profiles are

φm = (1.26 − 8.38ζ)

1/3 (128)

φs = (−28.86 − 98.96ζ)

1/3 (129)

The shape function The non-dimensional shape function G(σ) is a third order polynomial

with coefficients chosen to match the interior viscosity at the bottom of the boundary layer and

Monin-Obukhov similarity theory approaching the surface. This function is defined as a 3rd order

polynomial.

G(σ) = ao + a1σ + a2σ2 + a3σ3 (130)

with the coefficients specified to match surface boundary conditions and to smoothly blend with

the interior,

ao = 0 (131)

a1 = 1 (132)

(hsbl) ∂xνx(h) νx(h)∂σwx(1) a2 = −2 + 3

νx + + (133) hwx(1) wx(1) hw2(1) x

(hsbl) ∂xνx(h) νx(h)∂σwx(1) a3 = 1 − 2

νx − (134) hwx(1) − wx(1) hw2(1) x

where νx(h) is the viscosity calculated by the interior parameterization at the boundary layer depth.

Countergradient flux term The second term of the LMD scheme’s surface boundary layer

formulation is the non-local transport term γ which can play a significant role in mixing during

surface cooling events. This is a redistribution term included in the tracer equation separate from

the diffusion term and is written as

− ∂ Kγ. (135) ∂z

33
 
 

LMD base their formulation for non-local scalar transport on a parameterization for pure free

convection from Mailhôt and Benoit [1982]. They extend this parameterization to cover any unstable surface forcing conditions to give

wT0 + wTR

γT = Cs wT (σ)h (136)

for temperature and

wS0

γS = Cs

wS(σ)h (137)

for salinity (other scalar quantities with surface fluxes can be treated similarly). LMD argue that

although there is evidence of non-local transport of momentum as well, the form the term would

take is unclear so they simply specify γm = 0.

The interior scheme The interior scheme of Large, McWilliams and Doney estimates the viscosity coefficient by adding the effects of several generating mechanisms: shear mixing, double-diffusive

mixing and internal wave generated mixing.

νx(d) = νs + νd + νw (138) x x x

Shear generated mixing The shear mixing term is calculated using a gradient Richardson

number formulation, with viscosity estimated as:

νs

x =

⎧

⎪⎪⎨

⎪⎪⎩

ν0 Rig < 0,

ν0[1 − (Rig/Ri0)2]

3 0 < Rig < Ri0, (139)

0 Rig > Ri0.

where ν0 is 5.0 × 10−3, Ri0 = 0.7.

Double diffusive processes The second component of the interior mixing parameterization represents double diffusive mixing. From limited sources of laboratory and field data LMD

parameterize the salt fingering case (Rρ > 1.0)

νd

s (Rρ) =

⎧

⎨

⎩

1 × 10−4[1 − (

(R

ρ

ρ

−

−

1

1

)2)3 for 1.0 < Rρ < R0 = 1.9, R0 ρ

0 otherwise.
 (140)
 

νd(Rρ) = 0.7νd (141) θ s

For diffusive convection (0 < Rρ < 1.0) LMD suggest several formulations from the literature

and choose the one with the most significant impact on mixing [Fedorov, 1988].

νd = (1.5−6)(0.909 exp(4.6 exp[−0.54(R−1 − 1)]) (142) θ ρ

for temperature. For other scalars,

νd(1.85 − 0.85R−1)Rρ for 0.5 <= Rρ < 1.0, θ ρ νd = (143) s νd0.15Rρ otherwise. θ

34
 
� � � � 

 

� � � � � � � � � �

 

 

 

 

 

Internal wave generated mixing Internal wave generated mixing serves as the background

mixing in the LMD scheme. It is specified as a constant for both scalars and momentum. Eddy

diffusivity is estimated based on the data of Ledwell et al. [1993]. While Peters et al. [1988] suggest

eddy viscosity should be 7 to 10 times larger than diffusivity for gradient Richardson numbers

below approximately 0.7. Therefore LMD use

2 −1 νw = 1.0 × 10 m −4m s (144)

2 −1 νw = 1.0 × 10−5m s (145) s

4.11.2 Mellor-Yamada

One of the more popular closure schemes is that of Mellor and Yamada [1974, 1982]. They actually

present a hierarchy of closures of increasing complexity. ROMS provides only the “Level 2.5” closure

with the Galperin et al. [1988] modifications as described in Allen et al. [1995]. This closure scheme

adds two prognostic equations, one for the turbulent kinetic energy ( 1

2 q2) and one for the turbulent

kinetic energy times a length scale (q2l).

The turbulent kinetic energy equation is:

D q ∂ ∂ q 2 2

− Kq = Ps + Pb − ξd (146) Dt 2 ∂z ∂z 2

where Ps is the shear production, Pb is the buoyant production and ξd is the dissipation of turbulent

kinetic energy. These terms are given by

2 2 ∂u ∂v Ps = Km + , (147) ∂z ∂z

Pb = −KsN2, (148)

3

ξd = q (149) B1l

where B1 is a constant. One can also add a traditional horizontal Laplacian or biharmonic diffusion

(Dq) to the turbulent kinetic energy equation. The form of this equation in the model coordinates

becomes

2 2 2 2 ∂ Hzq ∂ Hzuq ∂ Hzvq ∂ HzΩq ∂ Kq ∂q2

+ + + − = ∂t mn ∂ξ n ∂η m ∂s mn ∂s mnHz ∂s

2 2 3 2HzKm ∂u ∂v 2HzKs 2Hzq Hz + + N2 − + Dq. (150) mn ∂z ∂z mn mnB1l mn

The vertical boundary conditions are:

top (z = ζ(x, y,t)) HzΩ = 0 mn

K 2/3 h2 q ∂q2 B1 )

2 = τ ξ + (τ η

mnHz ∂s ρo s s h h ∂u ∂v 1 = τ ξ, τ H η zKm ∂z , ∂z ρo s s

N2 Q HzKs = ρocP

and bottom (z = −h(x, y)) HzΩ = 0 mn

B K 2/3 h2 q ∂q2 1 ξ η 2 = τ + τ mnHz ∂s ρo b b

h h ∂u ∂v 1 ξ η HzKm = τ , τ ∂z , ∂z ρo b b

HzKsN2 = 0

35
 
 

 

� � � � � � � � � �

 

 

There is also an equation for the turbulent length scale l:
 

D h ∂ ∂lq2 q3

lq2 − Kl = lE1(Ps + Pb) − W ̃ (151) Dt ∂z ∂z B1

where W ̃ is the wall proximity function:

2 l W ̃ = 1 + E2 (152) kL

1 1 L−1 = + (153) ζ − z H + z

The form of this equation in the model coordinates becomes

∂ Hzq2l ∂ Hzuq2l ∂ Hzvq2l ∂ HzΩq2l ∂ Kq ∂q2l

+ + + − = ∂t mn ∂ξ n ∂η m ∂s mn ∂s mnHz ∂s

H 3 z Hzq Hz lE1(Ps + Pb) − W ̃ + Dql. (154) mn mnB1 mn

where Dql is the horizontal diffusion of the quantity q2l. Both equations (150) and (154) are

timestepped much like the model tracer equations, including an implicit solve for the vertical

operations and options for centered second and fourth-order advection. They are timestepped with

a predictor-corrector scheme in which the predictor step is only computing the advection.

Given these solutions for q and l, the vertical viscosity and diffusivity coefficients are:

Km = qlSm + Kmbackground (155)

Ks = qlSh + Ksbackground (156)

Kq = qlSq + Kqbackground (157)

and the stability coefficients Sm, Sh and Sq are found by solving

Ss [1 − (3A2B2 + 18A1A2)Gh] = A2 1 − 6A1B−1 (158) 1

Sm [1 − 9A1A2Gh] − Ss Gh(18A2

1 + 9A1A2)Gh = A1 1 − 3C1 − 6A1B1

−1 (159)

l

2N2

Gh = min(− 2 , 0.028). (160) q

Sq = 0.41Sm (161)

The constants are set to (A1, A2, B1, B2, C1, E1, E2) = (0.92, 0.74, 16.6, 10.1, 0.08, 1.8, 1.33). The

quantities q2 and q2l are both constrained to be no smaller than 10−8 while l is set to be no larger

than 0.53q/N.

4.11.3 Generic length scale

[Umlauf and Burchard, 2003] have come up with a generic two-equation turbulence closure scheme

which can be tuned to behave like several of the traditional schemes, including that of Mellor and

Yamada §(4.11.2). This is known as the Generic Length Scale, or GLS vertical mixing scheme and

was introduced to ROMS in [Warner et al., 2005]. Its parameters are set in the ROMS input file.

The first of Warner et al.’s equations is the same as (146) with k = 1/2q2. Their dissipation is

given by:

0 )3+p/nk3/2+m/nψ−1/n E = (c (162) μ

36
 
 

 

 

where ψ is a generic parameter that is used to establish the turbulence length scale. The equation

for ψ is:

Dψ ∂ ∂ψ ψ = Kψ + (c1Ps + c3Pb − c2EFwall) (163) Dt ∂z ∂z k

Coefficients c1 and c2 are chosen to be consistent with observations of decaying homogeneous,

+ − isotropic turbulence. The parameter c3 has differing values for stable (c3 ) and unstable (c3 )

stratification. Also,

ψ = (c0 )pkml

n (164) μ

0 )3k3/2 l = (c E−1 (165) μ

Depending on the choice of the various parameters, these two equations can be made to solve

a variety of traditional two-equation turbulence closure models. The list of parameters is shown in

table 4.11.3 and is also given inside the comments section of the ROMS input file.

k − kl

ψ = k1l

1

k − E

0 )3k3/2l−1 ψ = (cμ

k − ω

0 )−1k1/2l−1 ψ = (cμ

gen

0 )2k1l−2/3 ψ = (cμ

p 0.0 3.0 -1.0 2.0

m 1.0 1.5 0.5 1.0

n 1.0 -1.0 -1.0 -0.67

σ KM k = Kk 1.96 1.0 2.0 0.8

σ KM ψ = Kψ1.96 1.3 2.0 1.07

c1 0.9 1.44 0.555 1.0

c2 0.52 1.92 0.833 1.22 −c3 2.5 -0.4 -0.6 0.1

+c3 1.0 1.0 1.0 1.0

kmin 5.0e-6 7.6e-6 7.6e-6 1.0e-8

ψmin 5.0e-6 1.0e-12 1.0e-12 1.0e-8

0cμ 0.5544 0.5477 0.5477 0.5544

Table 4: Generic length scale parameters. Note that Mellor-Yamada 2.5 is an example of a k − kl

scheme.

4.12 Timestepping vertical viscosity and diffusion

The Du, Dv, and DC terms in equations (18)–(20) represent both horizontal and vertical mixing

processes. The horizontal options were covered in §4.10. The model has several options for computing the vertical coefficients; these are described in §4.11. The vertical viscosity and diffusion

terms have the form:

∂ K ∂φ (166) ∂σ Hzmn ∂σ

where φ represents one of u, v, or C, and K is the corresponding vertical viscous or diffusive

coefficient. This is timestepped using a semi-implicit Crank-Nicholson scheme with a weighting of

0.5 on the old timestep and 0.5 on the new timestep. Specifically, the equation of motion for φ can

be written as:

∂(Hzφ) = mnRφ +

∂ K ∂φ (167) ∂t ∂σ Hz ∂σ

where Rφ represents all of the forcing terms other than the vertical viscosity or diffusion. Since we

want the diffusion term to be evaluated partly at the current timestep n and partly at the next

37
 
 � �

 

 

 

timestep n + 1, we introduce the parameter λ and rewrite equation (167) as:

∂(Hzφ)	 K ∂φn ∂ K ∂φn+1

= mnRφ + (1 − λ) ∂+ λ . (168) ∂t	 ∂σ Hz ∂σ ∂σ Hz ∂σ

The discrete form of equation (168) is:

Hn+1φn+1 − Hn φn

k zk k zk (1 − λ) Kk	 Kk−1 = mnRφ + 2 (φn

k+1 − φnk ) − (φnk − φnk−1) Δt Δs Hzk Hzk−1

λ Kk (φn+1 Kk−1 (φn+1 − φn+1 +	 2 k+1 − φn

k

+1) − k k−1) (169) Δs Hzk Hzk−1

where k is used as the vertical level index. This can be reorganized so that all the terms involving

φn+1 are on the left and all the other terms are on the right. The equation for φn+1 will contain k

terms involving the neighbors above and below (φn+1

k+1) which leads to a set of coupled k+1 and φn−1

equations with boundary conditions for the top and bottom. The general form of these equations

is:

Akφn+1 + Ckφn+1

k+1 + Bkφn+1 = Dk	 (170) k k−1

where the boundary conditions are written into the coefficients for the end points. In this case the

coefficients become:

A(1) = 0 (171)

λΔt Kk−1 A(2 : N) = −	 (172) Δσ2Hn+1 zk−1

λΔt K1 Hn+1 B(1) = +	 (173) z1 Δσ2Hn+1 z1

Hn+1 λΔt Kk λΔt Kk−1 B(2 : Nm) = + +	 (174) zk Δσ2Hn+1 Δσ2Hn+1 zk zk−1

Hn+1 λΔt KNm B(N) = +	 (175) zN Δσ2Hn+1 zNm

λΔt Kk C(1 : Nm) = −	 (176) Δσ2Hn+1 zk

C(N) = 0 (177)

Δt(1 − λ) K1 Δt D(1) = Hn φ1

n + Δt mnRφ1 + (φn

2 − φn	 (178) z1	 1 ) − τb Δσ2 Hz

n Δσ 1

D(2 : Nm) = Hn φn + Δt mnRφk +	 (179) zk k

Δt(1 − λ) Kk	 Kk−1 (φn

k+1 − φnk ) − (φnk − φnk−1)	 (180) Δσ2 Hn	 Hn zk	 zk−1

Δt(1 − λ) KNm	 Δt D(N) = Hn φN

n + Δt mnRφN −	 (φNn − φNmn ) + τs (181) zN Δσ2 Hz

n

Nm	 Δσ

This is a standard tridiagonal system for which the solution procedure can be found in any standard

reference, such as Press et al. [1986].

4.13 Lateral Boundary Conditions

ROMS comes with a variety of boundary conditions, including open, closed, and periodic. See

Marchesiello et al. [2001] for a more thorough exploration of the options. Some options require a

value for the boundary points from either an included analytic expression (§6.5) or from an external

NetCDF file. Here, φext represents the exterior value of a quantity φ.

38
 
�

4.13.1 Gradient boundary condition
 

This boundary condition is extremely simple and consists of setting the gradient of a field to zero

at the edge. The outside value is set equal to the closest interior value.

4.13.2 Wall boundary condition

ROMS now assumes a wall condition if no other boundary condition is chosen. This is a zero

gradient condition for tracers and the surface elevation and zero flow for the normal velocity. For

tangential velocities, the wall is treated as either no-slip or free-slip, depending on the value of

gamma2 chosen by the user.

4.13.3 Clamped boundary condition

Almost as simple is setting the boundary value to a known exterior value.

φ = φext (182)

4.13.4 Flather boundary condition

For the normal component of the barotropic velocity, one option is to radiate out deviations from

exterior values at the speed of the external gravity waves [Flather, 1976]:

ext − (ζ − ζext u = u ) g (183) D

The exterior values are often used to provide tidal boundary contitions to the barotropic mode.

However, there are times when only the tidal elevation is known. A reduced physics option is

available for estimating uext in that case.

4.13.5 Shchepetkin boundary condition

Similar to Flather in purpose is the Shchepetkin boundary condition [Mason et al., 2010]. As in

the Flather condition, it will use a reduced physics option if uext is not available.

4.13.6 Chapman boundary condition

The condition for surface elevation to be used with either the Flather or Shchepetkin momentum

boundary is that of [Chapman, 1985], assuming all outgoing signals leave at the shallow-water wave

speed of √gD.

∂ζ ∂ζ = ± gD (184) ∂t ∂ξ

The time derivative here can be handled either explicitly or implicitly in ROMS, depending on how

the term ∂ζ is evaluated. ∂ξ

4.13.7 Radiation boundary condition

In realistic domains, open boundary conditions can be extremely difficult to get right. There are

situations in which incoming flow and outgoing flow happen along the same boundary or even at

different depths at the same horizontal location. Orlanski [1976] proposed a radiation scheme in

which a local normal phase velocity is computed and used to radiate things out (if it is indeed going

out). This works well for a wave propagating normal to the boundary, but has problems when waves

approach the boundary at an angle. Raymond and Kuo [1984] have modified the scheme to account

39
 
 

 

 

for propagation in all three directions. In ROMS, only the two horizontal directions are accounted

for (with the recommended RADIATION_2D option):

∂φ ∂φ ∂φ = − φξ + φη ) (185) ∂t ∂ξ ∂η

where

F ∂φ

∂ξ φξ = h2 h2 (186) ∂φ ∂φ

∂ξ + ∂η

F ∂φ

∂η φη = h2 h2 (187) ∂φ ∂φ

∂ξ + ∂η

F = −∂φ (188) ∂t

These terms are evaluated at the closest interior point in a manner consistent with the time stepping

scheme used. The phase velocities are limited so that the local CFL condition is satisfied. They are

then applied to the boundary point using equation (185), again using a consistent time stepping

scheme. Raymond and Kuo give the form used for centered differencing and a leapfrog time step

while ROMS uses one-sided differences.

The radiation approach is appropriate for waves leaving the domain. A check is made to see

which way the phase velocity is headed. If it is entering the domain, a zero gradient condition is

applied unless an additional nudging option is also specified as described below.

4.13.8 Mixed radiation-nudging boundary condition

As described in [Marchesiello et al., 2001], ROMS has an option for providing radiation conditions

on outflow and nudging to a known exterior value on inflow. This is implemented as a variation

on the radiation condition, requiring two timescales: the inflow nudging timescale and the outflow

nudging timescale. These timescales are provided in the input to ROMS (§7.1.12).

40
 
� �

- -

5 Ice Model Formulation

The sea-ice component of ROMS is a combination of the elastic-viscous-plastic (EVP) rheology

[Hunke and Dukowicz, 1997, Hunke, 2001] and simple one-layer ice and snow thermodynamics with

a molecular sublayer under the ice [Mellor and Kantha, 1989]. It is tightly coupled, having the

same grid (Arakawa-C) and timestep as the ocean and sharing the same parallel coding structure

for use with MPI or OpenMP [Budgell, 2005].

5.1 Dynamics

The momentum equations describe the change in ice/snow velocity due to the combined effects of

the Coriolis force, surface ocean tilt, air and water stress, and internal ice stress: (189) and (190):

d ∂ζw (Aρihiu) = Aρihifv − Aρihig + A(τ x + τ x) + Fx (189) a w dt ∂x

d ∂ζw (Aρihiv) = −Aρihifu − Aρihig + A(τ y + τ y ) + Fy. (190) a w dt ∂y

In this model, we neglect the nonlinear advection terms as well as the curvilinear terms in the

internal ice stress. Nonlinear formulae are used for both the ocean-ice and air-ice surface stress:

-τa = ρaCa|V10|V10 (191)

Ca = 2

1

Cd [1 − cos(2π min(hi + .1, .5)] (192)

-τw = ρwCw|-vw − -v|(-vw − -v). (193)

The force due to the internal ice stress is given by the divergence of the stress tensor σ. The rheology

is given by the stress-strain relation of the medium. We would like to emulate the viscous-plastic

rheology of Hibler [1979]:

P

σij = 2ηE ̇ij + (ζ − η) ̇Ekkδij − 2

δij (194)

1 ∂ui ∂uj E ̇ij ≡ + (195) 2 ∂xj ∂xi

where the nonlinear viscosities are given by:

ζ = P

 (196) 1/2 2 (E11

2 + E222 )(1 + 1/e2) + 4e−2E122 + 2E11E22(1 − 1/e2)

η = ζ (197) 2 . e

Hibler’s ice strength is given by: −C(1−A) P = P∗Ahie (198)

while Overland and Pease [1988] advocate a nonlinear strength:

−C(1−A) P = P∗Ah2

i e (199)

in which P∗ now depends on the grid spacing. Both options are available in ROMS.

We would also like to have an explicit model that can be solved efficiently on parallel computers.

The EVP rheology has a tunable coefficient E (the Young’s modulus) which can be chosen to make

the elastic term small compared to the other terms. We rearrange the VP rheology:

1 η − ζ P

σij + σkkδij + δij = E ̇ij (200) 2η 4ηζ 4ζ

41
 
 

then add the elastic term:
 

1 ∂σij 1 η − ζ P + σij + σkkδij + δij = E ̇ij (201) E ∂t 2η 4ηζ 4ζ

Much like the ocean model, the ice model has a split timestep. The internal ice stress term is

updated on a shorter timestep so as to allow the elastic wave velocity to be resolved.

Once the new ice velocities are computed, the ice tracers can be advected using the MPDATA

scheme [Smolarkiewicz and Grabowski, 1990]. The tracers in this case are the ice thickness, ice

concentration, snow thickness, internal ice temperature, and surface melt ponds. The continuity

equations describing the evolution of these parameters (equations (202)–(204)) also include thermodynamic terms (Sh, Ss and SA), which will be described in §5.2:

∂Ahi ∂(uAhi) ∂(vAhi) = − − + Sh + Dh (202) ∂t ∂x ∂y
 

∂Ahs ∂(uAhs) ∂(vAhs) = − − + Ss + Ds (203) ∂t ∂x ∂y

∂A ∂(uA) ∂(vA) = − − + SA + DA 0 ≤ A ≤ 1. (204) ∂t ∂x ∂y

The first two equations represent the conservation of ice and snow. Equation (204) is discussed

in some detail in MK89, but represents the advection of ice blocks in which no ridging occurs as

long as there is any open water. The symbols used in these equations along with the values for the

constants are listed in Table 5.

Note that Hibler’s hI variable is equivalent to our Ahi combination - his hI is the average

thickness over the whole gridbox while our hi is the average thickness over the ice-covered fraction

of the gridbox.

5.1.1 Landfast ice

The Arctic ocean has many shallow shelves, on which landfast ice can form every winter. The

model as described thus far has no way to reproduce the landfast ice, but Lemieux et al. [2015]

describes a landfast ice parameterization. Basal stress terms τb are added to equations (189) and

(190) representing a bottom drag applied to the deepest ice keels. Included is a parameterization

of the ice thickness distribution so that even a single ice-category model can use the landfast ice

option. Figure 7 shows a cartoon of an ice floe with a keel.

Introducing the tunable parameters k1 and k2, plus the expression hc = Ahw/k1, the ucomponent of the basal stress due to the dragging of ice keels is computed as:

⎧

⎨0

h

if hw ≥ hc, τbu = (205)

⎩k2 u (hw − hc) exp−Cb(1−A) if hw < hc. |�v|+u0

where hw is the water depth. The expression for the v-component is similar.

5.2 Thermodynamics

The thermodynamics is based on calculating how much ice grows and melts on each of the surface,

bottom, and sides of the ice floes, as well as frazil ice formation [Mellor and Kantha, 1989]. Once

the ice tracers are advected, the ice concentration and thickness are timestepped according to the

terms on the right.

42
 
Variable Value Description

A(x, y,t)

Ca

Cd

Cw

(Dh, Ds, DA)

δij

E

e

Eij (x, y,t)

η(x, y,t)

f(x, y)

(Fx, Fy)

g

H

hi(x, y,t)

ho

hs(x, y,t)

M(x, y,t)

P(x, y,t)

(P∗, C)

(Sh, Ss, SA)

σij (x, y,t)

-τa

-τw

(u, v)

-(V10, -vw)

(ρa, ρw)

ζ(x, y,t)

ζw(x, y,t)

2.2 × 10−3

10 × 10−3

2

−2 9.8 m s

1 m

(2.75 × 104, 20)

(1.3 kg m−3, 1025 kg m−3)

ice concentration

nonlinear air drag coefficient

air drag coefficient

water drag coefficient

diffusion terms

Kronecker delta function

Young’s modulus

eccentricity of the elliptical yield curve

strain rate tensor

nonlinear shear viscosity

Coriolis parameter

internal ice stress

acceleration of gravity

Heaviside function

ice thickness of ice-covered fraction

ice cutoff thickness

snow thickness on ice-covered fraction

ice mass (density times thickness)

ice pressure or strength

ice strength parameters

thermodynamic terms

stress tensor

air stress

water stress

the (x, y) components of ice velocity -v

10 meter air and surface water velocities

air and water densities

nonlinear bulk viscosity

height of the ocean surface

Table 5: Variables used in the ice momentum equations.
 

Variable Value Description

hw

hrt

hrb

hd

li

Δx

β

k1

k2

u0

Cb

O(0.01)

8.0

15.0

5.0 × 10−5

20

water depth

ridge height above level ice

draft of keels below level ice bottom

draft of ice below sea surface

length of ice floe within grid cell

x-dimension of grid cell

ridged fraction of floe

tunable parameter

tunable parameter

small velocity

ice strength parameter

Table 6: Variables used in the landfast ice parameterization.

43
Δx

li

hl hrt

hd

hw hrb

βli

Figure 7: Cartoon of an ice floe with a deep keel.

Equations (202) and (204) become:

DAhi ρo = [A(Wio − Wai) + (1 − A)Wao + Wfr] (206) Dt ρi
 

DA ρo = [Φ(1 − A)Wao + (1 − A)Wfr] 0 ≤ A ≤ 1. (207) Dt ρihi

The term Ahi is the “effective thickness”, a measure of the ice volume. Its evolution equation

is simply quantifying the change in the amount of ice. The ice concentration equation is more

interesting in that it provides the partitioning between ice melt/growth on the sides vs. on the top

and bottom. The parameter Φ controls this and has differing values for ice melt and retreat. In

principle, most of the ice growth is assumed to happen at the base of the ice while rather more of

the melt happens on the sides of the ice due to warming of the water in the leads.

The heat fluxes through the ice are based on a simple one layer Semtner [1976] type model with

snow on top. The temperature is assumed to be linear within the snow and within the ice. The ice

contains brine pockets for a total ice salinity of 3.2 or the surface salinity, whichever is less. The

surface ocean temperature and salinity is half a dz below the surface. The water right below the

surface is assumed to be at the freezing temperature; a logarithmic boundary layer is computed

having the temperature and salinity matched at freezing.

Here, the W variables are the freeze or melt rates as shown in Fig. 8 and Table 7. The frazil

ice growth Wfr will be discussed further in §5.2.2—note that it contributes to changes in A as well

as to changes in hi. The other term that contributes to A is Wao. This term includes a factor Φ

which Mellor and Kantha set to different values depending on whether ice is melting or freezing:

Φ = 4.0 Wao ≥ 0 (208)

Φ = 0.5 Wao < 0 (209)

Similar to Eq. (206) is the snow equation:

DAhs = [A(Ws − Wsm)] (210) Dt

where Ws and Wsm are the snowfall and snow melt rates, respectively, in units of equivalent water.

Currently all snow and ice melt ends up in Wro, though melt ponds could be added with some

effort.

44
 
Variable Value Description

αw 0.10 shortwave albedo of water

αi 0.60, 0.65 shortwave albedo of wet, dry ice

αs 0.72, 0.85 shortwave albedo of wet snow

Ck snow correction factor

Cpi 2093 J kg−1 K−1 specific heat of ice

Cpo 3990 J kg−1 K−1 specific heat of water

Ew 0.97 longwave emissivity of water

Ei 0.97 longwave emissivity of ice

Es 0.99 longwave emissivity of snow

E(T, r) enthalpy of the ice/brine system

FT ↑ heat flux from the ocean into the ice

h* test for thick snow pushing ice surface under water

H ↓ sensible heat

Io part of (1 − αi)SW↓ shortwave radiation entering ice

iw fraction of the solar heating transmitted

through a lead into the water below

ki 2.04 W m−1 K−1 thermal conductivity of ice

ks 0.31 W m−1 K−1 thermal conductivity of snow

Li 302 MJ m−3 latent heat of fusion of ice

Ls 110 MJ m−3 latent heat of fusion of snow

LE ↓ latent heat

LW↓ incoming longwave radiation

m −0.054◦C coefficient in linear Tf (S) = mS equation

Φ contribution to A equation from freezing water

Qai heat flux out of the snow/ice surface

Qao heat flux out of the ocean surface

Qi2 heat flux up out of the ice

Qio heat flux up into the ice

Qs heat flux up through the snow

r 0 ≤ r ≤ 0.2 brine fraction in ice

ρi 910 m3/kg density of ice

Si 3.2 salinity of the ice

SW↓ incoming shortwave radiation

σ 5.67 × 10−8 W m−2 K−4 Stefan-Boltzmann constant

T0 temperature of the bottom of the ice

T1 temperature of the interior of the ice

T2 temperature at the upper surface of the ice

T3 temperature at the upper surface of the snow

Tf freezing temperature

Tmelt_i mSi melting temperature of ice

Tmelt_s 0◦ C melting temperature of snow

Wai melt rate on the upper ice/snow surface

Wao freeze rate at the air/water interface

Wfr rate of frazil ice growth

Wio freeze rate at the ice/water interface

Wro rate of run-off of surface melt water

Ws snowfall rate

Wsm snow melt rate

Table 7: Variables used in the ice thermodynamics
 

45
 
t t

t

_

 

 

_

 

 

 

Wao

Wro

Wai

Wfr Wio

Ws

Wsm

Figure 8: Diagram of the different locations where ice melting and freezing can occur.

















FT

FT

Qio

Qi2

Q Qao s

Qai

hi

hs

T0

T3

T2

T1

Figure 9: Diagram of internal ice temperatures and fluxes. The hashed layer is the snow.

Figure 9 shows the locations of the ice and snow temperatures and the heat fluxes. The temperature profile is assumed to be linear between adjacent temperature points. The interior of the

ice contains “brine pockets”, leading to a prognostic equation for the temperature T1.

The surface flux to the air is:

Qai = −H ↓ −LE ↓ −EsLW↓ −(1 − αs)SW↓ +Esσ(T3 + 273)4 (211)

The incoming shortwave and longwave radiations are assumed to come from an atmospheric model.

The formulae for sensible heat, latent heat, and outgoing longwave radiation are the same as in

Parkinson and Washington [1979] and are shown in Appendix E. The sensible heat is a function

of T3, as is the heat flux through the snow Qs. Setting Qai = Qs, we can solve for T3 by setting

T n+1

3 = T3

n + ΔT3 and linearizing in ΔT3. As in Parkinson and Washington, if T3 is found to be

above the melting temperature, it is set to Tmelt and the extra energy goes into melting the snow

or ice:

Qai − Qi2 Wai = ρoL3

(212)

L3 ≡ [E(T3, 1) − E(T1, r)] (213)

Note that L3 = (1 − r)Li plus a small sensible heat correction.

46
 
 

 

If there is no snow, there is an option to allow some of the incoming shortwave radiation

(Io = 0.17(1−αi)SW↓) to enter the ice and contribute to heating up the ice rather than to melting

the surface ice [Maykut and Untersteiner, 1971]. It essentially reduces Qi2 by the amount Io in

both equations (212) and (221).

Inside the ice there are brine pockets in which there is salt water at the in situ freezing temperature. It is assumed that the ice has a uniform overall salinity of Si and that the freezing

temperature is a linear function of salinity. The brine fraction r is given by

Sim

r = T1

The enthalpy of the combined ice/brine system is given by

E(T, r) = r(Li + CpoT) + (1 − r)CpiT (214)

Substituting in for r and differentiating gives:

∂E SimLi = − + Cpi (215) ∂T1 T1

2

Inside the snow, we have

Qs = ks (T2 − T3) (216) hs

The heat conduction in the upper part of the ice layer is

2ki QI2 = (T1 − T2) (217) hi

These can be set equal to each other to solve for T2

T3 + CkT1 T2 = (218) 1 + Ck

where

2kihs Ck ≡ . hiks

Substituting into (217), we get:

2ki (T1 − T3) Qs = QI2 = (219) hi (1 + Ck)

Note that in the absence of snow, Ck becomes zero and we recover the formula for the no-snow

case in which T3 = T2.

At the bottom of the ice, we have

2ki QI0 = (T0 − T1) (220) hi

The difference between QI0 and QI2 goes into the enthalpy of the ice:

ρihi

∂E

+ -v · 'E = QI0 − QI2 (221) ∂t

We can use the chain rule to obtain an equation for timestepping T1:

∂E ∂T1

ρihi + -v · 'T1 = QI0 − QI2 (222) ∂T1 ∂t

47
 
 

 

Variable Value Definition

b 3.14 factor

E ̇ evaporation

k 0.4 von Karman’s constant

Km vertical viscosity of seawater

ν 2 −1 1.8 × 10−6m s kinematic viscosity of seawater

P ̇ precipitation

Pr 13.0 molecular Prandtl number

Prt 0.85 turbulent Prandtl number

S internal ocean salinity

S0 surface salinity

Sc 2432.0 molecular Schmidt number

τio stress on the ocean from the ice

τao stress on the ocean from the wind

T internal ocean temperature

T0 surface temperature

uτ −1/2 friction velocity |τio|

1/2ρo

z0 roughness parameter

Table 8: Ocean surface variables
 

where
 

2ki (T1 − T3) QI0 − QI2 = (T0 − T1) − hi 1 + Ck

2ki T3 − (2 + Ck)T1 = (T0 + hi 1 + Ck

When the snow gets thick enough, it pushes the ice down to the point where seawater floods

onto the ice where it can possibly refreeze. The condition for such thick snow is h* > 0, where

h* = hs − (ρw − ρi)

hi (223) ρs

An optional model process is to convert the thick snow into ice directly:

hs = hs − ρih*

(224) ρw

h*

hi = hi + ρs (225) ρw

5.2.1 Ocean surface boundary conditions

The ocean receives surface stresses from both the atmosphere and the ice, according to the ice

concentration:

∂uw A 1 − A = τ x + τ x Km (226) io ao ∂z ρo ρo

∂vw A 1 − A

τ y τ K y m = + ao (227) ∂z ρo io ρo

where the relevant variables are in Table 8.

The surface ocean is assumed to be at the freezing temperature for the surface salinity (T0 =

mS0) in the presense of ice. We also have T and S at the uppermost computed ocean point

48
 
 

 

 

1

2 dz below the surface. In order to solve for T0 and S0, we assume a [Yaglom and Kader, 1974]

logarithmic boundary layer. The upper ocean heat flux is:

FT = −CTz (T0 − T) z → 0 (228) ρoCpo

where

uτ CTz = (229) Prtk−1 ln(−z/z0) + BT

z0uτ BT = b

1/2

P r2/3 (230) ν

Likewise, we have the following equation for the surface salt flux:

FS = −CSz (S0 − S) z → 0 (231)

where

uτ CSz = (232) Prtk−1 ln(−z/z0) + BS

z0uτ BS = b

1/2

Sc2/3 (233) ν

The ocean model receives the following heat and salt fluxes:

FT = AQio + (1 − A)Qao − WoLo (234)

FS = (Wo − AWro)(Si − S0) + (1 − A)So(P ̇ − E ̇ ) (235)

Wo ≡ AWio + (1 − A)Wao (236)

[Mellor and Kantha, 1989] describe solving simultaneously for the five unknowns Wo, T0, S0,

FT and FS. Instead, we use the old value of T0 to find Wio and therefore Wo. Using the new value

of Wio, solve for a new value of S0 and then find the new T0 as the freezing temperature for that

salinity:

Wio = 1 Qio + CpoCTz (To − T) (237) Lo ρo

CSzS + (Wro − Wio))Si S0 = CSz + Wro − Wio

(238)

5.2.2 Frazil ice formation

Following Steele et al. [1989], we check to see if any of the ocean temperatures are below freezing

at the end of each timestep. If so, frazil ice is formed, changing the local temperature and salinity.

The ice that forms is assumed to instantly float up to the surface and add to the ice layer there.

We balance the mass, heat, and salt before and after the ice is formed:

mw1 = mw2 + mi (239)
 

mw1 (CpwT1 + L) = mw2 (CpwT2 + L) + miCpiT2 (240)
 

mw1S1 = mw2S2. (241)
 

The variables are defined in Table 9. Defining γ = mi/mw2 and dropping terms of order γ2 leads

49
 
 � � 

 

Variable Value Definition

Cpi 1994 J kg−1 K−1 specific heat of ice

Cpw 3987 J kg−1 K−1 specific heat of water

γ mi/mw2 fraction of water that froze

L 3.16e5 J kg−1 latent heat of fusion

mi mass of ice formed

mw1 mass of water before freezing

mw2 mass of water after freezing

m −0.0543 constant in freezing equation

S1 salinity before freezing

S2 salinity after freezing

T1 temperature before freezing

T2 temperature after freezing

Table 9: Frazil ice variables
 

to:
 

L Cpi T2 = T1 + γ + T1 1 − (242) Cpw Cpw

S2 = S1(1 + γ). (243)

We also want the final temperature and salinity to be on the freezing line, which we approximate

as:

Tf = mS. (244)

We can then solve for γ: −T1 + mS1

γ = h . (245)

1 − Cpi LCpw + T1 C − mS1 pw

The ocean is checked at each depth k and at each timestep for supercooling. If the water is below

freezing, the temperature and salinity are adjusted as in equations (242) and (243) and the ice

above is thickened by the amount:

Δh = γkΔzk

ρw . (246) ρi

Note that Steele et al. [1989] include a compressibility term in equations (244) and (245), but we

use potential temperature instead of in situ temperature.

5.2.3 Differences from Mellor and Kantha

We have tried to modify the hakkis model to more closely follow MK89. However, there are also

ways in which we have deviated from it.

• Add advection of snow.

• Add lateral melting of snow when ice is melting laterally.

• Add various limiters:

– Ice concentration:Amin ≤ A ≤ 1.0, Amin = 1.e − 30.

– Ice thickness: hi ≥ 0.0.

– Snow thickness: hs ≥ 0.0.

– Brine fraction: r ≤ rmax, rmax = 0.2

50
 
6 Details of the Code

6.1 Directory structure

The directory structure is as shown in Fig. 10, with the ability to run the ocean alone or coupled to

atmospheric and/or wave models. If running just the ocean, the model can be run forward in time

(the nonlinear model) or as an adjoint, tangent linear, or representer model for data assimilation

purposes. This document describes the uncoupled forward model only, specifically the version used

for our domains containing sea ice and other changes from the main trunk code. Details are subject

to change without notice—check your own source code for specific details as they apply to you.

The directories shown here are:

Apps	 This directory contains a subdirectory for each of my personal applications and is not

in the trunk code. In fact, it is now a git submodule, i.e., a separate repository. Its

subdirectories contain files used by each application: the ROMS header file for setting

cpp definitions, the analytic formulations for fields computed in the model rather than

read from files (bottom heat flux of zero, for instance), and ASCII input files read by

ROMS on startup to set things such as forcing file names and model time-step. Some of

these applications are:

Arctic This is the curvilinear grid covering the whole Arctic ocean, with 5–6 km resolution off Alaska, coarser to the far side.

Beaufort This is shared code for two different Beaufort Sea domains, one curvilinear at

3 km, the other rectangular at 0.5 km.

Bering This is a 10 km grid of the Bering Sea, generated for coupling to WRF in the

COAWST branch, but also tested with CICE.

Circle This is a circular domain wave propagation problem with an analytic solution

used as a test problem [Lamb, 1932].

NEP	 This is the Northeast Pacific domain covering the waters off the west coast of

the US, from California to the Bering Sea. It is a rectangular domain at about

11 km resolution when viewed in a conformal conic projection with standard

latitudes of 40 and 60 N.

NWGOA This is a 1.5 km grid of the Northwest Gulf of Alaska. It gets boundary

conditions from the Northeast Pacific grid, but is not aligned with it.

Other unsupported applications are also here. The application-specific files included in

the main trunk ROMS are elsewhere.

Atmosphere This directory is under development, not currently distributed. If you want to

run with a coupled atmosphere or wave model, contact John Warner for access to the

COAWST branch or wait for the NUOPC coupling code to become public.

Compilers This contains makefile fragments as described in §G.3.

Data	 Directories under here contain example forcing, grid, and initial condition NetCDF files.

There is also a directory containing the headers of these files in the format produced by

ncdump (CDL).

Lib	 The ARPACK and MCT libraries are needed by the data assimilation codes and by the

coupled models, respectively.

makefile This is the standard ROMS makefile as described in §G.

51
 
Arctic/
 

Apps/
 Bering/
 

Atmosphere/
 Circle/ Adjoint/
 

Compilers/
 NEP/ Bin/
 

Data/
 .../ Drivers/
 

Lib/
 External/

makefile Functionals/

Master/ Include/

License_ROMS.txt

Modules/

README

README.CICE

ROMS/ Nonlinear/ Biology/

SeaIce/ Obsolete/ Sediment/

User/ Programs/
 

Waves/
 Representer/

SeaIce/

Tangent/

Utility/

Version

Figure 10: ROMS directory structure.

Master	 The ROMS main program is here, in various forms for the forward model, coupled models

and others. See §6.2.

README A few words about the code, recommended for all github projects.

README.CICE A few words about the “fake” (direct) coupling to the Los Alamos CICE model.

ROMS These files are for the ocean model, as opposed to other components of the coupled system.

Adjoint This is the adjoint of the forward model, for data assimilation.

Bin	 Various shell and Perl scripts for use with the model. Note that the .sh files are

actually csh scripts, not sh scripts.

Drivers	 The main program includes one of these files, depending on how you are running

the model. The forward model is in nl_ocean.h.

External ROMS reads an ASCII file on startup. Here are examples for various applications, also examples of the optional files for extra components such as a sediment

model or a stations file.

Functionals The file analytical.F can include one or more code bits for the analytic

specification of for instance the initial conditions. Here are examples for the

supported model test problems.

Include	 Each application has a header file with C preprocessor options for that application. For instance, the UPWELLING case has the include file upwelling.h

containing C preprocessor options for its periodic channel domain. The full

52
 
list of available options is in cppdefs.h. Another important include file is

globaldefs.h, which can define some helper C preprocessor tags.

License_ROMS.txt The open source license under which ROMS is copyrighted.

Modules The ROMS data structures are now in Fortran 90 module files, located here.

Nonlinear The routines used by the nonlinear forward model are here, implementing the

physics described in §4.

Biology The files for the ecosystem parts of the forward model are here.

Sediment The files for the sediment parts of the forward model are here.

Obsolete Long unused versions of the boundary conditions are stored here.

Programs Not all computer architectures or compilers are the same. The types.F program checks your compiler for the sizes of the Fortran floating point types.

Representer This is the representer of the forward model, for data assimilation.

SeaIce The sea ice model described in §5 is here.

Tangent This is the tangent linear of the forward model, for data assimilation.

Utility Here are utility functions used by the various ROMS routines, many dealing with

I/O.

Version	 A file containing the time and date of this svn revision, also the svn URL—if

and only if you check out with svn. At least the date in the file is kept current

by Hernan. “git log” can show you the svn history, in commits such as:

commit b4e77b3a45208ee8059ca7b83e038371a60bc0d0

Author: arango <arango@f091316a-d328-0410-a40a-876eff57d070>

Date: Fri Jun 23 23:22:54 2017 +0000

src:ticket:733

git-svn-id: https://www.myroms.org/svn/src/trunk@851

f091316a-d328-0410-a40a-876eff57d070

where the commit is associated with trac ticket 733 and svn revision 851.

SeaIce	 This is code for coupling to the Los Alamos CICE model with some tips in README.CICE.

It works, although the coupling is inefficient. The Norwegians have also been working on

this problem and they have released the metroms coupling using MCT on github.

User	 Some might choose to use this directory rather than the Apps directory. It serves the

same purpose but is arranged by file type rather than by application.

Waves	 The SWAN wave model is here.

6.2 Main subroutines

6.2.1 master.F

The main program is in master.F. It is simply a shell, including one of mct_coupler.h,

esmf_coupler.h or ocean.h. In our case, ocean.h contains the actual main program, which

initializes MPI (if needed), calls ROMS_initialize, calls ROMS_run with an argument for how

long to run for, then ROMS_finalize, and finally wraps up the MPI. See Fig. 11.

53
 
mpi_init initialize_parallel

ROMS_initialize inp_par main3d or main2d

ROMS_run wclock_on

ROMS_finalize mod_arrays if (trouble)wrt_rst

mpi_finalize initial wclock_off

close_out

Figure 11: ROMS main structure.

6.2.2 ocean_control.F

This is again a shell which includes one of many other files to do the actual work. In this case, the

worker files all contain ROMS_initialize, ROMS_run and ROMS_finalize and live in the

ROMS/Drivers directory. The driver file we will be looking at is nl_ocean.h.

6.2.3 ROMS_initialize

This is called at the beginning of the run and therefore starts off by finding out how many parallel

processes are running and which one this is, then calls the following ROMS routines:

initialize_parallel In the mod_parallel module, set up a few variables, including some for the

built-in profiling.

inp_par Call read_phypar to read in the ASCII input file(s) used by ROMS, set up the parallel

tiles, then call routines to read in the rest of the ASCII input for biology, ice, etc.

wclock_on In timers.F, initialize a timer for the built-in profiling.

mod_arrays Allocate and initialize the dynamically sized arrays in ROMS based on the grid sizes

read in by inp_par.

initial	 Read in the initial conditions from one or more NetCDF files (one per grid) or compute

them analytically. Likewise for the grid, plus set up the vertical grid spacing to be used

and many other details.

6.2.4 ROMS_run

Call one of these two routines with the RunInterval argument:

main3d	 or main2d Solve the full equations described in §4 (main3d) or the depth-integrated

version only (main2d).

6.2.5 ROMS_finalize

This is called at the end of the run, whether it was otherwise successful or not. The routines called

are:

wrt_rst If the run had an error code set, write out a restart record of the current model fields in

case they are useful in diagnosing the trouble.

wclock_off End the built-in timers and cause them to print out a report.

close_io Close all open files so as to flush the buffers and put NetCDF files into a finished state.

54
 
6.3 Initialization

checkdefs Report on which C preprocessor variables have been #defined and check their consistency (called from inp_par).

ana_grid Compute the grid(s) internally.

ana_mask Compute the land mask internally.

get_grid Read in the curvilinear coordinate arrays as well as f and h from one NetCDF file per

grid.

set_scoord Set and initialize relevant variables associated with the vertical transformation to

nondimensional σ-coordinate described in Appendix B.

set_weights Set the barotropic time-step average weighting function.

metrics	 Compute the metric term combinations which do not depend on the surface elevation and

therefore remain constant in time.

ini_strengthcoef Compute a field for the nonlinear ice strength option [Overland and Pease,

1988].

ana_wtype Compute the Jerlov water type field internally.

ana_nudgcoef Compute the nudging time scales.

get_nudgcoef Read the nudging time scales from a file.

ini_hmixcoef Initialize the horizontal mixing coefficients.

ana_sponge Compute the horizontal mixing coefficients internally.

ana_initial Analytic initial conditions for momentum and active tracers.

ana_passive Analytic initial conditions for passive tracers.

ana_biology Analytic initial conditions for ecosystem tracers.

ana_sediment Analytic initial conditions for sediment tracers.

ana_ice	 Analytic initial conditions for ice variables.

get_state Read initial fields from disk—either restart or from some other source which has been

converted to the appropriate NetCDF format.

get_wetdry or wetdry Initialize wet/dry masks.

set_depth Compute time-evolving depths.

set_massflux Compute initial horizontal mass fluxes.

omega	 Compute initial vertical velocities.

rho_eos Compute initial density fields.

ana_psource Set up analytic point sources.

check_multifile Read input files in which one or more file is specified to find out time-ranges,

etc.

55
 
<loop> set_vbc

Figure 12: Flow chart of the model main program in forward mode. Calls to nesting for nested

grid applications have been left out.

get_idata Read in time-invariant forcing data.
 

get_data Read in the first record of time-varying forcing fields, boundary conditions, etc.
 

set_masks Compute I/O masks.
 

ana_drag Compute bottom drag coefficients internally.
 

stiffness Compute grid stiffness.
 

grid_coords Convert initial float and station locations to fractional grid coordinates.
 

nesting Perform inter-grid communications.
 

6.3.1 main3d

This solves the full three-dimensional equations described in §4. It has siblings main2d for solving

the depth-integrated equations and main3d_offline for reading files from a prior simulation and

using them to advect the biological tracers or the Lagrangian floats. The full version is shown

in Fig. 12, where the outer loop is over both timesteps and grids. Note that many subroutines

are optional and only get called if the appropriate C preprocessor switches have been set. The

subroutines are described as follows:

_ _

ntimesteps set_tides

get_data First rst step only:

set_data ice_flux_rst

First step only: seaice

ini_zeta ana_vmix

set_depth lmd_vmix

ini_fields bvf_mix

set_massflux hmixing

rho_eos omega

diag wvelocity

radiation_stress set_zeta

cawdir_eval set_diags

ana_albedo set_filter

albedo_eval set_avg

ccsm_flux set_avg2

bulk_flux output

ncep_flux Exit if time

bblm is reached

rhs3d

my25_prestep

gls_prestep

<loop>

step2d

step2d

set_depth

step3d_uv

omega

my25_corstep

gls_corstep

biology

sediment

step3d_t

step_floats

56
 
ntimesteps Find out how many timesteps to take on each grid.
 

get_data Read in the second and subsequent records of time-varying forcing fields, boundary
 

conditions, etc.

set_data Time interpolate between the records read in by get_data.

ini_zeta Check for wet/dry cells if needed and initialize all the time levels of zeta.

ini_fields Initialize the 2-D velocities to match the vertical integral of the 3-D velocities, making

all the time levels match.

set_massflux Compute horizontal mass fluxes, Hzu and Hzv . n m

rho_eos Compute the nonlinear equation of state.

diag	 Compute some global sums, print them, and check them to see if they are sensible. If

not, stop the model run.

radiation_stress Compute the radiation stresses due to wave-current interactions [Mellor, 2003,

2005].

cawdir_eval , ana_albedo or albedo_eval Compute the albedo at the ice and ocean surface.

ccsm_flux Compute the surface fluxes from the atmosphere based on a marine boundary layer.

This version comes from CCSM [Large and Yeager, 2009] and is reputed to do better

outside of the tropics than the default bulk flux computation.

bulk_flux Compute the surface fluxes from the atmosphere based on a marine boundary layer.

This version comes from COARE version 3.0 [Fairall et al., 2003, Taylor and Yelland,

2001, Oost et al., 2002].

ncep_flux Compute the surface fluxes from the NCEP atmospheric model.

bblm	 Compute the bottom stresses from one of three bottom boundary layer models.

set_vbc Compute the surface and bottom fluxes and stresses that aren’t computed elsewhere—set

vertical boundary conditions.

set_tides Compute the tidal boundary conditions from the tidal constituents.

seaice	 Run the sea ice model described in §5. It changes the surface boundary conditions for the

ocean and therefore gets called before the call to output or anything else that would be

needing the surface boundary conditions. In the case of a perfect restart, ice_flux_rst

is called instead on the first timestep—this restores surface fluxes from the restart file.

ana_vmix Called if there’s an analytic profile for the vertical mixing coefficient.

lmd_vmix Called when using the K-profile parameterization of vertical mixing [Large et al., 1994,

Large, 1998].

bvf_mix Compute the vertical mixing as a function of the Brunt-Väisälä frequency.

hmixing	 Compute time-dependent horizontal mixing coefficients [Smagorinsky, 1963, Holland

et al., 1998, Webb et al., 1998, Griffies and Hallberg, 2000].

omega Compute the Ω vertical velocity from the horizontal divergences.

57
 
wvelocity Compute the physical vertical velocity for the model output.
 

set_zeta Set the surface elevation to the time-mean over the last baroclinic time-step.
 

set_diags Accumulate the time-average of the diagnostics fields.
 

set_filter Accumulate a weighted sum using a Lanczos filter for detiding the most important of
 

the output fields. Not in the trunk code.

set_avg Accumulate time-averaged fields for the averages output.

set_avg2 Accumulate the time-averaged surface fields for the second averages output. Not in the

trunk code.

output Write to various output NetCDF files.

rhs3d Call pre_step3d to compute a predictor step on the three-dimensional variables, then

compute right-hand-sides of the three-dimensional velocity fields for the corrector step,

including pressure gradients.

my25_prestep Compute the predictor step for turbulent kinetic energy prognostic variables, tke

and gls.

gls_prestep Compute the predictor step for turbulent kinetic energy prognostic variables, tke

and gls.

step2d Compute the depth-integrated time-step. It is called in a loop over all the short timesteps, first as a predictor step, then as a corrector step.

step3d_uv Complete the time-step for the three-dimensional velocities.

omega Compute the Ω vertical velocity.

my25_corstep Perform the corrector step for turbulent kinetic energy and length scale prognostic

variables, tke and gls [Mellor and Yamada, 1982, Galperin et al., 1988].

gls_corstep Perform the corrector step for turbulent kinetic energy and length scale prognostic

variables, tke and gls [Umlauf and Burchard, 2003, Warner et al., 2005].

biology Compute the changes to the biological tracers due to biological activity using one of

several options for the ecosystem model.

sediment Compute changes to the sediment tracers [Warner et al., 2008].

step3d_t Complete the tracer time-step.

ice_frazil Compute the frazil ice growth, if any. Now called from inside step3d_t.

step_floats Time-step the Lagrangian floats.

6.4 Modules

Now that we are using Fortran 90, the method of choice for managing data structures is modules.

The ROMS/Modules directory contains all of the ROMS modules that contain globally used

variables. The complete list is:

mod_arrays.F This actually has no data structures, but has the routine that calls the allocate

and initialize routines for all the others.

58
 
mod_average.F If AVERAGES is defined, this will provide the storage for the running means

of the fields you are averaging.

mod_average2.F If AVERAGES2 is defined, this will provide the storage for the surface running means of the fields you are averaging.

mod_bbl.F If BBL_MODEL is defined, this will provide the storage for the bottom boundary

fields.

mod_behavior.F If FLOAT_BIOLOGY is defined, this will include a file for adding behavior

to floats, such as oysters sinking to the bottom.

mod_biology.F If BIOLOGY is defined, this will provide the storage for the biology interaction

parameters.

mod_boundary.F This contains the storage for the open boundary conditions. If they aren’t

provided analytically, this will also provide the storage for fields read from a file that need

to be time-interpolated.

mod_clima.F If one of LsshCLM or several other options is set to .true., this will provide the

storage for the climatology fields.

mod_coupler.F If either MODEL_COUPLING or ESMF_LIB is defined, this will set up

the requisite fields and data structures for the coupling.

mod_coupling.F If SOLVE3D is defined, this will provide the storage for the fields used in

coupling the 2-D and 3-D components of the simulation.

mod_diags.F If DIAGNOSTICS is defined, this will provide the storage for the various tendency terms.

mod_eclight.F If both BIOLOGY and ECOSIM are defined, this will set up the spectral

irradiance variables.

mod_eoscoef.F If NONLIN_EOS is defined, this will provide the polynomial expansion coefficients for the nonlinear equation of state for sea water.

mod_filter.F If FILTERED is defined, this will provide the storage for the weighted means

used in detiding the averages. Not in the trunk code.

mod_floats.F If FLOATS is defined, this will provide the storage for the float tracking variables.

mod_forces.F This provides the storage for the surface and bottom forcing fields.

mod_fourdvar.F If either FOUR_DVAR or VERIFICATION is defined, this will set up

the variational data assimilation variables.

mod_grid.F This provides the storage for the model grid fields.

mod_ice.F If ICE_MODEL or CICE_MODEL is defined, this will provide storage for the

ice fields.

mod_iounits.F This contains a number of variables used by the I/O, including file names and

file IDs.

mod_kinds.F This contains the integers associated with the various integer and real Fortran

types. If you find more systems supporting 128-bit reals, let us know.

59
 
mod_mixing.F This contains the arrays for the various optional horizontal and vertical mixing

parameterizations.

mod_ncparam.F This contains all sorts of parameters relating to the NetCDF I/O files, including those read from the varinfo.dat file. The parameters MV and NV are set here,

giving the maximum number of variables that can be read [this is a change from the

trunk code].

mod_nesting.F If NESTING is defined, this module defines generic structures used for nesting,

composed, and mosaic grids. Still in development, but some cases do now work.

mod_netcdf.F This brings in netcdf.mod and defines wrapper functions for many of the netcdf

functions.

mod_ocean.F This contains the 2-D and 3-D fields of the primitive ocean variables and optionally

the sediment variables.

mod_parallel.F This sets up some global variables such as Master, which is true for the master

thread or process. It also initializes the internal ROMS profiling arrays.

mod_param.F This contains the sizes of each grid used, plus things like how many tidal constituents are being used. Many of these are read from the input files during initialization,

not known at compile time.

mod_scalars.F This contains a large number of scalars, i.e. values which don’t have spatial

dependence. Some are fixed constants such as itemp referring to the temperature tracer.

Others could have a different value on each grid.

mod_sedbed.F If either SEDIMENT or BBL_MODEL is defined, this contains parameters

for the sediment bed layers.

mod_sediment.F If either SEDIMENT or BBL_MODEL is defined, this contains parameters for the respective model.

mod_sources.F This contains the variables used for point sources, only allocated if one of

LuvSrc, LwSrc or LtracerSrc is set to .true.

mod_stepping.F This contains the time-stepping variables used to point to the relevant time

level.

mod_storage.F If PROPAGATOR is defined, this module defines the work space for the

Generalized Stability Theory (GST) Analysis package (ARPACK).

mod_strings.F This contains strings such as a title for the run, the list of cpp options defined,

and the names of the sections of code being profiled.

mod_tides.F If SSH_TIDES and/or UV_TIDES is defined, this will provide the storage for

the tidal constituents.

mod_trc_sources.F If TRC_PSOURCE is defined, this contains the variables used for tracer

point sources. Not in the trunk code.

60
 
6.5 Functionals

The Functionals directory contains analytical.F which conditionally includes code bits for computing analytic values for a wide variety of fields. Many are alternates for reading from NetCDF

files, especially for idealized problems.

ana_aiobc Provide analytic open boundary conditions for the ice concentration.
 

ana_albedo Compute analytic surface albedo.
 

ana_biology Provide analytic initial conditions for the biology tracers.
 

ana_btflux Compute analytic kinematic bottom flux of tracer type variables (default of zero).
 

ana_cloud Provide analytic cloud fraction.
 

ana_diag Compute customized diagnostics.
 

ana_dqdsst Provide analytic variation of surface heat flux as a function of SST.
 

ana_drag Compute customized bottom drag coefficients.
 

ana_fsobc Provide analytic open boundary conditions for the free surface.
 

ana_grid Set up an analytic grid.
 

ana_hiobc Provide analytic open boundary conditions for the ice thickness.
 

ana_hsnobc Provide analytic open boundary conditions for the snow thickness.
 

ana_hsnobc Provide analytic open boundary conditions for the snow thickness.
 

ana_humid Provide analytic atmospheric humidity.
 

ana_ice Provide analytic initial conditions for the sea ice.
 

ana_initial Set up analytic initial conditions for the ocean.
 

ana_lrflux Provide analytic kinematic surface downward longwave radiation.
 

ana_m2clima Set up an analytic climatology for the two-dimensional momentum.
 

ana_m2obc Provide analytic open boundary conditions for the two-dimensional momentum.
 

ana_m3clima Set up an analytic climatology for the three-dimensional momentum.
 

ana_m3obc Provide open boundary conditions for the three-dimensional momentum.
 

ana_mask Set up an analytic mask.
 

ana_ncep Set up analytic fields as if they came from NCEP.
 

ana_nudgcoef Set up spatially dependent nudging coefficients for nudging to a climatology.
 

ana_pair Provide analytic sea-level air pressure.
 

ana_passive Provide analytic initial conditions for passive tracers.
 

ana_perturb Provide analytic perturbations to the initial conditions.
 

ana_psource Provide analytic point source fluxes.
 

61
 
ana_rain Provide analytic rainfall.
 

ana_scope Set adjoint sensitivity spatial scope masking arrays.
 

ana_sediment Provide analytic initial conditions for the sediment tracers.
 

ana_smflux Provide analytic kinematic surface momentum flux (wind stress).
 

ana_snow Provide analytic snowfall.
 

ana_specir Set surface solar downwelling spectral irradiance at just beneath the sea surface.
 

ana_spinning Set time-variable rotation force as the sum of Coriolis and Centripetal accelerations. This is used in polar coordinate applications (annulus grid).

ana_sponge Provide spatially variable horizontal mixing coefficients.

ana_srflux Provide analytic kinematic surface shortwave radiation.

ana_ssh Provide analytic sea surface height.

ana_sss Provide analytic sea surface salinity.

ana_sst Provide analytic sea surface temperature.

ana_stflux Provide analytic kinematic surface flux of tracer type variables.

ana_tair Provide analytic air temperature.

ana_tclima Provide analytic tracer climatology fields.

ana_tobc Provide analytic open boundary conditions for all tracers (active, passive, biology, and

sediment).

ana_vmix Provide analytic vertical mixing coefficients.

ana_winds Provide analytic winds.

ana_wwave Provide analytic wind-induced wave amplitude, direction and period.

6.6 Other subroutines and functions

The ROMS/Utility directory contains an assortment of useful routines, many of which deal with

I/O:

NetCDF I/O The I/O has been cleaned up so that all processes check for the return code on

reads and writes and exit on failure. In many cases, the master process is the only one

actually doing the I/O, but it then broadcasts its status to the rest.

def_*	 Create the ROMS NetCDF file of the appropriate type, including dimensions,

attributes, and variables.

def_info Add some standard scalar variables to any NetCDF file.

get_*fld Read a field from a netcdf file, perhaps using one of nf_fread2d and its kin.

nf_fread2d is special in that it can read uniformly gridded forcing files and call

regrid to regrid them onto the ROMS grid.

wrt_*	 Write to the ROMS NetCDF file of the appropriate type.

62
 
Text input ROMS can parse text files in a specific format. These files replace what many other

models use namelists for.

read_*	 The ocean_xx.in file is read by read_phypar while others read the stations,

floats, and other text files.

6.7 C preprocessor variables

Before it can be compiled, the model must be run through the C preprocessor cpp, as described

in Appendix F. The C preprocessor has its own variables, which may be defined either with an

explicit #define command or with a command line option to cpp. We have chosen to define these

variables in an application-specific include file, except for some machine-dependent ones, which are

defined in the makefile. These variables allow you to conditionally compile sections of the code.

For instance, if MASKING is not defined, the masking code will not be seen by the compiler, and

the masking variables will not be declared.

The top of each Fortran file contains #include “cppdefs.h”. This file will include the userprovided file of cpp options and then include globaldefs.h. This latter will set some internal

ROMS cpp variables for you, depending on choices you have made. For instance, if you don’t

provide analytic surface forcing fields by one means or another, globaldefs.h will set FRC_FILE

and ROMS will attempt to read a forcing file. Some combinations of options don’t make sense, so

checkdefs will complain and cause the run to end if it finds any incompatibilities in your setup.

The exact list of user-selectable cpp variables will depend on which ROMS branch you have.

Those listed below are in the sea-ice branch on github. They can be grouped into several categories:

Momentum terms The default horizontal advection is 3rd-order upstream bias for 3D momentum and 4th-order centered for 2D momentum. The default vertical advection is 4th-order

centered for 3D momentum. If this is what you want, no flags for momentum advection

need to be activated except for UV_ADV.

The 3rd-order upstream split advection (UV_U3ADV_SPLIT) can be used to correct
 

for the spurious mixing of the advection operator in terrain-following coordinates. If this
 

is chosen, the advection operator is split into advective and viscosity components and
 

several internal flags are activated in globaldefs.h. Notice that horizontal and vertical
 

advection of momentum is 4th-order centered plus biharmonic viscosity to correct for
 

spurious mixing.
 

UV_ADV Define to compute the momentum advection terms.
 

CURVGRID Define to compute the extra non-linear advection terms which arise when
 

using curvilinear coordinates.

UV_COR Define to compute the Coriolis term.

UV_U3ADV_SPLIT Define for 3rd-order upstream split momentum advection.

UV_C2ADVECTION Define for 2nd-order centered advection.

UV_C4ADVECTION Define for 4rd-order centered advection.

UV_SADVECTION Define for splines vertical advection (for shallow, vertically wellresolved domains).
 

UV_VIS2 Define to compute the horizontal Laplacian viscosity.
 

UV_VIS4 Define to compute the horizontal biharmonic viscosity.
 

UV_SMAGORINSKY Define for Smagorinsky-like viscosity.
 

UV_LOGDRAG Define for logarithmic bottom friction.
 

UV_LDRAG Define for linear bottom friction.
 

63
 
UV_QDRAG Define for quadratic bottom friction.

UV_WAVEDRAG Define for extra linear bottom wave drag.

UV_DRAG_GRID Define for spatially variable bottom drag.

SPLINES_VVISC Define for splines reconstruction of vertical viscosity.

LIMIT_BSTRESS Define for bottom drag limiter.

Tracers The default horizontal and vertical advection is 4th-order centered.

The 3rd-order upstream split advection (TS_U3ADV_SPLIT) can be used to correct for the spurious diapycnal diffusion of the advection operator in terrain-following

coordinates. If this is chosen, the advection operator is split in advective and diffusive

components and several internal flags are activated in globaldefs.h. Notice that horizontal and vertical advection of tracer is 4th-order centered plus biharmonic diffusion to

correct for spurious diapycnal mixing. The total time-dependent horizontal mixing coefficient are computed in hmixing.F. It is also recommended to use the rotated mixing

tensor along geopotentials (MIX_GEO_TS) for the biharmonic operator.

TS_U3ADV_SPLIT Define for 3rd-order upstream split tracer advection.

TS_A4HADVECTION Define for 4nd-order Akima horizontal advection.

TS_C2HADVECTION Define for 2nd-order centered horizontal advection.

TS_C4HADVECTION Define for 4rd-order centered horizontal advection.

TS_MPDATA Define for recursive MPDATA 3D advection [Margolin and Smolarkievicz,

1998].

TS_MPDATA_LIMIT Define to limit upwind corrector fluxes for stability.

TS_U3HADVECTION Define for 3nd-order upstream horizontal advection.

TS_A4VADVECTION Define for 4nd-order Akima vertical advection.

TS_C2VADVECTION Define for 2nd-order centered vertical advection.

TS_C4VADVECTION Define for 4rd-order centered vertical advection.

TS_SADVECTION Define for splines vertical advection (for shallow, vertically wellresolved domains).

TS_DIF2 Define to compute horizontal Laplacian diffusion.

TS_DIF4 Define to compute horizontal biharmonic diffusion.

TS_SMAGORINSKY Define for Smagorinsky-like diffusion.

TS_FIXED Define for a diagnostic calculation in which the tracer fields do not change

in time.

T_PASSIVE Define for passive tracers.

AGE_MEAN Define for computing mean age of passive tracers (requires two passive

tracers per age tracer).

SALINITY Define if salinity is used as one of the active tracers.

NONLIN_EOS Define to use the nonlinear equation of state.

QCORRECTION Define to use the net heat flux correction.

SCORRECTION Define to use freshwater flux correction.

SSSC_THRESHOLD Define to limit the freshwater flux correction.

LIMIT_STFLX_COOLING Define to limit the surface cooling to prevent water from

freezing.

64
 
SOLAR_SOURCE Define to use solar radiation source term.

SPLINES_VDIFF Define to use splines reconstruction of vertical diffusion.

SRELAXATION Define to use salinity relaxation as a freshwater flux.

TRC_PSOURCE Define for passive tracer point sources/sinks.

ONE_TRACER_SOURCE Define for one value per tracer for all sources.

TWO_D_TRACER_SOURCE Define for one value per tracer per source.

Pressure gradient options If no option is selected, the pressure gradient term is computed

using standard density Jacobian algorithm. Notice that there are two quartic pressure

Jacobian options. They differ on how the WENO reconciliation step is done and in the

monotonicity constraining algorithms.

DJ_GRADPS Define for splines density Jacobian [Shchepetkin and McWilliams, 2003].

PJ_GRADP Define for finite volume Pressure Jacobian [Lin, 1997].

PJ_GRADPSQ2 Define for quartic 2 Pressure Jacobian [Shchepetkin and McWilliams,

2003].

PJ_GRADPSQ4 Define for quartic 4 Pressure Jacobian [Shchepetkin and McWilliams,

2003].

WJ_GRADPS Define for weighted density Jacobian [Song, 1998].

ATM_PRESS Define to impose atmospheric sea-level pressure onto the sea surface.

Atmospheric boundary layer There are now four ways to provide longwave radiation in the atmospheric boundary layer: (1) Compute the net longwave radiation internally using the

Berliand (1952) equation (LONGWAVE) as function of air temperature, sea surface

temperature, relative humidity, and cloud fraction; (2) provide (read) longwave downwelling radiation only and then add outgoing longwave radiation (LONGWAVE_OUT)

as a function of the model sea surface temperature; (3) provide net longwave radiation

(default); (4) provide analytic longwave radiation (net or downwelling) via ana_lrflux.h.

BULK_FLUXES Define for bulk flux computation (required for either Fairall et al.

[2003] or Large and Yeager [2009]).

CCSM_FLUXES Define for CCSM version of bulk flux computation [Large and Yeager,

2009].

NCEP_FLUXES Define if NCEP forcing files are used.

GLOBAL_PERIODIC Define if letting ROMS interpolate onto a grid which spans

the full longitude range of the globe.

NL_BULK_FLUXES Define to use bulk fluxes computed by nonlinear (forward)

model.

COOL_SKIN Define for cool skin correction.

LONGWAVE Define to compute net longwave radiation.

LONGWAVE_OUT Define to compute outgoing longwave radiation. Sea-ice models

compute the outgoing longwave over the ice-covered part of the domain, requiring

this option over the ocean.

EMINUSP Define to compute evaporation minus precipitation.

EMINUSP_SSH Define to compute changes in SSH due to evaporation minus precipitation.

65
 
RUNOFF Define to read freshwater runoff as a second rain-like field.

RUNOFF_SSH Define to compute changes in SSH due to RUNOFF.

The shortwave radiation can be provided as net without an albedo correction for oceanonly simulations. For sea-ice, it is best to provide downwelling shortwave radiation and

perform an albedo correction. In addition, input shortwave radiation data computed from

averaged data (with snapshots greater or equal to 24 hours) can be modulated by the local

diurnal cycle which is a function longitude, latitude and day-of-year.

ALBEDO_CLOUD Define to use albedo equation for shortwave radiation (for water).

ALBEDO_CSIM Define to use albedo function of ice type from CSIM model (for ice).

ICE_ALB_EC92 Define to use albedo function of ice type from Ebert and Curry [1993]

(for ice).

ALBEDO_CURVE Define to use albedo function of latitude from Large and Yeager

[2009] (for water).

ALBEDO_FILE Define to use albedo from a file (for both ice and water).

DIURNAL_SRFLUX Define to impose the local diurnal cycle onto the shortwave

radiation.

Wave roughness in bulk fluxes

COARE_TAYLOR_YELLAND Define to use Taylor and Yelland wave roughness

[Taylor and Yelland, 2001].

COARE_OOST Define to use Oost et al. wave roughness [Oost et al., 2002].

DEEPWATER_WAVES Define to use deep water waves approximation.

Model output

PROFILE Define for time profiling.
 

AVERAGES Define to write out time-averaged model fields.
 

AVERAGES2 Define to write out secondary time-averaged model fields.
 

QUICK Define to write out secondary history fields.
 

NO_HIS Define to turn off writing of primary history fields.
 

AVERAGES_DETIDE Define to write out time-averaged detided fields, one method.
 

FILTERED Define to write out time-averaged detided fields, using a Lanczos filter.
 

DIAGNOSTICS_BIO Define to write out ecosystem diagnostics.
 

DIAGNOSTICS_UV Define to write out momentum diagnostics.
 

DIAGNOSTICS_TS Define to write out tracer diagnostics.
 

STATIONS Define to write out time-series information at specific points in the model.
 

STATIONS_CGRID Define if stations are on native C-grid.
 

Lagrangian floats

FLOATS Define for simulated Lagrangian drifters.
 

FLOAT_STICKY Define for floats to stick/reflect when hitting the bottom or surface.
 

FLOAT_VWALK Define if floats do vertical random walk.
 

VWALK_FORWARD Define for forward time stepping of vertical random walk.
 

66
 
FLOATS_OYSTER Define to activate oyster behavior on floats.

DIAPAUSE Define to simulate diapause on floats.

General model configuration

SOLVE3D Define to solve the 3-D primitive equations.

MASKING Define if there is land in the domain to be masked out.

BODYFORCE Define to apply the surface stress as a body force.

ICESHELF Define for ice shelf cavities.

ICESHELF_3EQ Define for floating three equation ice shelves.

SPHERICAL Define if lat/lon coordinates rather than x/y.

DEBUGGING Define to suppress timestamps for easier comparisons between files.

Analytic fields

ANA_ALBEDO Define for analytic albedo fields.

ANA_BIOLOGY Define for analytic biology initial conditions.

ANA_BPFLUX Define for an analytic bottom passive tracer flux.

ANA_BSFLUX Define for an analytic bottom salt flux.

ANA_BTFLUX Define for an analytic bottom heat flux.

ANA_CLOUD Define for an analytic cloud fraction.

ANA_DIAG Define for customized diagnostics.

ANA_DQDSST Define for an analytic surface heat flux sensitivity to SST.

ANA_DRAG Define for an analytic spatially variable bottom drag.

ANA_FSOBC Define for analytic free-surface boundary conditions.

ANA_GRID Define for an analytic model grid set-up.

ANA_HUMIDITY Define for analytic surface air humidity.

ANA_ICE Define for analytic ice initial conditions.

ANA_INITIAL Define for analytic initial conditions.

ANA_LRFLUX Define for analytic longwave radiation.

ANA_M2CLIMA Define for an analytic 2D momentum climatology.

ANA_M2OBC Define for analytic 2D momentum boundary conditions.

ANA_M3CLIMA Define for an analytic 3D momentum climatology.

ANA_M3OBC Define for analytic 3D momentum boundary conditions.

ANA_MASK Define for an analytic mask.

ANA_NUDGCOEF Define for an analytic nudging coefficients.

ANA_PAIR Define for an analytic surface air pressure.

ANA_PASSIVE Define for analytic initial conditions for inert tracers.

ANA_PERTURB Define for analytic perturbation of initial conditions.

ANA_PSOURCE Define for analytic point sources.

ANA_PTOBC Define for analytic passive tracer boundary conditions.

ANA_RAIN Define for analytic rain fall rate.

67
 
ANA_SEDIMENT Define for analytic sediment initial fields.
 

ANA_SMFLUX Define for an analytic kinematic surface momentum stress.
 

ANA_SNOW Define for analytic snow fall rate.
 

ANA_SPFLUX Define for analytic surface passive tracers fluxes.
 

ANA_SPINNING Define for an analytic time-varying rotation force.
 

ANA_SPONGE Define for analytic sponge (higher viscosity).
 

ANA_SRFLUX Define for an analytic kinematic surface shortwave radiation.
 

ANA_SSFLUX Define for an analytic kinematic surface freshwater flux.
 

ANA_SSH Define for an analytic sea surface height.
 

ANA_SSS Define for an analytic sea surface salinity.
 

ANA_SST Define for an analytic SST and ∂Q/∂SST.
 

ANA_STFLUX Define for an analytic kinematic surface heat flux.
 

ANA_TAIR Define for analytic surface air temperature.
 

ANA_TCLIMA Define for an analytic tracer climatology.
 

ANA_TOBC Define for analytic tracer open boundary conditions.
 

ANA_TRC_PSOURCE Define for analytic passive tracer point sources.
 

ANA_VMIX Define for analytic vertical mixing coefficients.
 

ANA_WINDS Define for analytic surface winds.
 

ANA_WWAVE Define for an analytic wind induced wave field.
 

Horizontal mixing of momentum

MIX_GEO_UV Define for viscosity along constant z (geopotential) surfaces.
 

MIX_S_UV Define for viscosity along constant s surfaces.
 

VISC_GRID Define for horizontally variable viscosity coefficient.
 

Horizontal mixing of tracers

DIFF_GRID Define for horizontally variable diffusion coefficient.
 

MIX_GEO_TS Define for diffusion along constant z (geopotential) surfaces.
 

MIX_ISO_TS Define for diffusion along constant potential density (epineutral) surfaces.

MIX_S_TS Define for diffusion along constant s surfaces.

TS_MIX_CLIMA Define for diffusion of tracer perturbation T − Tclm.

TS_MIX_MAX_SLOPE Define for maximum slope in epineutral diffusion.

TS_MIX_MIN_STRAT Define for minimum stratification in epineutral diffusion.

TS_MIX_STABILITY Define for weighting diffusion between two time levels.

Vertical mixing

BVF_MIXING Define to activate Brunt-Väisälä frequency mixing.

GLS_MIXING Define for Generic Length-Scale mixing.

CANUTO_A Define for Canuto A-stability function formulation.

CANUTO_B Define for Canuto B-stability function formulation.

68
 
CHARNOK Define for Charnok surface roughness from wind stress.

CRAIG_BANNER Define for Craig and Banner wave breaking surface flux.

KANTHA_CLAYSON Define for Kantha and Clayson stability function.

K_C2ADVECTION Define for 2th-order centered advection.

K_C4ADVECTION Define for 4th-order centered advection.

N2S2_HORAVG Define for horizontal smoothing of buoyancy/shear.

RI_SPLINES Define for splines reconstruction of vertical shear.

ZOS_HSIG Define for surface roughness from wave amplitude.

TKE_WAVEDISS Define for wave breaking surface flux from wave amplitude.

LMD_MIXING Define to activate Large/McWilliams/Doney interior closure.

LMD_BKPP Define to add a bottom boundary layer from a local K-Profile

Parameterization (KPP).

LMD_CONVEC Define to add convective mixing due to shear instabilities.

LMD_DDMIX Define to add double-diffusive mixing.

LMD_NONLOCAL Define to add convective nonlocal transport.

LMD_RIMIX Define to add diffusivity due to shear instabilities.

LMD_SHAPIRO Define to Shapiro filtering boundary layer depths.

LMD_SKPP Define to add a surface boundary layer from a local K-Profile

Parameterization (KPP).

WTYPE_GRID Define for spatially dependent Jerlov water type.

RI_SPLINES Define for splines reconstruction of vertical shear.

MY25_MIXING Define to activate Mellor/Yamada Level-2.5 closure.

KANTHA_CLAYSON Define for Kantha and Clayson stability function.

K_C2ADVECTION Define for 2th-order centered advection.

K_C4ADVECTION Define for 4th-order centered advection.

N2S2_HORAVG Define for horizontal smoothing of buoyancy/shear.

RI_SPLINES Define for splines reconstruction of vertical shear.

These two can be used if RI_SPLINES is not defined.

RI_HORAVG Define for horizontal Richardson number smoothing.

RI_VERAVG Define for vertical Richardson number smoothing.

Bottom boundary layer The Options MB_Z0BL and MB_Z0RIP should be activated concurrently.

MB_BBL Define to activate Meinte Blaas BBL closure.

MB_CALC_ZNOT Define to compute bottom roughness internally.

MB_CALC_UB Define to compute bottom orbital velocity internally.

MB_Z0BIO Define for biogenic bedform roughness for ripples.

MB_Z0BL Define for bedload roughness for ripples.

MB_Z0RIP Define for bedform roughness for ripples.

SG_BBL Define to activate Styles/Glenn bottom boundary layer formulation.

SG_CALC_ZNOT Define to compute bottom roughness internally.

SG_CALC_UB Define to compute bottom orbital velocity internally.

SG_LOGINT Define for logarithmic interpolation of (Ur,Vr).

69
 
SSW_BBL Define to activate Sherwood/Signell/Warner bottom boundary layer closure.

SSW_CALC_ZNOT Define to compute bottom roughness internally.

SSW_CALC_UB Define to compute bottom orbital velocity internally.

SSW_LOGINT Define for logarithmic interpolation of (Ur,Vr).

SSW_FORM_DRAG_COR Define to activate form drag coefficient.

SSW_Z0BIO Define for biogenic bedform roughness for ripples.

SSW_Z0BL Define for bedload roughness for ripples.

SSW_Z0RIP Define for bedform roughness for ripples.

Sea ice

ICE_MODEL Define to use ice component of the model (see §5).

ICE_THERMO Define for ice thermodynamics.

ICE_MK Define for Mellor and Kantha [1989] ice thermodynamics—this is the

only choice.

ICE_MOMENTUM Define for momentum component of the ice.

ICE_MOM_BULK Define for alternate ice-water stress computation.

ICE_EVP Define for elastic-viscous-plastic rheology [Hunke and Dukowicz, 1997,

Hunke, 2001].

ICE_QUAD_STRENGTH Define for quadratic ice strength from Overland

and Pease [1988].

ICE_ADVECT Define for advection of ice tracers.

ICE_SMOLAR Define to use MPDATA for ice tracers (no other option).

ICE_UPWIND Define for upwind advection.

ICE_SHOREFAST Define for simple shorefast-ice algorithm [Budgell, 2005].

ICE_LANDFAST Define for alternate shorefast-ice algorithm [Lemieux et al.,

2015].

FASTICE_CLIMATOLOGY Define for clamping to a climatology of landfast

ice.

ICE_BULK_FLUXES Define for ice part of bulk flux computation.

ICE_I_O Define to allow light into the ice as heat.

ICE_CONVSNOW Define for conversion of flooded snow to ice.

OUTFLOW_MASK Define for Hibler style outflow cells.

INI_GLORYS_ICE Define to read initial conditions for aice, hice, and surface temperature from a GLORYS file.

NO_SCORRECTION_ICE Define to turn off surface salinity nudging under

the ice.

Boundary conditions

RADIATION_2D Define for tangential phase speed in radiation conditions.

Tides The tidal data is processed in terms of tidal constituents, classified by period. The tidal

forcing is computed for the full horizontal grid. If requested, the tidal forcing is added to

the processed open boundary data.

Both tidal elevation and tidal currents are required to force the model properly. However,

if only the tidal elevation is available, the tidal currents at the open boundary can be estimated by reduced physics. Only the pressure gradient, Coriolis, and surface and bottom

70
 
stresses terms are considered at the open boundary. See u2dbc_im.F or v2dbc_im.F

for details. Notice that there is an additional option (FSOBC_REDUCED) for the

computation of the pressure gradient term in both Flather or reduced physics boundary

conditions.

SSH_TIDES Define if imposing tidal elevation.
 

UV_TIDES Define if imposing tidal currents.
 

POT_TIDES Define if imposing potential tides.
 

RAMP_TIDES Define if ramping (over one day) tidal forcing from zero.
 

ADD_FSOBC Define to add tidal elevation to processed OBC data.
 

ADD_M2OBC Define to add tidal currents to processed OBC data.
 

FSOBC_REDUCED Define for reduced physics when providing SSH boundary conditions but not the 2D momentum fields.

TIDES_ASTRO Define to add contributions from the long-period tides as done by

Foreman.

Climatology These are for OFFLINE options only.

OCLIMATOLOGY Define for reading (rather than computing) the vertical momentum

climatology arrays.

AKTCLIMATOLOGY Define for reading the tracer vertical diffusion climatology arrays.

Ecosystem models

BIO_FENNEL Define for Fennel et al. [2006] nitrogen-based model.

BIO_SEDIMENT Define to restore fallen material to the nutrient pool.

CARBON Define to add carbon constituents.

DENITRIFICATION Define to add denitrification processes.

OXYGEN Define to add oxygen dynamics.

OCMIP_OXYGEN_SC Define if Schmidt number from Keeling et al. [1998].

TALK_NONCONSERV Define for nonconservative computation of alkalinity.

BEST_NPZ Define for Gibson et al. (personal communication) Bering Sea model.

STATIONARY Define for extra output.

BENTHIC Define for benthic components.

ICE_BIO Define for ice algae.

JELLY Define for jellyfish.

CLIM_ICE_1D Define if 1-D with ice.

BIO_UMAINE Define for Xiu and Chai [2014], Xiu and Chai [2012] model with iron

limitation.

OXYGEN
 

CARBON
 

SINK_OP1
 

SINK_OP2
 

TALK_NONCONVERV
 

DIURNAL_LIGHT
 

71
 
OPTIC_UMAINE

BIO_GOANPZ Define for Hinckley et al. [2009] Gulf of Alaska model.

NPZD_FRANKS Define for NPZD model of Franks et al. [1986].

NPZD_IRON Define for NPZD model with iron limitation.

NPZD_POWELL Define for NPZD model of Powell et al. [2006].

IRON_LIMIT Define for iron limitation on phytoplankton growth (for a few of the

biology models).

IRON_RELAX Define for nudging to iron over the shelf.

ECOSIM Define for bio-optical EcoSim model.

NEMURO Define for Nemuro ecosystem model [Kishi et al., 2007]. Need to choose a zooplankton grazing option (HOLLING_GRAZING or IVLEV_EXPLICIT).

The default implicit IVLEV algorithm does not work yet.

BIO_SEDIMENT Define to restore fallen material to the nutrient pool.

NEMURO_SED1 Define for sediment remineralization.

PRIMARY_PROD Define for primary productivity output.

HOLLING_GRAZING Define for Holling-type s-shaped curve grazing (implicit).

IVLEV_EXPLICIT Define for Ivlev explicit grazing algorithm.

RED_TIDE Define for red tide biological model.

Sediment transport model

SEDIMENT Define to activate sediment transport model [Warner et al., 2008].

BEDLOAD_MPM Define to activate Meyer-Peter-Mueller bed load.

BEDLOAD_SOULSBY Define to activate Soulsby wave/current bed load.

SED_DENS Define to allow sediment to affect equation of state.

SED_MORPH Define to allow bottom model elevation to evolve.

SUSPLOAD Define to activate suspended load transport.

Nearshore options

WET_DRY Define to allow wetting and drying of cells.

NEARSHORE_MELLOR05 Define for radiation stress terms from waves [Mellor,

2005].

NEARSHORE_MELLOR08 Define for radiation stress terms from waves [Mellor,

2008].

Nesting

NESTING Define to activate composite/refinement nesting.

NO_CORRECT_TRACER Define to avoid two-way correction of boundary tracer.

ONE_WAY Define for one-way nesting in refinement grids.

TIME_INTERP_FLUX Time-interpolate coarse mass flux instead of persist.

NetCDF input/output

DEFLATE Define to set compression of NetCDF-4/HDF5 format files.

72
 
HDF5 Define to create NetCDF-4/HDF5 format files.
 

NO_LBC_ATT Define to not check NLM_LBC global attribute on restart.
 

NO_READ_GHOST Define to not include ghost points during read/scatter.
 

NO_WRITE_GRID Define to omit writing of grid arrays.
 

PARALLEL_IN Define for parallel input via HDF5 or pnetcdf libraries.
 

PARALLEL_OUT Define for parallel output via HDF5 or pnetcdf libraries.
 

PERFECT_RESTART Define to include reading and writing enough variables and
 

time levels for a perfect restart.

PNETCDF Define for parallel I/O via pnetcdf library (classic format).

POSITIVE_ZERO Define for positive zero on output.

READ_WATER Define to only read water points.

WRITE_WATER Define to only write water points.

RST_SINGLE Define to write single precision restart fields.

OUT_DOUBLE Define to write double precision output fields.

INLINE_2DIO Define to read/write 3D fields level by level.

6.8 Important parameters

The following is a list of some of the important parameters in the model. These are in mod_param.F

and are read from the standard input file as described in §7.1.12.

Ngrids	 Number of grids.

NestLayers Number of levels of refinement grids.

GridsInLayer Number of grids in each refinement level (needs NestLayers values).

Lm	 Number of interior grid points in the ξ-direction for each grid.

Mm	 Number of interior grid points in the η-direction for each grid.

N	 Number of grid points in the vertical for each grid.

NAT	 Number of active tracers (usually 2 for temperature and salinity).

NBT	 Number of biological tracers. This will depend on the ecosystem model used.

NST	 Number of sediment tracers.

NPT	 Number of passive tracers.

NT	 Total number of tracer fields. NT = NAT+NBT+NST+NPT

NtileI	 Number of tiles in the ξ-direction for each grid.

NtileJ	 Number of tiles in the η-direction for each grid.

LBC	 In the old days, the choice of boundary condition was made via cpp options. After a

significant rewrite to support nesting, the boundary condition choices are now loaded into

this array.

73
 
6.9 Domain decomposition

ROMS supports serial, OpenMP, and MPI computations, with the user choosing between them at

compile time. The serial code can also take advantage of multiple small tiles which can be sized

to fit in cache. All are accomplished through domain decomposition in the horizontal. All of the

horizontal operations are explicit with a relatively small footprint, so the tiling is a logical choice.

Some goals in the parallel design of ROMS were:

•	 Minimize code changes.

•	 Don’t hard-code the number of processes.

•	 MPI and OpenMP share the same basic structure.

•	 Don’t break the serial optimizations.

•	 Same result as serial code for any number of processes (not always the case for large numbers

of processes).

•	 Portability—able to run on any (Unix) system.

First, some cpp options. If we’re compiling for MPI, the option -DMPI gets added to the

argument list for cpp. Then, in globaldefs.h, we have:

#if defined MPI

# define DISTRIBUTE

#endif

The rest of the code uses DISTRIBUTE to identify distributed memory jobs. The OpenMP case

is more straightforward, with -D_OPENMP getting passed to cpp and _OPENMP being the

tag to check within ROMS.

The whole horizontal ROMS grid is shown in Fig. 13. The computations are done over the

cells inside the darker line; the cells are numbered 1 to Lm in the ξ-direction and 1 to Mm in the

η-direction. Those looking ahead to running in parallel would be wise to include factors of two in

their choice of Lm and Mm. ROMS will run in parallel with any values of Lm and Mm, but the

computations might not be load-balanced.

6.9.1 ROMS internal numbers

A domain with tiles is shown in Fig. 14. The overlap areas are known as ghost points or halo

points. Each tile is an MPI process, an OpenMP thread, or a discrete unit of computation in a

serial run. The tile contains enough information to time-step all the interior points, so the number

of ghost points is dictated by the footprint of the algorithm using the largest number of neighbor

points. In ROMS, the halo area would be two grids points wide unless the MPDATA advection or

biharmonic viscosity scheme is used, in which case it needs three. The variable NghostPoints is

set accordingly in inp_par:

#if defined TS_MPDATA || defined UV_VIS4
 

NghostPoints=3
 

#else
 

NghostPoints=2
 

#endif
 

IF (ANY(ComposedGrid).or.ANY(RefinedGrid)) THEN
 

NghostPoints=MAX(3,NghostPoints)
 

END IF
 

74
 
M 0 × 0 × 0 × 0 × 0 × 0 × 0 × 0

M

Mm 0 ×
 × 0

Mm
 

0 × × 0 .

.

.

 2 0 × × 0

η 2
 

1 0 ×
 × 0

1

0 × 0 × 0 × 0 × 0 × 0

0 × 0 × 0 × 0 × 0 × 0

0 × 0 × 0 × 0 × 0 × 0

0 × 0 × 0 × 0 × 0 × 0

j = 0 0 × 0 × 0 × 0 × 0 × 0 × 0 × 0

i = 0 1 1 2 2 3 3 · · · Lm Lm L L

_

ξ

× – u points

– v points

0 – ρ points

Figure 13: The whole grid. Note that there are Lm by Mm interior computational points. The

points on the thick outer line and those outside it are provided by the boundary conditions.

75
 
Itile = 0 1

1

3

5

6 7

ChunkSizeI

4

2

Jtile = 0 tile = 0

1

2

3

ChunkSizeJ

MarginJ

Figure 14: A tiled grid with some ROMS tile variables.
 

The grid nesting requires NghostPoints=3, hence the last bit there.
 

The number of tiles is set in the input file as NtileI and NtileJ. For an MPI job, the product

of the two must equal the number of MPI processes. For an OpenMP job, the number of tiles must

be a multiple of the number of threads. For instance, for NtileI= 4 and NtileJ= 6, you must have

24 MPI processes while 2, 3, 4, 6, 8, 12 and 24 are all valid numbers of OpenMP threads. Also, a

serial run could have 24 tiles and would just compute them sequentially.

Once the input file has been read, we can compute the tile sizes in get_bounds:

ChunkSizeI = (Lm+NtileI-1)/NtileI

ChunkSizeJ = (Mm+NtileJ-1)/NtileJ

MarginI = (NtileI*ChunkSizeI-Lm)/2

MarginJ = (NtileJ*ChunkSizeJ-Mm)/2

Some internal ROMS numbers are shown in Fig. 14 and are in the BOUNDS structure in

mod_param.F. MarginI and MarginJ are zero if the numbers work out perfectly, i.e. Lm/NtileI

and Mm/NtileJ are integers. The tile numbers match the MPI process numbers.

In picking a numbering scheme for indices within a tile, there are two common choices, as shown

in Fig. 15. Each tile can be numbered from 1 to ChunksizeI or it can retain the numbering it

would have in the whole grid. We have chosen this second option for ease when debugging features

such as river inputs which apply to specific locations on the grid. It is simple to do using Fortran

90 dynamic memory allocation.

With the tile sizes known, we can assign beginning and ending indices for each tile. Some of

the details depend on whether or not the domain is periodic in that direction, as shown in Fig. 16.

6.9.2 MPI exchange

For MPI jobs, the ghost points need to be updated between interior point computations. The

routines mp_exchange2d, mp_exchange3d and mp_exchange4d can be used to update the

76
 
(b)

321

5432 109876 1514131211

54321

1

(a)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

54321 54

Figure 15: A choice of numbering schemes: (a) each tile is numbered the same, and (b) each tile

retains the numbering of the parent domain.

halo points of up to four arrays at a time. Each of these routines calls tile_neighbors to figure

out which tiles are neighboring and whether or not there really is a neighboring tile on each side.

The mp_exchangexd routines then call:

mpi_irecv
 

mpi_send
 

mpi_wait
 

The exchanges happen first in the east-west direction, then in the north-south direction, saving

the need for diagonal exchanges. A figure with interior points colored by tile and grey halo points

needing an update is shown in Fig. 17(a). The updated halo points are shown in Fig. 17(b).

6.9.3 Code syntax

In main3d, many function calls are surrounded by nesting code such as:

DO ig=1,GridsInLayer(nl)
 

ng=GridNumber(ig,nl)
 

DO tile=first_tile(ng),last_tile(ng),+1
 

CALL set_data (ng, tile)
 

END DO
 

!$OMP BARRIER

END DO

IF (exit_flag.ne.NoError) RETURN

where first_tile and last_tile are set in nl_ocean.h:

#if defined _OPENMP

MyThread=my_threadnum()

#elif defined DISTRIBUTE

MyThread=MyRank

#else

MyThread=0

#endif

77
 
IendR

IstrUIstr Iend

IstrR IendR

eastern tile

IstrUIstr Iend

IstrR IendR

central tile

IstrUIstr Iend

IstrR IendR

central tile

Non−periodic

Periodic

2 5 6 7 8 Lm L

321 654 Lm87

0 1 3 4

IstrIstrU Iend

IendR

western tile

IstrUIstr Iend

IendR

western tile

IstrUIstr Iend

IstrR

eastern tile

IstrR

IstrR

Figure 16: Some ROMS variables for tiles, for both a periodic and non-periodic case. Shown are

the variables in the i-direction, the j-direction is similar.

DO ng=1,Ngrids
 

chunk_size=(NtileX(ng)*NtileE(ng)+numthreads-1)/numthreads
 

first_tile(ng)=MyThread*chunk_size
 

last_tile (ng)=first_tile(ng)+chunk_size-1
 

END DO

NtileX and NtileE no longer depend on whether we’re using MPI:

NtileX(1:Ngrids)=NtileI(1:Ngrids)
 

NtileE(1:Ngrids)=NtileJ(1:Ngrids)
 

Instead, numthreads varies, being set to 1 for serial jobs, to the number of MPI processes for

MPI and to omp_get_max_threads() for OpenMP jobs. For MPI jobs, integer division gives

chunk_size the value of 1 and first_tile and last_tile are both set to the process number.

In looking at a typical routine that’s called from main3d, the routine is usually quite short,

calling a _tile version of itself in which the actual work happens:

SUBROUTINE set_data (ng, tile)
 

# include "tile.h"
 

CALL set_data_tile (ng, tile, &
 

& LBi, UBi, LBj, UBj, &
 

& IminS, ImaxS, JminS, JmaxS)
 

RETURN
 

END SUBROUTINE set_data
 

Here, there are two sets of array lower and upper bounds, those in the LBi family and those in the

IminS family. Both depend on the Istr family shown in Fig. 16. The IminS family is for work

arrays that are local to an MPI process or to an OpenMP thread, also local to a _tile routine.

They are initialized:

78
 
(a)

(b)

Figure 17: A tiled grid with out-of-date halo regions shown in grey and the interior points colorcoded by tile: (a) before an exchange and (b) after an exchange.

79
IminS=BOUNDS(ng)%Istr(tile)-3

ImaxS=BOUNDS(ng)%Iend(tile)+3

JminS=BOUNDS(ng)%Jstr(tile)-3

JmaxS=BOUNDS(ng)%Jend(tile)+3

and used:

real(r8), dimension(IminS:ImaxS,JminS:JmaxS) :: work1

real(r8), dimension(IminS:ImaxS,JminS:JmaxS) :: work2

The Istr and LBi families are dimensioned by the number of tiles once it is known by inp_par:

DO ng=1,Ngrids

Ntiles=NtileI(ng)*NtileJ(ng)-1

allocate ( BOUNDS(ng) % LBi (-1:Ntiles) )

:

allocate ( BOUNDS(ng) % Jend (-1:Ntiles) )

END DO

They are then initialized in calls to the routines in get_bounds.F. Imin is set to -NghostPoints

or zero, for periodic and non-periodic domains, respectively. The UBi family is set once the Istr

family is known:

IF ((Itile.eq.-1).or.(Itile.eq.0)) THEN

LBi=Imin

ELSE

LBi=Istr-Nghost

END IF

In the case of set_data, we are simply passing array indices for the tiled arrays. To access the

tiled arrays from within set_data_tile, we need to use the relevant modules and then refer to

the array with its full name:

USE mod_forces

:

CALL set_2dfld_tile (ng, tile, iNLM, idCfra, &

& LBi, UBi, LBj, UBj, &

& FORCES(ng)%cloudG, &

& FORCES(ng)%cloud, &

& update)

In other cases, the parent routine would have the use, then would pass the relevant array to the

_tile routine:

USE mod_grid

:

CALL prsgrd_tile (ng, tile, &

:

& GRID(ng) % Hz, &

:

SUBROUTINE prsgrd_tile (ng, tile, &

:

& Hz, z_r, z_w, &

:

real(r8), intent(in) :: Hz(LBi:,LBj:,:)

This allows the _tile routine to use Hz with the same syntax as the pre-parallel, pre-module code

once had.

80
 
6.9.4 Input/output

In ROMS, the distributed memory I/O is all happening on the master process (0) unless you specifically ask it to use MPI-I/O, which requires either or both of PARALLEL_IN, PARALLEL_OUT

and either of the HDF5 or PNETCDF cpp flags to be defined. If you choose HDF5, you will

be reading and/or writing HDF5 files and will need to update your pre- and post-processing tools

accordingly. I have tentatively tried the parallel I/O and found it to be exceedingly slow—I’ve been

told since that this is the fault of the NetCDF-4 layer sitting on top of HDF5—HDF5 alone

should be fast.

In the case of having all the I/O pass through the master process, we can still read and write

classic NetCDF-3 files. Care must be taken though, in the event of an error. ROMS has been

cleaned up so that the master process will broadcast its return state to the other processes and

they can all die gracefully together when there is a problem.

An example of a routine which reads from disk is get_grid, called from initial. Each MPI

process calls get_grid:

CALL get_grid (ng, iNLM)

# ifdef DISTRIBUTE

CALL mp_bcasti (ng, iNLM, exit_flag)

# endif

if (exit_flag.ne.NoError) RETURN

If any one of the processes has trouble, it will enter into the exit_flag which is then shared by all.

To read in an array variable, all processes in get_grid use nf_fread2d and friends:

status=nf_fread2d(ng, model, ncname, ncGRDid(ng), &

& var_name(it), var_id(it), &

& 0, gtype, Vsize, &

& LBi, UBi, LBj, UBj, &

& Fscl, Fmin, Fmax, &

& GRID(ng) % rmask, &

& GRID(ng) % rmask)

IF (status.ne.nf90_noerr) THEN

exit_flag=2

ioerror=status

EXIT

END IF

Within nf_fread2d, we get to a call to the NetCDF library from just the master process:

IF (InpThread) THEN

status=nf90_get_var(ncid, ncvarid, wrk, start, total)

:

END IF

# ifdef DISTRIBUTE

CALL mp_bcasti (ng, model, status)

# endif

IF (status.ne.nf90_noerr) THEN

exit_flag=2

ioerror=status

nf_fread2d=status

RETURN

END IF

81
 
At this point, the master process has the entire 2-D array stored in wrk. This then needs to be

divvied out to the various tiles to their copy of the array in question (stored in the A argument to

nf_fread2d):

# ifdef DISTRIBUTE
 

CALL mp_scatter2d (ng, model, LBi, UBi, LBj, UBj, &
 

& Nghost, MyType, Amin, Amax, &
 

# if defined READ_WATER && defined MASKING
 

& NWpts, SCALARS(ng)%IJwater(:,wtype), &
 

# endif
 

& Npts, wrk, A)
 

Something similar happens when writing to output files.

PIO I have a branch in which I linked to the Parallel-IO or PIO library from NCAR. It’s unstable

for fewer writers than processes, but let me know if you’d like to try it.

82
 
7 Configuring ROMS for a Specific Application

This chapter describes the parts of ROMS for which the user is responsible when configuring it for

a given application. Section 7.1 describes the process in a generic fashion while §7.2, §7.3, §7.4 and

§7.5 step through the application of ROMS to idealized upwelling/downwelling and several realistic

wind-driven domains respectively. As distributed, ROMS is ready to run quite a few examples,

where the C preprocessor flags determine which is to be executed. Some of these examples are

described in Haidvogel and Beckmann [1999], some are listed here:

BASIN	 This is a rectangular, flat-bottomed basin with double-gyre wind forcing. When run, it

produces a western boundary current flowing into a central “Gulf Stream” which goes unstable and generates eddies. The goal is to run adiabatically to study the homogenization

of potential vorticity. It earned its nickname of Big Bad Basin by taking a long time to

run and by causing difficulties for the spectral versions of SPEM.

GRAV_ADJ The gravitational adjustment problem takes place in a long narrow domain which

is initialized with dense water at one end and light water at the other. At time zero, the

water is released and it generates two propagating fronts as the light water rushes to fill

the top and the dense water rushes to fill the bottom. This configuration can be used to

test various advection schemes.

OVERFLOW This configuration is similar to the GRAV_ADJ problem, but is initialized with

dense water in the shallow part of a domain with a sloping bottom.

SEAMOUNT The seamount test has been used to test the pressure gradient errors. It has

an idealized seamount in a periodic channel. See Beckmann and Haidvogel [1993] and

McCalpin [1994] for more information.

UPWELLING The upwelling/downwelling example was contributed by Macks and Middleton

[1993] and consists of a periodic channel with shelves on each side. There is along-channel

wind forcing and the Coriolis term leads to upwelling on one side and downwelling on the

other side. If you run it for several days without vertical mixing, you end up with dense

water over light water.

Some NetCDF input files for the ROMS examples can be found under Data/ROMS in the ROMS

distribution while many others can be downloaded as part of the ROMS test package. The ASCII

input files are under ROMS/External.

7.1 Configuring ROMS

The four main sections you need to change in ROMS are the makefile or build.bash, an include

file with cpp options, any analytic functions, and the ASCII input file. If more realistic fields are

desired, you will have to provide other NetCDF input files as well, for instance for the grid and the

wind forcing.

7.1.1 Case Name

First, you need to decide on a name for your particular application or configuration. This name is

provided via the ROMS_APPLICATION in either the makefile or the build script. This name

should be reasonably short, all uppercase, with spaces converted to underscores. For example, let’s

say we pick the name WIKI_TEST. This name gets defined during the build, so you can add

code protected by #ifdef WIKI_TEST as needed. This would be a good time to either copy

the makefile or the build Script to create one specific to this case prior to editing it.

83
 
7.1.2 Case-specific Include File

Each application has its own include file, included by cppdefs.h. The name of this file is the name of

your application (WIKI_TEST here) turned into lower case, with ’.h’ appended (wiki_test.h).

The location of this file is set by MY_HEADER_DIR, pointing to User/Include or some

other location of your choosing.

The complete list of options to be set prior to compilation are listed in §6.7. Place those you

need in the wiki_test.h file. These include algorithm choices (e.g. advection and turbulence

closure schemes), output options (averages, diagnostics, stations, floats), and application modules

(biology, sediments). Each line should be of the form:

#define SOME_VAR

Note that any undefined variable need not be mentioned.

Also note that if you copy a predefined application from ROMS/Include as a template for

your application, you must rename it. If you don’t change the name, ROMS will use the one in

ROMS/Include and your file will be ignored during the build procedure.

7.1.3 Functionals

Some of the cpp options have names beginning with ANA_. For each one of these, you will be

expected to provide an analytic expression for the field in question in the corresponding include file.

These files are listed in §6.5 and their location is determined by MY_ANALYTICAL_DIR.

You may chose to copy those from User/Functionals to some new directory and place your version

of the assignments within

#ifdef WIKI_TEST
 

! Set weird and wonderful winds
 

:
 

#endif
 

This makes it easy to search for later, if nothing else.

7.1.4 checkdefs.F

If you add new cpp variables to the code (other than your application name), it is recommended

that you also add the appropriate code to checkdefs.F, such as:

#ifdef SLEET

IF (Master) WRITE(stdout,20) ’SLEET’, &

& ’Sleet falling on the ice option.’

is=lenstr(Coptions)+1

Coptions(is:is+7)=’ SLEET,’

#endif /* SLEET */

Note that the number “7” on the Coptions line must be set according to the length of the string

you are adding. In this case 7 is for “ SLEET,”, including the comma and the space. You do

not need to do this for your application name (WIKI_TEST here), since checkdefs will print

whatever is in MyAppCPP.

84
 
7.1.5 Model domain
 

It is assumed in this manual that Ngrids will be set to 1. Having multiple grids talk to each other

is brand new in ROMS 3.7, with documentation appearing on the ROMS wiki.

One of the first things the user must decide is how many grid points to use, and can be

afforded. There are three parameters in ocean.in which specify the grid size and one parameter

for the number of active tracers:

Lm Number of finite-difference points in ξ.

Mm Number of finite-difference points in η.

N Number of finite-difference points in the vertical.

NAT Number of active tracers.

The number of biological tracers is set in the biology.in file. There are no constraints on these

except Lm ≥ 2, Mm ≥ 2, N ≥ 1 and NAT ≥ 1. Lm and Mm should be at least 3 if the domain

is periodic in that direction. If N == 1, don’t define SOLVE_3D.

7.1.6 x, y grid

The subroutine get_grid or ana_grid is called by initial to set the grid arrays, the bathymetry,

and the Coriolis parameter. Most of the simple test problems have their grid information specified

in ana_grid.h in the directory ROMS/Functionals. More realistic problems require a NetCDF

grid file, produced by the grid generation programs described in Wilkin and Hedstrom [1991], by

the Matlab SeaGrid, or by some other method. The variables which are read by get_grid are:

xl, el, spherical, f, h, pm, pn, x_rho, y_rho, lon_rho, lat_rho, angle.

If the grid is curved, get_grid will also read:

dndx, dmde.

Likewise, if MASKING is defined, it will read:

mask_rho, mask_u, mask_v, mask_psi.

7.1.7 ξ, η grid

Before providing initial conditions and boundary conditions, the user must understand the model

grid. The fields are laid out on an Arakawa C grid as in Fig. 1. The overall grid is shown in Fig. 13.

The thick outer line shows the position of the model boundary. The points inside this boundary

are those which are advanced in time using the model physics. The points on the boundary and

those on the outside must be supplied by the boundary conditions.

The three-dimensional model fields are carried in four-dimensional arrays, where the fourth

array index refers to one of two or three time levels. The tracers have a fifth array index telling

which tracer is being referred to. For instance, itemp = 1 refers to potential temperature while

isalt = 2 refers to salinity. The integers i, j, and k are used throughout the model to index the

three spatial dimensions:

i Index variable for the ξ-direction.

j Index variable for the η-direction.

k Index variable for the σ-direction. k = 1 refers to the bottom

while k = N refers to the surface.

In terms of grid extent, the model is expecting values for rho-point indices from i = 0 through

i =Lm+1, even though the computational domain extends from i = 1 through i =Lm.

85
 
7.1.8 Initial conditions

The initial values for the model fields are provided by either ana_initial or get_state. get_state

is also used to read a restart file if the model is being restarted from a previous run.

Also in initial, rho_eos is called to initialize the density field. rho_eos also computes rhoA,

the vertically averaged density, and rhoS, the density perturbation. Both rhoA and rhoS are

used in the barotropic pressure gradient.

7.1.9 Equation of state

The equation of state is defined in the subroutine rho_eos. Two versions are provided in ROMS:

a nonlinear ρ = ρ(T, S, z) from Jackett and McDougall [1995] and a linear ρ(T, S). The linear form

is

ρ = R0 − Tcoef · (T − T0) + Scoef · (S − S0)

or

ρ = R0 + Tcoef · (T − T0),

depending on whether or not SALINITY is defined. Specify which equation of state you would

like to use with the NONLIN_EOS C preprocessor flag in your application include file. The

linear coefficients R0, T0, Tcoef, S0, and Scoef are set in ocean.in. Note that we are computing

in situ density from potential temperature and salinity. Some of the vertical mixing schemes require

potential density and some other fields, which are computed by rho_eos as well.

7.1.10 Boundary conditions

The horizontal boundary conditions are provided by the subroutines in u3dbc_im, v3dbc_im,

u2dbc_im, v2dbc_im, t3dbc_im, and zetabc. They are called every time-step and provide

the boundary values for the fields u, v, u, v, all tracers, and ζ, respectively. They are currently

configured for a closed basin, a periodic channel, a doubly periodic domain or a domain with

various open boundary conditions. Each side is controlled independently with “West” being the

i=1 boundary, “East” being the i=L boundary, “South” being the j=1 boundary, and “North”

being the j=M boundary. These choices are set in ocean.in via the LBC array.

Many of the choices for open boundaries require that the model have some boundary values for the field in question. These can be specified in the appropriate ana_xxx.h file for say

ANA_TOBC or they can be read from a boundary NetCDF file. There is logic in globaldefs.h

by which ROMS decides whether or not it needs to read a boundary file.

7.1.11 Model forcing

(a) Winds and thermal fluxes

There are two different ways to apply a wind forcing: as a surface momentum flux in the vertical

viscosity term, or as a body force over the upper water column. Usually, we set the vertical σcoordinate parameters to retain some resolution near the surface and apply the fluxes as boundary

conditions to the vertical viscosity/diffusivity. In either case, the surface and bottom fluxes are

either defined analytically, read from a forcing file, or computed inside ROMS using a bulk flux

formula from the appropriate atmospheric fields (air temperature and winds, for instance). You

must either edit the appropriate ana_xxx.h or create a NetCDF forcing file in the format expected

by ROMS. Note that it is quite common to put the surface variables in the forcing file while having

an analytic bottom heat flux. ROMS now has the capability of reading in a list of forcing fields—it

can be convenient to have one file per field rather than stuffing tides, winds, river inputs all into

one file.

86
 
In the past, our vertical resolution was relatively coarse and the vertical viscosity would have

to have been unreasonably large for us to resolve the surface Ekman layer. If that is your situation,

define BODYFORCE in cppdefs.h and provide a value for levsfrc in ocean.in. The forcing is

applied over the levels from levsfrc to N. The above caution about vertical resolution also applies

to the surface fluxes of T and S, although BODYFORCE only refers to wind stress, not the

surface tracer fluxes.

(b) Climatology

One way to force the model is via a nudging to the tracer and/or momentum climatologies.

Nudging to values from a global model near the boundaries can be useful when the boundary

conditions alone are not entirely well-behaved. Set the climatologies in ana_tclima.h or in a file

read by get_data, set LtracerCLM in ocean_wiki_test.in and also set the array Tnudgcof

in either ana_nudgcoef.h or in a file (with name in NUDNAME variable in the ocean.in).

(c) Tides

There is also more than one way to force with tides. One way is to provide boundary conditions

with enough temporal resolution to resolve the tides. Another is to provide ROMS with the tidal

constituents at all grid points and to have ROMS reconstruct the tidal currents (UV_TIDES)

and/or elevations (SSH_TIDES) for any given time. An example of such a tidal forcing file is in

Data/ROMS/Forcing/test_head_frc.nc.

The non-trunk code with ice, etc. includes the TIDES_ASTRO option to add on the longperiod tides from Foreman [1996a] and Foreman [1996b]. There is also an option to include the

tidal potential forcing term (POT_TIDES), requiring the tidal potential to be included in the

tides forcing file.

(d) Rivers

Point sources can be used to provide river inflow to the model. These have their own input file

(SSFNAME) if not provided via ana_psource.

7.1.12 ocean.in

ROMS expects to read a number of variables from an ASCII file as described in §2.5. Example input

files are in ROMS/External with names like ocean_grav_adj.in, where “grav_adj” refers to

the name of the application. Lines beginning with “!” are comments and will be ignored by ROMS

on reading them.

The input is organized as key/value pairs, separated by one or two equals signs. It is possible

for ROMS to run on more than one grid simultaneously, with the number of grids being read from

the input file. If there is one equals sign (=), ROMS will use the corresponding value for all grids.

If there are two (==), ROMS will read a value for each grid. Thus far, most domains have used

just one grid, though I did run a two-grid Palau domain—check in Apps/Palau for a two-grid

ocean.in file (which I generated using a Python script).

ROMS will ignore the parameters not needed by the current simulation, e.g., the GLS parameters will not be read if you are not using that mixing scheme. Most of the example files contain all

possible parameters, with User/External/ocean.in being for sure updated with each addition.

The parameters are described in comments at the bottom of the ocean.in files. Skipping those

having to do with data assimilation, the input parameters are as follows:

Header

TITLE A text string to put in the output files.
 

MyAppCPP The shorthand name for this application.
 

VARNAME The location of the varinfo.dat file containing information about fields to
 

read/write from/to NetCDF files.

87
 
Ngrids The number of grids to compute on.
 

NestLayers The number of levels of nested grids.
 

GridsInLayer The number of grids in each nesting level.
 

Grid-dimension parameters

Lm Number of i-direction INTERIOR RHO-points.
 

Mm Number of j-direction INTERIOR RHO-points.
 

N Number of vertical levels.
 

Nbed Number of sediment bed layers.
 

NAT Number of active tracers (usually 2).
 

NPT Number of passive tracers.
 

NCS Number of cohesive (mud) sediment tracers.
 

NNS Number of non-cohesive (sand) sediment tracers.
 

Domain-decomposition parameters

NtileI Number of i-direction partitions.
 

NtileJ Number of j-direction partitions.
 

Boundary conditions

LBC	 A line is given for each field, each line containing four values, one for each side

in the order “West”, “South”, “East”, “North”. Each value is a string, with the

possible values being:

Cha Chapman implicit.
 

Che Chapman explicit.
 

Cla Clamped.
 

Clo Closed.
 

Fla Flather.
 

Gra Gradient.
 

Mix Ice only—mixed clamped and radiation.
 

Nes Nested.
 

Per Periodic—should match the opposing boundary.
 

Rad Radiation.
 

RadNud Radiation and nudging.
 

Red Reduced Physics—note that this also requires the FSOBC_REDUCED
 

cpp flag.
 

Shc Schepetkin.
 

VolCons Four values for the volume conservation, one for each edge.
 

Time-stepping parameters

NTIMES Number of time-steps to evolve the 3-D equations in the current run. This is

actually the total number, including any previous segments of the same run. For

instance, if you already did a three-month run and wish to continue for another

three months, set NTIMES to the number of steps needed for six months.

DT	 Time-step in seconds for the 3-D equations.

88
 
NDTFAST Number of time-steps for the 2-D equations to be executed each dt. This is

the number shown as M in Figure 5—the model will compute M∗ at run time.

Input/Output parameters

ROMS has several possible output files. The output files can include a restart file, a

history file, an averages file, and a station file, for instance. The restart file often contains

only two records with the older record being overwritten during the next write. The

history file can contain a subset of the restart fields, for instance just the surface elevation

and the surface temperature. The averages file contains time-averages of the model fields,

for instance daily or monthly means, depending on NAVG. The station file contains

timeseries for specified points, possibly quite frequently since each record is small. For

some, machinery is in place to write multiple files, numbering them _0001, _0002, etc.

NRREC Record number of the restart file to read as the initial conditions. Set to 0 at

the beginning of the run, -1 to read the latest record.

LcycleRST Logical, true to cycle between two records of the restart file.

NRST	 Number of time-steps between writing of restart fields.

NSTA	 Number of time-steps between writing fields into the stations file.

NFLT	 Number of time-steps between writing fields into the floats file.

NINFO	 Number of time-steps between calling diag to write some global information

and to check for NaN values.

History, average, diagnostic output parameters

LDEFOUT True for creating new output files for stations, history, floats, etc. If false,

output is appended to these files.

NHIS	 Number of time-steps between writing history records.

NDEFHIS Number of time-steps between starting new history files.

NQCK	 Number of time-steps between writing secondary history records.

NDEFQCK Number of time-steps between starting new secondary history files.

NTSAVG Starting time-step for the accumulation of output time-averaged data. For

instance, you might want to average over the last day of a thirty-day run.

NAVG	 Number of time-steps between writing time-averaged data into the averages file.

NDEFAVG Number of time-steps between starting new averages files.

NTSAVG2 Starting time-step for the accumulation of output secondary time-averaged

data. For instance, you might want to average over the last day of a thirty-day

run.

NAVG2 Number of time-steps between writing time-averaged data into the secondary

averages file.

NDEFAVG2 Number of time-steps between starting new secondary averages files.

NTSDIA Starting time-step for the accumulation of output diagnostics data. For instance, you might want to write diagnostics for the last day of a thirty-day run.

NDIA	 Number of time-steps between writing diagnostics data into the diagnostics file.

NDEFDIA Number of time-steps between starting new diagnostics files.

Horizontal mixing of tracers

89
 
TNU2 Constant mixing coefficient for the horizontal Laplacian diffusion of each tracer.

A value is expected for each of the NAT+NPT tracers.

TNU4 Constant mixing coefficient for the horizontal biharmonic diffusion of each tracer.

A value is expected for each of the NAT+NPT tracers.

Horizontal viscosity coefficients

VISC2 Constant mixing coefficient for the horizontal Laplacian viscosity.

VISC4 Constant mixing coefficient for the horizontal biharmonic viscosity.

Horizontal sponge

LuvSponge Logical for applying a sponge (increased horizontal viscosity) to momentum

fields.

LtracerSponge Logical for applying a sponge (increased horizontal diffusivity) to tracer

fields.

Vertical mixing coefficients for tracers

AKT_BAK Background vertical mixing coefficient for the tracers (NAT+NPT values).

Vertical mixing coefficient for momentum

AKV_BAK Background vertical mixing coefficient for momentum.

Turbulent closure parameters

AKK_BAK Background vertical mixing coefficient for turbulent kinetic energy.

AKP_BAK Background vertical mixing coefficient for turbulent generic statistical field.

TKENU2 Constant mixing coefficient for the horizontal Laplacian diffusion of turbulent

kinetic energy.

TKENU4 Constant mixing coefficient for the horizontal biharmonic diffusion of turbulent kinetic energy.

Generic length-scale turbulence closure parameters

GLS_P Stability exponent.
 

GLS_M Turbulent kinetic energy exponent.
 

GLS_N Turbulent length scale exponent.
 

GLS_Kmin Minimum value of specific turbulent kinetic energy.
 

GLS_Pmin Minimum value of dissipation.
 

GLS_CMU0 Stability coefficient.
 

GLS_C1 Shear production coefficient.
 

GLS_C2 Dissipation coefficient.
 

GLS_C3M Buoyancy production coefficient (minus).
 

GLS_C3P Buoyancy production coefficient (plus).
 

GLS_SIGK Constant Schmidt number (non-dimensional) for turbulent kinetic energy
 

diffusivity.

90
 
GLS_SIGP Constant Schmidt number (non-dimensional) for turbulent generic statistical field, “psi”.

Constants used in surface TKE flux computation

CHARNOK_ALPHA Charnok surface roughness.

ZOS_HSIG_ALPHA Roughness from wave amplitude.

SZ_ALPHA Roughness from wave dissipation.

CRGBAN_CW Craig and Banner wave breaking.

Bottom drag coefficients

RDRG Linear bottom drag coefficient.

RDRG2 Quadratic bottom drag coefficient.

Zob Bottom roughness.

Zos Surface roughness (under ice shelves).

Height of atmospheric measurements for bulk flux parameterizations

BLK_ZQ Height of air humidity values.

BLK_ZT Height of air temperature values.

BLK_ZW Height of wind values.

Wetting and drying parameter

DCRIT Minimum depth for dry cells.

Various parameters

WTYPE Jerlov water type.

LEVSFRC Deepest level to apply surface momentum stresses as a body force. Used

when the C-preprocessor option BODYFORCE is defined.

LEVBFRC Shallowest level to apply bottom momentum stresses as a body force. Used

when the C-preprocessor option BODYFORCE is defined.

Vertical coordinate parameters

Vtransform Transformation equation, 1 for old style, 2 now recommended.

Vstretching Stretching function, 1 for old style, 4 now recommended.

Vertical σ-coordinates parameters

THETA_S σ-coordinate surface control parameter, [0 < theta_s < 20 for old style].

THETA_B σ-coordinate bottom control parameter, [0 < theta_b < 1 for old style].

TCLINE Width of the surface or bottom boundary layer in which higher vertical resolution is required during stretching.

Mean Density and Brunt-Väisälä frequency

RHO0 Mean density used in the Boussinesq approximation.

BVF_BAK Background Brunt-Väisälä frequency squared.

Time parameters

91
 
DSTART Time stamp assigned to model initialization (days).

TIDE_START Time of tidal origin relative to model origin.

TIME_REF Reference time in the format yyyymmdd.dd or else a special value for

specific calendars as documented in the ocean.in files.

Nudging time scales

TNUDG Time scale (days) of nudging towards tracer climatology at the interior and at

the boundaries. A value is expected for each active tracer.

ZNUDG Time scale (days) of nudging towards free surface climatology at the interior

and at the boundaries.

M2NUDG Time scale (days) of nudging towards 2-D momentum climatology at the

interior and at the boundaries.

M3NUDG Time scale (days) of nudging towards 3-D momentum climatology at the

interior and at the boundaries.

TNUDG_SSS Time scale (days) of nudging towards surface salinity climatology.

SSS_MISMATCH_THRESHOLD Parameter when both SCORRECTION and

SSSC_THRESHOLD are used.

Open boundary factor

OBCFAC Ratio of inflow and outflow nudging time scales.

Linear equation of state parameters

R0 Background density value used in the linear equation of state.

T0 Background potential temperature constant.

S0 Background salinity constant.

TCOEF Thermal expansion coefficient in the linear equation of state.

SCOEF Saline contraction coefficient in the linear equation of state.

Slipperiness parameter

GAMMA2 Slipperiness variable, either 1.0 (free slip) or −1.0 (no slip).

Point sources

LuvSrc Logical for lateral flux point sources.

LwSrc Logical for vertical flux point sources.

LtracerSrc Logical, one value per tracer, whether or not point sources have tracer values.

Climatology nudging

LsshCLM Logical for SSH climatology.

Lm2CLM Logical for 2D momentum climatologies.

Lm3CLM Logical for 3D momentum climatologies.

LtracerCLM Logical, one value per tracer, whether or not to have tracer climatologies.

LnudgeM2CLM Logical for nudging to 2D momentum climatologies.

LnudgeM3CLM Logical for nudging to 3D momentum climatologies.

92
 
LnudgeTCLM Logical, one value per tracer, whether or not to nudge to tracer climatologies.

Logical switches to activate the writing of history/averages fields Values for all of these

for the ice model are now set in ice.in.

Hout	 History output switches, one line per variable. Check the ocean.in file for the

names of the fields involved.

Qout	 Secondary history output switches, one line per variable. Check the ocean.in

file for the names of the fields involved.

Aout	 Averages output switches, one line per variable. Check the ocean.in file for the

names of the fields involved.

Aout2	 Averages2 output switches, one line per variable. Check the ocean.in file for

the names of the fields involved.

Dout	 Diagnostics output switches, one line per term. Check the ocean.in file for the

names of the fields involved.

User parameters

NUSER Number of user parameters
 

USER Values of user parameters (NUSER values).
 

NetCDF-4/HDF5 parameters

NC_SHUFFLE If non-zero, turn on shuffle filter.
 

NC_DEFLATE If non-zero, turn on deflate filter.
 

NC_DLEVEL Deflate level (0–9).
 

Input NetCDF file names

GRDNAME Grid file.
 

ININAME Initial conditions file.
 

ITLNAME Initial tangent linear file.
 

IRPNAME Initial representer file.
 

IADNAME Initial adjoint file.
 

FWDNAME Forward model file.
 

ADSNAME Adjoint sensitivity functional file.
 

NGCNAME Nesting connectivity file.
 

NUDNAME Nudging coefficients file.
 

SSFNAME Sources/sinks forcing file.
 

Boundary, Climatology and Forcing NetCDF files These can each have an array of files,

with one set of files per variable, each file in a set for a different range of times. See

ocean.in for syntax. Note that the trunk code only does this for forcing files, while if

you use the code described here, you must specify the number of files of the appropriate

type before providing the filenames. For instance, not setting NBCFILES before setting

BRYNAME will lead to a segmentation fault.

NCLMFILES Number of climatology files.

93
 
CLMNAME Climatology file.
 

NBCFILES Number of boundary files.
 

BRYNAME Boundary condition file.
 

NFFILES Number of forcing files.
 

FRCNAME Forcing files.
 

Output NetCDF file names

DAINAME Data assimilation next cycle initial conditions file.
 

GSTNAME GST analysis restart file.
 

RSTNAME Restart file.
 

HISNAME History file.
 

QCKNAME Secondary history file.
 

TLMNAME Tangent linear file.
 

TLFNAME Impulse forcing file (for tangent linear model).
 

ADJNAME Adjoint file.
 

AVGNAME Averages file.
 

AVG2NAME Secondary averages file.
 

DIANAME Diagnostics file.
 

STANAME Stations file.
 

FLTNAME Lagrangian floats file.
 

ASCII input file names

APARNAM Assimilation parameters.
 

SPOSNAM Stations positions.
 

FPOSNAM Initial drifter positions.
 

IPARNAM Ice parameters.
 

BPARNAM Biology parameters.
 

SPARNAM Sediment parameters.
 

USRNAME User’s generic input.
 

The bottom of the sample files contain comments describing some of these in greater detail, including the data assimilation parameters.

7.1.13 User variables and subroutines

It is possible for the user to add new variables and functionality, though it is discouraged. The

design goal is to isolate the most common features a user would change to the cpp switches (§6.7),

the ana_xx.h files (§6.5) and the ASCII input file (§7.1.12). A query on the ROMS forum might

be in order if you have something specific in mind.

If you do choose to add bits, know that the makefile fragments called Module.mk will attempt

to compile and link into a library anything with a .F extension in the directories already generating

a library from such files.

94
 
7.2 Upwelling/Downwelling Example

The application which ROMS is configured to run as distributed is a wind-driven upwelling and

downwelling example, described in Macks and Middleton [1993]. There is a shelf on each wall

of a periodic channel and an along-channel wind forcing, which drives upwelling at one wall and

downwelling at the other. This problem depends on the Ekman layer, therefore a surface stress is

used with vertical viscosity. The Ekman depth is estimated to be 9 m if Av = 0.01m2/s, so the

vertical grid spacing must resolve this. The maximum depth is 150 m and our choice of the vertical

grid parameters leads to a surface Δz of 1.6 m. A possible update to this example would be to

increase N and decrease Lm, the latter because the solution does not vary in the along-channel

direction.

7.2.1 cppdefs.h

The C preprocessor variable UPWELLING is used for the upwelling configuration of the model.

The makefile will direct cppdefs.h to include the file upwelling.h:

#define UV_ADV
 

#define UV_COR
 

#define UV_LDRAG
 

#define UV_VIS2
 

#undef MIX_GEO_UV
 

#define MIX_S_UV
 

#define SPLINES_VDIFF
 

#define SPLINES_VVISC
 

#define TS_U3HADVECTION
 

#define TS_C4VADVECTION
 

#undef TS_MPDATA
 

#define DJ_GRADPS
 

#define TS_DIF2
 

#undef TS_DIF4
 

#undef MIX_GEO_TS
 

#define MIX_S_TS
 

#define SALINITY
 

#define SOLVE3D
 

#define AVERAGES
 

#define DIAGNOSTICS_TS
 

#define DIAGNOSTICS_UV
 

#define ANA_GRID
 

#define ANA_INITIAL
 

#define ANA_SMFLUX
 

#define ANA_STFLUX
 

#define ANA_SSFLUX
 

#define ANA_BTFLUX
 

#define ANA_BSFLUX
 

#if defined GLS_MIXING || defined MY25_MIXING
 

# define KANTHA_CLAYSON
 

# define N2S2_HORAVG
 

# define RI_SPLINES
 

95
 
#else
 

# define ANA_VMIX
 

#endif
 

#if defined BIO_FENNEL || defined ECOSIM || \
 

defined NPZD_POWELL || defined NEMURO || \
 

defined BIO_UMAINE
 

# define ANA_BIOLOGY
 

# define ANA_SPFLUX
 

# define ANA_BPFLUX
 

# define ANA_SRFLUX
 

#endif
 

#if defined NEMURO
 

# define HOLLING_GRAZING
 

# undef IVLEV_EXPLICIT
 

#endif
 

#ifdef BIO_FENNEL
 

# define CARBON
 

# define DENITRIFICATION
 

# define BIO_SEDIMENT
 

# define DIAGNOSTICS_BIO
 

#endif
 

#ifdef BIO_UMAINE
 

# define OXYGEN
 

# undef CARBON
 

#endif
 

#ifdef PERFECT_RESTART
 

# undef AVERAGES
 

# undef DIAGNOSTICS_BIO
 

# undef DIAGNOSTICS_TS
 

# undef DIAGNOSTICS_UV
 

# define OUT_DOUBLE
 

#endif
 

Here we have declared that we want a three-dimensional solution, but no masking. There is salinity

but we’re using a linear equation of state. The momentum equations have advection, Coriolis force

and pressure gradients. There is both horizontal viscosity and diffusion, but they are along constant

σ-surfaces and if you check the input file, you find that the horizontal diffusion is set to zero.

There are ifdefs for various biology cases, none of which have been defined. Likewise, we are

using the default of ANA_VMIX as distributed. We are asking for many other analytic functions

too, including the grid. We are asking for diagnostic output with the DIAGNOSTICS_TS and

DIAGNOSTICS_UV.

7.2.2 Model domain

The flow does not vary in x, so Lm can be almost anything. Set the values for Lm, Mm, N and

NAT in the input file:

96
 
Lm == 41 ! Number of I-direction INTERIOR RHO-points

Mm == 80 ! Number of J-direction INTERIOR RHO-points

N == 16 ! Number of vertical levels

NAT = 2 ! Number of active tracers (usually, 2)

A historical note: Lm here really is 41 for 41 boxes in the x-direction. It was originally set to 41

for 40 boxes, but the periodic boundary conditions are now more efficient and no longer need an

extra box. One could just as well set it to 40 (or even 5).

7.2.3 ana_grid

For this geometry one has a choice of using one of the external grid-generation programs or of using

ana_grid to create the grid analytically. The code in ana_grid.h was modified to produce a

bathymetry with a shelf on both walls of the channel when UPWELLING is defined. The fluid

depth ranges from 27 m on the shelves to 150 m in the center of the channel. The horizontal grid

spacing is uniform at 1 km and the Coriolis parameter f is set to a constant value suitable for

Sydney, Australia.

7.2.4 Initial conditions and the equation of state

We would like the initial conditions to be a motionless fluid with an exponential stratification. The

UPWELLING section of ana_initial.h is configured accordingly.

The stratification can be provided by either T or S, or by both T and S. For simplicity we

will only have an active temperature field and we will use the linear equation of state by setting

NONLIN_EOS to #undef. We want the density to be 26.35 at the bottom and 24.22 at the top

with an e-folding scale of 50 meters. The initial temperature is set to T0+8ez/50 in ana_initial.

The linear equation of state parameters are set in ocean_upwelling.in:

R0 == 1027.0d0 ! kg/m3

T0 == 14.0d0 ! Celsius

S0 == 35.0d0 ! PSU

TCOEF == 1.7d-4 ! 1/Celsius

SCOEF == 0.0d0 ! 1/PSU

Since density does not depend on salinity, we have a choice of how to handle the second tracer.

The salinity is set to a uniform value of S0, though it could be left out entirely if we undefine

SALINITY and set NAT to 1.

7.2.5 Boundary conditions

In ocean_upwelling.in, the variable LBC is set to be periodic in the East-West direction, with

closed walls to the North and South. No boundary values are required.

7.2.6 Model forcing

In this problem we want to resolve the surface Ekman layer and to use a surface wind stress rather

than a body force. We want the amplitude of the wind to ramp up with time so we modify

ana_smflux.h accordingly. The wind stress will build to an amplitude of 0.1 Pascals / ρo, or

10−4m2s−2.

We need to edit ana_vmix.h to make sure that the vertical viscosity Akv is set to the value

2 we want. This must be large at the surface (10−2m s−1) to create a thick Ekman layer, but has

been chosen to decrease with depth. We also need to check that ana_btflux, ana_stflux, etc.

97
 
are set correctly, in this case taking the default of zero rather than explicitly setting anything for

UPWELLING. However, we do set ana_srflux to be non-zero in case we opt to turn on one of

the biology models.

7.2.7 ocean.in

The model has been set up to run for five days with an internal time-step of 300 s and an external

time-step of 10 s.

NTIMES == 1440
 

DT == 300.0d0
 

NDTFAST == 30
 

We will write history, averages, and diagnostics records every 1/4 day, restart records once a day.

NRREC == 0
 

LcycleRST == T
 

NRST == 288
 

LDEFOUT == T
 

NHIS == 72
 

NDEFHIS == 0
 

NQCK == 0
 

NDEFQCK == 0
 

NTSAVG == 1
 

NAVG == 72
 

NDEFAVG == 0
 

NTSDIA == 1
 

NDIA == 72
 

NDEFDIA == 0
 

As stated before, there is no horizontal diffusivity, but a small horizontal viscosity has been set.

The value of the linear bottom friction coefficient rdrg is set to 3.0 × 10−4 and the channel walls

are set to be free-slip:

TNU2 == 0.0d0 0.0d0 ! m2/s
 

VISC2 == 5.0d0 ! m2/s
 

RDRG == 3.0d-04 ! m/s
 

GAMMA2 == 1.0d0
 

The vertical stretching is set to a modest value of theta_s= 3:

Vtransform == 2 ! transformation equation

Vstretching == 4 ! stretching function
 

THETA_S == 3.0d0 ! surface stretching parameter
 

THETA_B == 0.0d0 ! bottom stretching parameter
 

TCLINE == 25.0d0 ! critical depth (m)

7.2.8 Output

The model writes much information to standard out. While I like having ninfo set to 1 in general,

I set it to 72 here to save space. Also note that this was run before the secondary history output

option was added to the trunk code.

98
 
-----------------------------------------------------------------------------

Model Input Parameters:	 ROMS/TOMS version 3.7

Thursday - December 3, 2015 - 9:53:11 AM

Wind-Driven Upwelling/Downwelling over a Periodic Channel

Operating system : Darwin

CPU/hardware : x86_64

Compiler system : gfortran

Compiler command : /usr/local/bin/gfortran

Compiler flags : -frepack-arrays -O3 -ffree-form -ffree-form

SVN Root URL : https:://myroms.org/svn/src

SVN Revision : Unversioned directory

Local Root : /Users/kate/roms/feedme

Header Dir : /Users/kate/roms/feedme/ROMS/Include

Header file : upwelling.h

Analytical Dir: /Users/kate/roms/feedme/ROMS/Functionals

Resolution, Grid 01: 0041x0080x016, Parallel Threads: 1, Tiling: 001x001

Physical Parameters, Grid: 01

=============================

1440 ntimes

300.000 dt

30 ndtfast

1 ERstr

1 ERend

0 nrrec

T LcycleRST

288	 nRST

72 ninfo

T ldefout

72 nHIS

1 ntsAVG

72 nAVG

1 ntsDIA

72 nDIA

Number of timesteps for 3-D equations.
 

Timestep size (s) for 3-D equations.
 

Number of timesteps for 2-D equations between
 

each 3D timestep.

Starting ensemble/perturbation run number.

Ending ensemble/perturbation run number.

Number of restart records to read from disk.

Switch to recycle time-records in restart file.

Number of timesteps between the writing of data

into restart fields.

Number of timesteps between print of information

to standard output.

Switch to create a new output NetCDF file(s).

Number of timesteps between the writing fields

into history file.

Starting timestep for the accumulation of output

time-averaged data.

Number of timesteps between the writing of

time-averaged data into averages file.

Starting timestep for the accumulation of output

time-averaged diagnostics data.

Number of timesteps between the writing of

time-averaged data into diagnostics file.

99
 
0.0000E+00 nl_tnu2(01) NLM Horizontal, harmonic mixing coefficient

(m2/s) for tracer 01: temp

0.0000E+00 nl_tnu2(02) NLM Horizontal, harmonic mixing coefficient

(m2/s) for tracer 02: salt

5.0000E+00 nl_visc2 NLM Horizontal, harmonic mixing coefficient

(m2/s) for momentum.

F LuvSponge Turning OFF sponge on horizontal momentum.

F LtracerSponge(01) Turning OFF sponge on tracer 01: temp
 

F LtracerSponge(02) Turning OFF sponge on tracer 02: salt
 

1.0000E-06 Akt_bak(01)

1.0000E-06 Akt_bak(02)

1.0000E-05 Akv_bak

3.0000E-04 rdrg

3.0000E-03 rdrg2

2.0000E-02 Zob

2 Vtransform

4 Vstretching

3.0000E+00 theta_s

0.0000E+00 theta_b

25.000 Tcline

1025.000 rho0

0.000 dstart

0.00 time_ref

0.0000E+00 Tnudg(01)

0.0000E+00 Tnudg(02)

9.0000E+01 Tnudg_SSS

0.0000E+00 Znudg

0.0000E+00 M2nudg

0.0000E+00 M3nudg

0.0000E+00 obcfac

F VolCons(1)

F VolCons(2)

F VolCons(3)

F VolCons(4)

14.000 T0
 

35.000 S0
 

1027.000 R0
 

1.7000E-04 Tcoef

Background vertical mixing coefficient (m2/s)

for tracer 01: temp

Background vertical mixing coefficient (m2/s)

for tracer 02: salt

Background vertical mixing coefficient (m2/s)

for momentum.

Linear bottom drag coefficient (m/s).

Quadratic bottom drag coefficient.

Bottom roughness (m).

S-coordinate transformation equation.

S-coordinate stretching function.

S-coordinate surface control parameter.

S-coordinate bottom control parameter.

S-coordinate surface/bottom layer width (m) used

in vertical coordinate stretching.

Mean density (kg/m3) for Boussinesq approximation.

Time-stamp assigned to model initialization (days).

Reference time for units attribute (yyyymmdd.dd)

Nudging/relaxation time scale (days)

for tracer 01: temp

Nudging/relaxation time scale (days)

for tracer 02: salt

Nudging/relaxation time scale (days)

for sea surface salinity.

Nudging/relaxation time scale (days)

for free-surface.

Nudging/relaxation time scale (days)

for 2D momentum.

Nudging/relaxation time scale (days)

for 3D momentum.

Factor between passive and active

open boundary conditions.

NLM western edge boundary volume conservation.

NLM southern edge boundary volume conservation.

NLM eastern edge boundary volume conservation.

NLM northern edge boundary volume conservation.

Background potential temperature (C) constant.

Background salinity (PSU) constant.

Background density (kg/m3) used in linear Equation

of State.

Thermal expansion coefficient (1/Celsius).

100
 
0.0000E+00 Scoef

1.000 gamma2

F LuvSrc
 

F LwSrc
 

F LtracerSrc(01)
 

F LtracerSrc(02)
 

F LsshCLM
 

F Lm2CLM
 

F Lm3CLM
 

F LtracerCLM(01)
 

F LtracerCLM(02)
 

F LnudgeM2CLM
 

F LnudgeM3CLM
 

F LnudgeTCLM(01)
 

F LnudgeTCLM(02)
 

T Hout(idFsur)
 

T Hout(idUbar)
 

T Hout(idVbar)
 

T Hout(idUvel)
 

T Hout(idVvel)
 

T Hout(idWvel)
 

T Hout(idOvel)
 

T Hout(idTvar)
 

T Hout(idTvar)
 

T Aout(idFsur)
 

T Aout(idUbar)
 

T Aout(idVbar)
 

T Aout(idUvel)
 

T Aout(idVvel)
 

T Aout(idWvel)
 

T Aout(idOvel)
 

T Aout(idTvar)
 

T Aout(idTvar)
 

T Dout(M2rate)
 

T Dout(M2pgrd)
 

T Dout(M2fcor)
 

T Dout(M2hadv)
 

T Dout(M2xadv)
 

T Dout(M2yadv)
 

T Dout(M2hvis)
 

T Dout(M2xvis)
 

T Dout(M2yvis)
 

T Dout(M2sstr)
 

T Dout(M2bstr)
 

T Dout(M3rate)
 

T Dout(M3pgrd)
 

Saline contraction coefficient (1/PSU).

Slipperiness variable: free-slip (1.0) or

no-slip (-1.0).

Turning OFF momentum point Sources/Sinks.

Turning OFF volume influx point Sources/Sinks.

Turning OFF point Sources/Sinks on tracer 01: temp

Turning OFF point Sources/Sinks on tracer 02: salt

Turning OFF processing of SSH climatology.

Turning OFF processing of 2D momentum climatology.

Turning OFF processing of 3D momentum climatology.

Turning OFF processing of climatology tracer 01: temp

Turning OFF processing of climatology tracer 02: salt

Turning OFF nudging of 2D momentum climatology.

Turning OFF nudging of 3D momentum climatology.

Turning OFF nudging of climatology tracer 01: temp

Turning OFF nudging of climatology tracer 02: salt

Write out free-surface.

Write out 2D U-momentum component.

Write out 2D V-momentum component.

Write out 3D U-momentum component.

Write out 3D V-momentum component.

Write out W-momentum component.

Write out omega vertical velocity.

Write out tracer 01: temp

Write out tracer 02: salt

Write out averaged free-surface.
 

Write out averaged 2D U-momentum component.
 

Write out averaged 2D V-momentum component.
 

Write out averaged 3D U-momentum component.
 

Write out averaged 3D V-momentum component.
 

Write out averaged W-momentum component.
 

Write out averaged omega vertical velocity.
 

Write out averaged tracer 01: temp
 

Write out averaged tracer 02: salt
 

Write out 2D momentum acceleration.
 

Write out 2D momentum pressure gradient.
 

Write out 2D momentum Coriolis force.
 

Write out 2D momentum horizontal advection.
 

Write out 2D momentum horizontal X-advection.
 

Write out 2D momentum horizontal Y-advection.
 

Write out 2D momentum horizontal viscosity.
 

Write out 2D momentum horizontal X-viscosity.
 

Write out 2D momentum horizontal Y-viscosity.
 

Write out 2D momentum surface stress.
 

Write out 2D momentum bottom stress.
 

Write out 3D momentum acceleration.
 

Write out 3D momentum pressure gradient.
 

101
 
T Dout(M3fcor)

T Dout(M3hadv)

T Dout(M3xadv)

T Dout(M3yadv)

T Dout(M3vadv)

T Dout(M3hvis)

T Dout(M3xvis)

T Dout(M3yvis)

T Dout(M3vvis)

T Dout(iTrate)

T Dout(iTrate)

T Dout(iThadv)

T Dout(iThadv)

T Dout(iTxadv)

T Dout(iTxadv)

T Dout(iTyadv)

T Dout(iTyadv)

T Dout(iTvadv)

T Dout(iTvadv)

T Dout(iThdif)

T Dout(iThdif)

T Dout(iTxdif)

T Dout(iTxdif)

T Dout(iTydif)

T Dout(iTydif)

T Dout(iTvdif)

T Dout(iTvdif)

Output/Input Files:

Write out 3D momentum Coriolis force.
 

Write out 3D momentum horizontal advection.
 

Write out 3D momentum horizontal X-advection.
 

Write out 3D momentum horizontal Y-advection.
 

Write out 3D momentum vertical advection.
 

Write out 3D momentum horizontal viscosity.
 

Write out 3D momentum horizontal X-viscosity.
 

Write out 3D momentum horizontal Y-viscosity.
 

Write out 3D momentum vertical viscosity.
 

Write out rate of change of tracer 01: temp
 

Write out rate of change of tracer 02: salt
 

Write out horizontal advection, tracer 01: temp
 

Write out horizontal advection, tracer 02: salt
 

Write out horizontal X-advection, tracer 01: temp
 

Write out horizontal X-advection, tracer 02: salt
 

Write out horizontal Y-advection, tracer 01: temp
 

Write out horizontal Y-advection, tracer 02: salt
 

Write out vertical advection, tracer 01: temp
 

Write out vertical advection, tracer 02: salt
 

Write out horizontal diffusion, tracer 01: temp
 

Write out horizontal diffusion, tracer 02: salt
 

Write out horizontal X-diffusion, tracer 01: temp
 

Write out horizontal X-diffusion, tracer 02: salt
 

Write out horizontal Y-diffusion , tracer 01: temp
 

Write out horizontal Y-diffusion , tracer 02: salt
 

Write out vertical diffusion, tracer 01: temp
 

Write out vertical diffusion, tracer 02: salt
 

Output Restart File: ocean_rst.nc
 

Output History File: ocean_his.nc
 

Output Averages File: ocean_avg.nc
 

Output Diagnostics File: ocean_dia.nc
 

Tile partition information for Grid 01: 0041x0080x0016 tiling: 001x001

tile Istr Iend Jstr Jend Npts

Number of tracers: 2

0 1 41 1 80 52480

Tile minimum and maximum fractional coordinates for Grid 01:

(interior points only)

tile Xmin Xmax Ymin Ymax grid

0 -1.50 42.50 0.50 81.50 RHO-points

102
 
--------- ---- ---------- ---------- ---------- ----------

0 -2.00 42.00 0.50 81.50 U-points

0 -1.50 42.50 0.00 81.00 V-points

Lateral Boundary Conditions: NLM

============================

Variable Grid West Edge South Edge East Edge North Edge

zeta 1 Periodic Closed Periodic Closed
 

ubar 1 Periodic Closed Periodic Closed
 

vbar 1 Periodic Closed Periodic Closed
 

u 1 Periodic Closed Periodic Closed
 

v 1 Periodic Closed Periodic Closed
 

temp 1 Periodic Closed Periodic Closed
 

salt 1 Periodic Closed Periodic Closed
 

Activated C-preprocessing Options:
 

UPWELLING Wind-Driven Upwelling/Downwelling over a Periodic Channel
 

ANA_BSFLUX Analytical kinematic bottom salinity flux.
 

ANA_BTFLUX Analytical kinematic bottom temperature flux.
 

ANA_GRID Analytical grid set-up.
 

ANA_INITIAL Analytical initial conditions.
 

ANA_SMFLUX Analytical kinematic surface momentum flux.
 

ANA_SSFLUX Analytical kinematic surface salinity flux.
 

ANA_STFLUX Analytical kinematic surface temperature flux.
 

ANA_VMIX Analytical vertical mixing coefficients.
 

ASSUMED_SHAPE Using assumed-shape arrays.
 

AVERAGES Writing out time-averaged nonlinear model fields.
 

DIAGNOSTICS_TS Computing and writing tracer diagnostic terms.
 

DIAGNOSTICS_UV Computing and writing momentum diagnostic terms.
 

DJ_GRADPS Parabolic Splines density Jacobian (Shchepetkin, 2002).
 

DOUBLE_PRECISION Double precision arithmetic.
 

MIX_S_TS Mixing of tracers along constant S-surfaces.
 

MIX_S_UV Mixing of momentum along constant S-surfaces.
 

NONLINEAR Nonlinear Model.
 

!NONLIN_EOS Linear Equation of State for seawater.
 

POWER_LAW Power-law shape time-averaging barotropic filter.
 

PROFILE Time profiling activated .
 

!RST_SINGLE Double precision fields in restart NetCDF file.
 

SALINITY Using salinity.
 

SOLVE3D Solving 3D Primitive Equations.
 

103
 
SPLINES_VDIFF Parabolic Spline Reconstruction for Vertical Diffusion.
 

SPLINES_VVISC Parabolic Spline Reconstruction for Vertical Viscosity.
 

TS_U3HADVECTION Third-order upstream horizontal advection of tracers.
 

TS_C4VADVECTION Fourth-order centered vertical advection of tracers.
 

TS_DIF2 Harmonic mixing of tracers.
 

UV_ADV Advection of momentum.
 

UV_COR Coriolis term.
 

UV_U3HADVECTION Third-order upstream horizontal advection of 3D momentum.
 

UV_C4VADVECTION Fourth-order centered vertical advection of momentum.
 

UV_LDRAG Linear bottom stress.
 

UV_VIS2 Harmonic mixing of momentum.
 

VAR_RHO_2D Variable density barotropic mode.
 

Process Information:
 

Thread # 0 (pid= 7266) is active.
 

INITIAL: Configuring and initializing forward nonlinear model ...
 

*******
 

Vertical S-coordinate System, Grid 01:
 

level S-coord Cs-curve Z at hmin at hc half way at hmax
 

16 0.0000000 0.0000000 0.000 0.000 0.000 0.000

15 -0.0625000 -0.0019442 -0.809 -0.806 -1.348 -1.589

14 -0.1250000 -0.0078455 -1.668 -1.661 -2.966 -3.687

13 -0.1875000 -0.0179119 -2.580 -2.568 -4.867 -6.321

12 -0.2500000 -0.0324983 -3.549 -3.531 -7.077 -9.535

11 -0.3125000 -0.0521190 -4.581 -4.558 -9.630 -13.397

10 -0.3750000 -0.0774659 -5.686 -5.656 -12.573 -17.996

9 -0.4375000 -0.1094327 -6.875 -6.837 -15.967 -23.445

8 -0.5000000 -0.1491465 -8.162 -8.114 -19.889 -29.890

7 -0.5625000 -0.1980075 -9.564 -9.506 -24.435 -37.512

6 -0.6250000 -0.2577387 -11.104 -11.034 -29.721 -46.531

5 -0.6875000 -0.3304460 -12.808 -12.724 -35.892 -57.218

4 -0.7500000 -0.4186931 -14.709 -14.609 -43.121 -69.903

3 -0.8125000 -0.5255915 -16.846 -16.726 -51.622 -84.987

2 -0.8750000 -0.6549105 -19.266 -19.124 -61.651 -102.953

1 -0.9375000 -0.8112096 -22.028 -21.859 -73.518 -124.388

0 -1.0000000 -1.0000000 -25.200 -25.000 -87.600 -150.000

Time Splitting Weights for Grid 01: ndtfast = 30 nfast = 42

==================================

Primary Secondary Accumulated to Current Step

1-0.0008094437383769 0.0333333333333333-0.0008094437383769 0.0333333333333333

2-0.0014053566728197 0.0333603147912792-0.0022148004111966 0.0666936481246126

3-0.0017877524645903 0.0334071600137066-0.0040025528757869 0.1001008081383191

104
 
4-0.0019566842408176 0.0334667517625262-0.0059592371166046 0.1335675599008454

5-0.0019122901320372 0.0335319745705535-0.0078715272486418 0.1670995344713989

6-0.0016548570247459 0.0335957175749547-0.0095263842733877 0.2006952520463537

7-0.0011849025289723 0.0336508794757796-0.0107112868023601 0.2343461315221333

8-0.0005032751608632 0.0336903762267453-0.0112145619632232 0.2680365077488786

9 0.0003887272597151 0.0337071520654408-0.0108258347035082 0.3017436598143194

10 0.0014892209965583 0.0336941944901170-0.0093366137069498 0.3354378543044363

11 0.0027955815694920 0.0336445537902317-0.0065410321374578 0.3690824080946680

12 0.0043042707117221 0.0335513677379153-0.0022367614257357 0.4026337758325833

13 0.0060106451121704 0.0334078920475245 0.0037738836864347 0.4360416678801078

14 0.0079087469427945 0.0332075372104522 0.0116826306292293 0.4692492050905600

15 0.0099910761708920 0.0329439123123590 0.0216737068001212 0.5021931174029191

16 0.0122483446563884 0.0326108764399960 0.0339220514565097 0.5348039938429150

17 0.0146692120341107 0.0322025982847830 0.0485912634906204 0.5670065921276980

18 0.0172400033810439 0.0317136245503127 0.0658312668716643 0.5987202166780107

19 0.0199444086685725 0.0311389577709445 0.0857756755402368 0.6298591744489552

20 0.0227631639997064 0.0304741441486588 0.1085388395399432 0.6603333185976140

21 0.0256737146312911 0.0297153720153352 0.1342125541712343 0.6900486906129493

22 0.0286498597812017 0.0288595815276255 0.1628624139524359 0.7189082721405748

23 0.0316613792205220 0.0279045862015855 0.1945237931729579 0.7468128583421604

24 0.0346736416507075 0.0268492068942347 0.2291974348236654 0.7736620652363951

25 0.0376471948657328 0.0256934188392112 0.2668446296893983 0.7993554840756063

26 0.0405373376992233 0.0244385123436867 0.3073819673886216 0.8237939964192931

27 0.0432936737565711 0.0230872677537126 0.3506756411451927 0.8468812641730057

28 0.0458596469320356 0.0216441452951603 0.3965352880772283 0.8685254094681659

29 0.0481720587108285 0.0201154903974257 0.4447073467880568 0.8886408998655917

30 0.0501605672561820 0.0185097551070648 0.4948679140442388 0.9071506549726565

31 0.0517471682814031 0.0168377361985254 0.5466150823256418 0.9239883911711818

32 0.0528456577069106 0.0151128305891453 0.5994607400325525 0.9391012217603271

33 0.0533610761022577 0.0133513086655816 0.6528218161348102 0.9524525304259086

34 0.0531891349131380 0.0115726061288397 0.7060109510479481 0.9640251365547483

35 0.0522156244733761 0.0097996349650684 0.7582265755213241 0.9738247715198167

36 0.0503158038019030 0.0080591141492892 0.8085423793232271 0.9818838856691059

37 0.0473537721847153 0.0063819206892258 0.8558961515079425 0.9882658063583316

38 0.0431818225418188 0.0048034616164019 0.8990779740497613 0.9930692679747335

39 0.0376397765791563 0.0033640675316746 0.9367177506289176 0.9964333355064081

40 0.0305543017255205 0.0021094083123694 0.9672720523544381 0.9985427438187775

41 0.0217382098544503 0.0010909315881854 0.9890102622088884 0.9996336754069629

42 0.0109897377911117 0.0003663245930371 1.0000000000000000 1.0000000000000000

ndtfast, nfast = 30 42 nfast/ndtfast = 1.40000

Centers of gravity and integrals (values must be 1, 1, approx 1/2, 1, 1):

1.000000000000 1.047601458608 0.523800729304 1.000000000000 1.000000000000

Power filter parameters, Fgamma, gamma = 0.28400 0.18933

Metrics information for Grid 01:

===============================

105
 
Minimum X-grid spacing, DXmin = 1.00000000E+00 km

Maximum X-grid spacing, DXmax = 1.00000000E+00 km

Minimum Y-grid spacing, DYmin = 1.00000000E+00 km

Maximum Y-grid spacing, DYmax = 1.00000000E+00 km

Minimum Z-grid spacing, DZmin = 8.08965824E-01 m

Maximum Z-grid spacing, DZmax = 2.56123321E+01 m

Minimum barotropic Courant Number = 2.22358627E-01

Maximum barotropic Courant Number = 5.42494240E-01

Maximum Coriolis Courant Number = 2.47800000E-02

Basin information for Grid 01:

Maximum grid stiffness ratios:	 rx0 = 6.931666E-02 (Beckmann and Haidvogel)

rx1 = 8.661243E-01 (Haney)

Initial basin volumes: TotVolume = 3.8843755884E+11 m3

MinVolume = 8.4521383562E+05 m3

MaxVolume = 2.5612332106E+07 m3

Max/Min = 3.0302783777E+01

NL ROMS/TOMS: started time-stepping: (Grid: 01 TimeSteps: 00000001 - 00001440)

STEP	 Day HH:MM:SS KINETIC_ENRG POTEN_ENRG TOTAL_ENRG NET_VOLUME

C => (i,j,k) Cu Cv Cw Max Speed

0 0 00:00:00 0.000000E+00	 6.585677E+02 6.585677E+02 3.884376E+11

(00,00,00) 0.000000E+00 0.000000E+00 0.000000E+00 0.000000E+00

DEF_HIS - creating history file, Grid 01: ocean_his.nc

WRT_HIS - wrote history fields (Index=1,1) into time record = 0000001

DEF_AVG - creating average file, Grid 01: ocean_avg.nc

DEF_DIAGS - creating diagnostics file, Grid 01: ocean_dia.nc

72 0 06:00:00 8.578634E-06 6.585677E+02 6.585677E+02 3.884376E+11

(01,01,13) 4.614888E-03 3.202922E-04 2.134530E-03 1.975948E-02

:

1440 5 00:00:00 2.448524E-02 6.585713E+02 6.585958E+02 3.884376E+11

(01,02,10) 1.604228E-01 8.313453E-03 4.693175E-02 6.425970E-01

WRT_HIS - wrote history fields (Index=1,1) into time record = 0000021

WRT_AVG - wrote averaged fields into time record = 0000020

WRT_DIAGS - wrote diagnostics fields into time record = 0000020

WRT_RST - wrote re-start fields (Index=1,1) into time record = 0000001

Elapsed CPU time (seconds):

Thread # 0 CPU: 219.878

Total: 219.878

106
 
Nonlinear ocean model elapsed time profile, Grid: 01

Allocation and array initialization ..............
 

Ocean state initialization .......................
 

Reading of input data ............................
 

Processing of input data .........................
 

Processing of output time averaged data ..........
 

Computation of vertical boundary conditions ......
 

Computation of global information integrals ......
 

Writing of output data ...........................
 

Model 2D kernel ..................................
 

2D/3D coupling, vertical metrics .................
 

Omega vertical velocity ..........................
 

Equation of state for seawater ...................
 

3D equations right-side terms ....................
 

3D equations predictor step ......................
 

Pressure gradient ................................
 

Harmonic mixing of tracers, S-surfaces ...........
 

Harmonic stress tensor, S-surfaces ...............
 

Corrector time-step for 3D momentum ..............
 

Corrector time-step for tracers ..................
 

Total:
 

All percentages are with respect to total time =

ROMS/TOMS - Output NetCDF summary for Grid 01:

0.083 ( 0.0377 %)

0.009 ( 0.0043 %)

0.006 ( 0.0028 %)

0.099 ( 0.0452 %)

22.404 (10.1894 %)

0.082 ( 0.0373 %)

0.047 ( 0.0213 %)

1.255 ( 0.5708 %)

111.940 (50.9101 %)

2.108 ( 0.9589 %)

1.248 ( 0.5676 %)

0.872 ( 0.3965 %)

9.147 ( 4.1603 %)

20.887 ( 9.4995 %)

5.047 ( 2.2954 %)

3.353 ( 1.5250 %)

3.882 ( 1.7657 %)

18.233 ( 8.2926 %)

15.304 ( 6.9604 %)

216.009 98.2406

219.878

number of time records written in HISTORY file = 00000021

number of time records written in RESTART file = 00000002

number of time records written in AVERAGE file = 00000020

Analytical header files used:

ROMS/Functionals/ana_btflux.h
 

ROMS/Functionals/ana_grid.h
 

ROMS/Functionals/ana_initial.h
 

ROMS/Functionals/ana_smflux.h
 

ROMS/Functionals/ana_stflux.h
 

ROMS/Functionals/ana_vmix.h
 

ROMS/TOMS: DONE... Thursday - December 3, 2015 - 9:56:54 AM

Note that it ends by printing out a profile of where the time was used—the ratios will vary depending

on the application.

NetCDF history and restart files are also created, containing the model fields at the requested

times. We have asked it to save restart records every day. In this case, the restart file has been told

to “cycle”, or to write over the second last record. The restart file at the end of the run contains

the fields at day 4 and day 5. The history file contains records for 0, 1/4, 1/2, 3/4, and 1 day,

etc., while the averages and diagnostics files are at 1/8, 3/8, 5/8, and 7/8 day, etc.. Plots can be

made from any one of these files, using the plotting software described in ROMS plotting package.

Selected frames from such plots for an older run are shown in Fig. 18 to 21.

107
 
Figure 18: The upwelling/downwelling bathymetry.
 

108
 
Figure 19: Surface velocities after one day, showing the flow to the left of the wind (southern

hemisphere).

109
 
Figure 20: Constant ξ slices of the u, v, T and w fields at day 1.
 

110
 
Figure 21: Constant ξ slices of the u, v, T, and w fields at day 5.
 

111
 
7.3 Arctic example

The upwelling/downwelling examples is one in which all the start-up fields are defined analytically.

The other extreme is one in which everything is read from files, as in our Arctic simulations. Figure

22 shows the bathymetry and the extent of this domain. The grid is non-uniform, having more

resolution close to Alaska—a measure of grid spacing is shown in Figure 23.

7.3.1 arctic.h

The C preprocessor variable ARCTIC has been chosen to label our Arctic domain—the header

include file then becomes arctic.h:

/*
 

** svn $Id$
 

*******************************************************************************
 

** Copyright (c) 2002-2015 The ROMS/TOMS Group
 

**
 

** Licensed under a MIT/X style license
 

**
 

** See License_ROMS.txt
 

**
 

*******************************************************************************
 

**
 

** Options for ARCTIC simulation
 

*/
 

#define NO_HIS
 

#define GLOBAL_PERIODIC
 

#define HDF5
 

#define DEFLATE
 

#undef PARALLEL_IN
 

#undef PARALLEL_OUT
 

#define PERFECT_RESTART
 

/* general */
 

#define CURVGRID
 

#define MASKING
 

#define NONLIN_EOS
 

#define SOLVE3D
 

#define SALINITY
 

#ifdef SOLVE3D
 

# define SPLINES_VDIFF
 

# define SPLINES_VVISC
 

# define RI_SPLINES
 

#endif
 

#define FLOATS
 

#define STATIONS
 

#undef WET_DRY
 

#define T_PASSIVE
 

#ifdef T_PASSIVE
 

112
 
Figure 22: Bathymetry of the Arctic domain.
 

113
 
Figure 23: Grid spacing of the Arctic domain.
 

114
 
# define ANA_BPFLUX /* analytical bottom passive tracers fluxes */
 

# define ANA_SPFLUX

# define ANA_PASSIVE

# define TRC_PSOURCE

# define ANA_TRC_PSOURCE

# define AGE_MEAN

# define PTOBC

#endif

/* ice */

#ifdef SOLVE3D

# undef CICE_MODEL

# ifdef CICE_MODEL

# define SNOWFALL

# define SNOW_FROM_RAIN

# define INI_GLORYS_ICE

# endif

# define ICE_MODEL

# ifdef ICE_MODEL

# define ANA_ICE

# define INI_GLORYS_ICE

# define OUTFLOW_MASK

# undef FASTICE_CLIMATOLOGY

# define ICE_LANDFAST

# define ICE_THERMO

# define ICE_MK

# define ICE_MOMENTUM

# define ICE_MOM_BULK

# define ICE_EVP

# define ICE_STRENGTH_QUAD

# define ICE_ADVECT

# define ICE_SMOLAR

# define ICE_UPWIND

# define ICE_BULK_FLUXES

# define ICE_CONVSNOW

# define ICE_I_O

# endif

#endif

/* output stuff */

#define NO_WRITE_GRID

#undef OUT_DOUBLE

#ifndef PERFECT_RESTART

# define RST_SINGLE

#endif

#define AVERAGES

#define AVERAGES2

115
 
#ifdef SOLVE3D

# undef AVERAGES_DETIDE

# undef DIAGNOSTICS_TS

#endif

#undef DIAGNOSTICS_UV

/* advection, dissipation, pressure grad, etc. */

#ifdef SOLVE3D

# define DJ_GRADPS

#endif

#define UV_ADV

#define UV_COR

#ifdef SOLVE3D

# define TS_U3HADVECTION

# define TS_C4VADVECTION

# undef TS_MPDATA

#endif

#define UV_VIS2

#undef UV_SMAGORINSKY

#undef VISC_3DCOEF

#define MIX_S_UV

#define VISC_GRID

#ifdef SOLVE3D

# define TS_DIF2

# define MIX_GEO_TS

# define DIFF_GRID

#endif

/* vertical mixing */

#ifdef SOLVE3D

# define WTYPE_GRID

# define LMD_MIXING

# ifdef LMD_MIXING

# define LMD_RIMIX

# define LMD_CONVEC

# define LMD_SKPP

# undef LMD_BKPP

# define LMD_NONLOCAL

# define LMD_SHAPIRO

# undef LMD_DDMIX

# endif

# undef GLS_MIXING

116
 
# undef MY25_MIXING
 

# if defined GLS_MIXING || defined MY25_MIXING

# define KANTHA_CLAYSON

# define N2S2_HORAVG

# endif

#endif

/* surface forcing */

#ifdef SOLVE3D

# define CORE_FORCING

# define BULK_FLUXES

# define CCSM_FLUXES

# undef ARCTIC_MERRA_HACK

# if defined BULK_FLUXES

# define LONGWAVE_OUT

# undef DIURNAL_SRFLUX

# define SOLAR_SOURCE

# define EMINUSP

# undef ALBEDO_CLOUD

# define ALBEDO_CURVE /* for water */

# undef ICE_ALB_EC92 /* for ice */

# define ALBEDO_CSIM /* for ice */

# undef ALBEDO_FILE /* for both */

# undef LONGWAVE

# endif

#endif

/* surface and side corrections */

#ifdef SOLVE3D

# define SCORRECTION

# define NO_SCORRECTION_ICE

#endif

#ifdef SOLVE3D

# define ANA_NUDGCOEF

#endif

/* point sources (rivers, line sources) */

/* Using both for different regions */

#ifdef SOLVE3D

# define RUNOFF

# define TWO_D_TRACER_SOURCE

#endif

/* tides */

117
 
#define LTIDES

#ifdef LTIDES

# if defined AVERAGES && !defined USE_DEBUG

# undef FILTERED

# endif

# define SSH_TIDES

# define UV_TIDES

# define ADD_FSOBC

# define ADD_M2OBC

# undef RAMP_TIDES

# define TIDES_ASTRO

# define POT_TIDES

# define UV_DRAG_GRID

# define ANA_DRAG

# define LIMIT_BSTRESS

# undef UV_LDRAG

# define UV_QDRAG

#else

# define UV_QDRAG

#endif

/* Boundary conditions...careful with grid orientation */

#define RADIATION_2D

/* roms quirks */

#ifdef SOLVE3D

# define ANA_BSFLUX

# define ANA_BTFLUX

#else

# define ANA_SMFLUX

#endif

We’ve got masking, salinity, sea ice and the non-linear equation of state. We want Laplacian viscosity on σ-surfaces, diffusion along constant z-surfaces and the full non-linear, curvilinear momentum

equations.

Notice that comments are allowed in this file as long as they are in the C syntax. Also,

conditionals are fine—some parts are only wanted when SOLVE3D is on. Sources are now turned

on in the ocean_arctic2.in file, while RUNOFF is still a cpp option. We have both in this

domain, using different fresh water inputs for different parts of the domain.

7.3.2 Arctic code chunks

The Apps/Arctic directory contains the ana_ family of include files for setting the bottom drag,

the horizontal nudging coefficients and the passive tracers. The bottom drag is a function of the

bottom depth with a tunable parameter which was adjusted to give a reasonable Bering Strait

throughflow.

Doing a search for ARCTIC shows up a few other bits of code. One is in output.F, forcing

a new station file after every restart:

118
 
#ifdef ARCTIC

CALL def_station (ng, .true.)

#else

CALL def_station (ng, ldefout(ng))

#endif

A better fix (someday) would be to allow the station files to be numbered just like the averages

files. Why not set ldefout to be true? It is shared by many outputs, including the floats, and you

can only restart floats with ldefout set to false.

There is a hack in grid_coords.F so that floats and stations are placed at the correct i, j

position. Otherwise, the code to find i, j from longitude, latitude finds two boxes with the correct

range, one where I want it, the other at the same latitude, but where the longitude jumps from 360

back to 0.

There is code in set_masks.F to position the mask for the ice outflow cells off the coast of

Greenland.

Finally, there is a kludge in step_floats.F for a float reuse strategy, restarting them each year.

Having a handy tag like ARCTIC allows me to search for these little hacks later.

7.3.3 Model domain

A number of horizontal grid points was chosen to resolve the domain at roughly 6 km near Alaska,

less well across the basin. Values for Lm, Mm, and N are:

Lm == 688 ! Number of I-direction INTERIOR RHO-points

Mm == 1088 ! Number of J-direction INTERIOR RHO-points

N == 50 ! Number of vertical levels

The vertical structure is set to our new favorite, using the new stretching function:

Vtransform == 2 ! transformation equation
 

Vstretching == 4 ! stretching function
 

THETA_S == 7.0d0 ! surface stretching parameter

THETA_B == 2.0d0 ! bottom stretching parameter

TCLINE == 250.0d0 ! critical depth (m)

7.3.4 Initial and boundary conditions

The best ocean fields we knew of when setting this up are known as SODA, a reanalysis from

1958 through 2008 by Carton et al. [2005]. There are global HYCOM fields for later years. There

are Python scripts to create initial, climatology and boundary conditions from the SODA and

HYCOM files. Ice initial thickness and concentration are extracted from a GLORYS simulation

while the other initial ice conditions are set to uniform values where the SST is cold, zero otherwise.

Ice boundary conditions are assumed to be not needed with the boundaries so far from the pack

ice.

The side known as “South” is across the North Pacific and the “East” and “West” sides are only

open for a short bit in the North Pacific. The “North” boundary is on the Atlantic side; these all

have our usual combination of radiation and nudging for the ocean. The ice boundary conditions

are closed here, but “outflow cells” have been created along the “North” side.

LBC(isFsur) == Che Che Che Che ! free-surface

LBC(isUbar) == Shc Shc Shc Shc ! 2D U-momentum

LBC(isVbar) == Shc Shc Shc Shc ! 2D V-momentum

119
 
LBC(isUvel) == RadNud RadNud RadNud RadNud ! 3D U-momentum

LBC(isVvel) == RadNud RadNud RadNud RadNud ! 3D V-momentum

LBC(isMtke) == Clo Clo Clo Clo ! mixing TKE

LBC(isTvar) == RadNud RadNud RadNud RadNud \ ! temperature

RadNud RadNud RadNud RadNud \ ! salinity

Clo Clo Clo Clo \ ! inert(1)

Clo Clo Clo Clo \ ! inert(2)

Clo Clo Clo Clo \ ! inert(3)

Clo Clo Clo Clo \ ! inert(4)

Clo Clo Clo Clo \ ! inert(5)

Clo Clo Clo Clo ! inert(6)

! Ice boundary conditions

LBC(isAice) == Clo Clo Clo Clo ! ice concentration

LBC(isHice) == Clo Clo Clo Clo ! ice thickness

LBC(isHsno) == Clo Clo Clo Clo ! snow thickness

LBC(isTice) == Clo Clo Clo Clo ! ice temperature

LBC(isApond)== Clo Clo Clo Clo ! surface water

LBC(isHpond)== Clo Clo Clo Clo ! surface water

LBC(isSig11)== Clo Clo Clo Clo ! sigma-11

LBC(isSig12)== Clo Clo Clo Clo ! sigma-12

LBC(isSig22)== Clo Clo Clo Clo ! sigma-22

LBC(isUice) == Gra Gra Gra Gra ! ice U-momentum

LBC(isVice) == Gra Gra Gra Gra ! ice V-momentum

7.3.5 Forcing

We are using the MERRA forcing files [Rienecker et al., 2011] and computing the momentum,

heat and salt fluxes from the atmospheric conditions and the model’s surface temperature. There

are two options, one provided by bulk_flux.F and the other provided by ccsm_flux.F. We are

using the second. We have Python scripts for downloading MERRA so that the MERRA files can

be used as is, on their native grid, then interpolated by ROMS internally to the domain at hand.

We have added a processing step to the MERRA files so that only values from over the ocean

are used—these are extrapolated into the land points in the MERRA fields (R. Dussin, personal

communication).

For fresh water input from land, we have the ARDAT fields for the Arctic ocean [Whitefield

et al., 2015], which includes monthly river temperature. These are applied as point sources to

include the effects of bringing in warm water. For the rest of the domain, we are applying Dai

et al. [2009] as the RUNOFF field. Other forcings include a nudging to sea surface salinity, tides

from OTPS and a nudging to a monthly climatology around the edges from the same sources as

the boundary conditions. We were using a landfast ice climatology for the Beaufort Sea from Andy

Mahoney, but have since switched to a landfast ice parameterization from Lemieux et al. [2015].

7.3.6 ocean.in

We use an internal time-step of 60 to 100 s and an external time-step of 10 s. The horizontal

viscosity and diffusion is:

TNU2 == 5.0d0 5.0d0 ! m2/s

120
 
VISC2 == 25.0d0 ! m2/s

7.3.7 Output

The model writes voluminous information to standard out, as shown for the UPWELLING case.

It also writes out NetCDF files for restart, daily averages, three-hourly averages2, stations and

floats. Plots can be made from any of these files; some examples are shown in Fig. 24–25 .

7.4 Northwest Gulf of Alaska example

The Northwest Gulf of Alaska (NWGOA) domain is set up much like the Arctic, described in §7.3.

This is a rectangular domain with uniform 1.5 km resolution over Cook Inlet and beyond, as shown

in Fig. 26.

The bathymetry was clipped to 20 m elevation above sea level for the wetting and drying algorithm. Smoothing and editing was done to give a stable simulation, but the usual bottom steepness

metric depends on water depth being everywhere positive. The assumption is that extremely shallow waters would be vertically well-mixed, so that pressure gradient errors would vanish. It was

discovered that the rotated mixing tensors for horizontal diffusion were unstable in the near-shore,

with high-and-dry points next to relatively deep wet points; thus the explicit horizontal diffusion

was turned off. The model ran stably with the third-order upwind advection scheme providing

some implicit diffusion.

The bathymetry is a combination of ARDEM (Danielson, personal communication) and a Cook

Inlet product from Lyon Lanerolle (personal communication). This melding was found to give

better tides at Anchorage than either product alone.

7.4.1 nwgoa.h

The C preprocessor variable NWGOA has been chosen to label this domain—the header include

file then becomes nwgoa.h. It shares many of the options with the Arctic domain, differing in that

has neither outflow cells nor any attempt at landfast ice. One notable change is the inclusion of

the WET_DRY option, which was challenging to get to run stably. Another change is that fresh

water is now all coming from the output of a hydrology model—RUNOFF is not used here.

7.4.2 NWGOA code chunks

The Apps/NWGOA directory contains the ana_ family of include files for setting the bottom

drag, the horizontal nudging coefficients and the passive tracers. Doing a search for NWGOA

shows up a few other bits of code. The only such bit not shared by ARCTIC is in mod_scalars.F:

#ifdef NWGOA

real(r8) :: max_speed = 80.0_r8 ! m/s

#else

real(r8) :: max_speed = 20.0_r8 ! m/s

#endif

This is because the summer river flows can be large enough to exceed the default “speed limit” put

into the code to check for growing instabilities.

7.4.3 Model domain

A number of horizontal grid points was chosen to resolve the domain at 1.5 km everywhere. Values

for Lm, Mm, and N are:

121
 
Figure 24: Ice concentration averaged over the month of September, 1982.
 

122
 
Figure 25: Ice thickness averaged over the month of September, 1982.
 

123
 
Figure 26: Bathymetry of the NWGOA domain.

Lm == 792 ! Number of I-direction INTERIOR RHO-points

Mm == 360 ! Number of J-direction INTERIOR RHO-points

N == 50 ! Number of vertical levels

The vertical structure is again our current favorite:

Vtransform == 2 ! transformation equation
 

Vstretching == 4 ! stretching function
 

THETA_S == 7.0d0 ! surface stretching parameter

THETA_B == 2.0d0 ! bottom stretching parameter

TCLINE == 250.0d0 ! critical depth (m)

7.4.4 Initial and boundary conditions

All initial and boundary conditions were extracted from prior runs in the NEP6 domain covering

the Northeast Pacific. The LBC options are the same as in the Arctic for all open boundaries.

There is no need for any nudging to climatology in this case.

7.4.5 Forcing

We are using the same MERRA forcing with extrapolation into land as described above, as well as

a tidal file from OTPS. The fresh water forcing is daily output from a hydrology model [Beamer

et al., 2016], interpolated to the ROMS grid and then converted to lateral “sources” along the

coastline.

7.4.6 ocean.in

We use an internal time-step of 30 s and an external time-step of 3 s. The horizontal viscosity is:

124
 
Figure 27: Surface salinity for 10 August, 2008.

VISC2 == 25.0d0 ! m2/s

while the explicit horizontal diffusivity was turned off.

7.4.7 Output

The model writes voluminous information to standard out, as shown for the UPWELLING case.

It also writes out NetCDF files for restart, daily averages, stations and floats. We also saved hourly

history output. Plots can be made from any of these files; some examples are shown in Fig. 27–28 .

7.5 Beaufort Sea example

The Beaufort Sea (BEAUFORT) domains are set up much like the Arctic, described in §7.3. This

is actually two domains, a roughly 3 km curvilinear domain (Beaufort2), and a rectangular domain

with uniform 0.5 km resolution over the coastal Beaufort (Beaufort3), as shown in Fig. 29–30.

The bathymetry was clipped to 3 m and 2 m depth for the two domains, respectively. The

wetting and drying algorithm was turned on later for the small domain after an enclosed bay

became unstable several years into the run. The bathymetry is from ARDEM version 2 (Danielson,

personal communication).

125
 
Figure 28: Surface temperature and velocities for 10 August, 2008.
 

Figure 29: Bathymetry of the larger Beaufort Sea domain.
 

126
 
Figure 30: Bathymetry of the smaller Beaufort Sea domain.
 

7.5.1 beaufort.h

The C preprocessor variable BEAUFORT has been chosen to label both of these domains—the

header include file then becomes beaufort.h. It shares many of the options with the Arctic domain,

differing in that there are no outflow cells and in that we’ve enabled the KPP bottom boundary

layer code. Like the Arctic, this domain uses the ARDAT hydrology for fresh water inputs, but

unlike the Arctic, does not also include RUNOFF.

7.5.2 BEAUFORT code chunks

The Apps/Beaufort directory contains the ana_ family of include files for setting the bottom

drag and the horizontal nudging coefficients. The only novelty here is that the two grids have

differing nudging coefficients along the eastern boundary:

! cff3-point wide linearly tapered nudging zone

IF (Lm(ng) == 936) THEN ! Beaufort3 grid only

DO i=MAX(IstrT,Lm(ng)+1-INT(cff3)),IendT ! EAST boundary

DO j=JstrT,JendT

wrk(i,j)=MAX(wrk(i,j), &

& cff1+REAL(Lm(ng)+1-i,r8)*(cff2-cff1)/cff3)

END DO

END DO

END IF

since it is only an open boundary in the smaller domain.

7.5.3 Model domains

A number of horizontal grid points was chosen to resolve the outer domain at roughly 3.0 km

everywhere, while the inner domain has a uniform 0.5 km resolution. Values for Lm, Mm, and N

are:

Lm == 400 ! Number of I-direction INTERIOR RHO-points

Mm == 96 ! Number of J-direction INTERIOR RHO-points

N == 50 ! Number of vertical levels

Lm == 936 ! Number of I-direction INTERIOR RHO-points

127
 
Mm == 160 ! Number of J-direction INTERIOR RHO-points

N == 30 ! Number of vertical levels

The smaller domain is entirely on the shelf and was deemed too shallow to require our usual 50

levels. The vertical structure is again our current favorite:

Vtransform == 2 ! transformation equation
 

Vstretching == 4 ! stretching function
 

THETA_S == 7.0d0 ! surface stretching parameter

THETA_B == 2.0d0 ! bottom stretching parameter

TCLINE == 250.0d0 ! critical depth (m)

7.5.4 Initial and boundary conditions

All initial and boundary conditions for the larger domain were extracted from prior runs of the

ARCTIC domain while those for the smaller domain were extracted from the larger domain. In

both cases, we use higher frequencies of one to three hours for the ice fields and daily averages

for the ocean fields. The LBC options are the same as in the ARCTIC for all open boundaries,

except we now use “RadNud” for the ice velocities and “Mix” for all the other ice variables. There

is no need for any nudging to climatology in the ocean, but we use a nudging band to monthly

climatologies for some sea ice variables with a relatively short (three-day) timescale close to the

boundary.

7.5.5 Forcing

We use the same MERRA forcing with extrapolation into land as described above, as well as a

tidal file from OTPS. I’ve omitted the tidal potential contribution in these domains. The fresh

water forcing is from the monthly ARDAT climatology [Whitefield et al., 2015], interpolated to the

ROMS grid and then converted to lateral “sources” along the coastline.

7.5.6 ocean.in

We use an internal time-step of 60 s for the larger domain and an internal time-step of 45 s for the

smaller domain. Both have an external-internal time-step ratio of 20. The horizontal diffusivity

and viscosity for the larger domain are:

TNU2 == 2*1.0d0 ! m2/s
 

VISC2 == 5.0d0 ! m2/s
 

while the smaller has:

TNU2 == 2*0.2d0 ! m2/s
 

VISC2 == 1.0d0 ! m2/s
 

7.5.7 Output

The model writes voluminous information to standard out, as shown for the UPWELLING case.

It also writes out NetCDF files for restart, daily averages, three-hourly averaged surface fields,

stations and floats. Plots can be made from any of these files; some examples are shown in Fig.

31–33. Fig. 31 shows the land-fast ice extent, in which it is clear that I’ve turned off the land-fast

ice parameterization within the boundary nudging band.

128
 
Figure 31: Ice speed contour at 0.0005 for the April climatology in the smaller Beaufort domain.

This contour was chosen as an estimate of the land-fast ice extent.

Figure 32: Sea-ice concentration for the June climatology in the smaller Beaufort domain. Note

the location of a few warm rivers between 148 and 151 W longitude.

129
 
Figure 33: Vertical slices for the August climatology in the smaller Beaufort domain, (a) temperature, (b) salinity, (c) longshore velocity and (d) cross-shore velocity. Note the warm, fresh water

right up on the shelf, trapped by a barrier island.

130
 
A Model Time-stepping Schemes

Numerical time stepping uses a discrete approximation to:

∂φ(t) = F(t) (247) ∂t

where φ represents one of u, v, C, or ζ, and F(t) represents all the right-hand-side terms. In ROMS,

the goal is to find time-stepping schemes which are accurate where they are valid and damping on

unresolved signals [Shchepetkin and McWilliams, 2009b]. Also, the preference is for time-stepping

schemes requiring only one set of the right-hand-side terms so that different time-stepping schemes

can be used for different terms in the equations. Finally, as mentioned in Table 4.3, not all versions

of ROMS use the same time-stepping algorithm. We list some time-stepping schemes here which

are used or have been used by the ROMS/SCRUM family of models, plus a few to help explain

some of the more esoteric ones.

A.1 Euler

The simplest approximation is the Euler time step:

φ(t + Δt) − φ(t) = F(t) (248) Δt

where you predict the next φ value based only on the current fields. This method is accurate to

first order in Δt; however, it is unconditionally unstable with respect to advection.

A.2 Leapfrog

The leapfrog time step is accurate to O(Δt

2):

φ(t + Δt) − φ(t − Δt) = F(t). (249) 2Δt

This time step is more accurate, but it is unconditionally unstable with respect to diffusion. Also,

the even and odd time steps tend to diverge in a computational mode. This computational mode

can be damped by taking correction steps. SCRUM’s time step on the depth-integrated equations

was a leapfrog step with a trapezoidal correction (LF-TR) on every step, which uses a leapfrog step

to obtain an initial guess of φ(t + Δt). We will call the right-hand-side terms calculated from this

initial guess F∗(t + Δt):

φ(t + Δt) − φ(t) 1 = [F(t) + F∗(t + Δt)] . (250) Δt 2

This leapfrog-trapezoidal time step is stable with respect to diffusion and it strongly damps the

computational mode. However, the right-hand-side terms are computed twice per time step.

A.3 Third-order Adams-Bashforth (AB3)

The time step on SCRUM’s full 3-D fields is done with a third-order Adams-Bashforth step. It

uses three time-levels of the right-hand-side terms:

φ(t + Δt) − φ(t) = αF(t) + βF(t − Δt) + γF(t − 2Δt) (251) Δt

where the coefficients α, β and γ are chosen to obtain a third-order estimate of φ(t + Δt). We use

a Taylor series expansion:

φ(t + Δt) − φ(t) Δ

2

t

φ'' Δt

2

φ''' = φ' + + + · · · (252) Δt 6

131
 
 

 

 

where
 

F(t) = φ'

Δ 2t φ' − Δtφ'' φ''' F(t − Δt) = + + · · · 2

φ' 2φ''' F(t − 2Δt) = − 2Δtφ'' + 2Δt + · · ·

We find that the coefficients are:

23 4 5

α = β = − γ = 3, 12, 12

This requires one time level for the physical fields and three time levels of the right-hand-side

information and requires special treatment on startup.

A.4 Forward-Backward

In equation 247 above, we assume that multiple equations for any number of variables are time

stepped synchronously. For coupled equations, we can actually do better by time stepping asynchronously. Consider these equations:

∂ζ = F(u) ∂t (253) ∂u = G(ζ) ∂t

If we time step them alternately, we can always be using the newest information:

ζn+1 = ζn + F(un)Δt

n+1 (254) n + G(ζn+1 u = u )Δt

This scheme is second-order accurate and is stable for longer time steps than many other schemes.

It is however unstable for advection terms.

A.5 Forward-Backward Feedback (RK2-FB)

One option for solving equation 253 is a predictor-corrector with predictor step:

ζn+1,* = ζn + F(un)Δt

(255) n+1,* u = un + βG(ζn+1,*) + (1 − β)G(ζn) Δt

and corrector step:

ζn+1 = ζn +

1

2 F(un+1,*) + F(un) Δt

(256) n+1 u = un +

1

EG(ζn+1) + (1 − E)G(ζn+1,*) + G(ζn) Δt

2

Setting β = E = 0 in the above, it becomes a standard second order Runge-Kutta scheme, which is

unstable for a non-dissipative system. Adding the β and E terms adds Forward-Backward feedback

to this algorithm, and allows us to improve both its accuracy and stability. The choice of β =

1/3 and E = 2/3 leads to a stable third-order scheme with αmax = 2.14093 [Shchepetkin and

McWilliams, 2009b].

132
 
 

 

 

 

 

A.6 LF-TR and LF-AM3 with FB Feedback

Another possibility is a leapfrog predictor:

ζn+1,* = ζn−1 + 2F(u

(1 − 2β)G(ζn) + β G(ζn+1,*) + G(ζn−1)

 

n)Δt

 (257)

u
 

n+1,* = un−1 + 2 Δt
 

and either a two-time trapezoidal or a three-time Adams-Moulton corrector:

ζn+1 = ζn +

1

2 − γ F(un+1,*) + 1

+ 2γ F(un) − γF(un−1) Δt

2 (258) 1 1 n+1 u = un + 2 − γ EG(ζn+1) + (1 − E)G(ζn+1,*) + + 2γ G(ζn) − γG(ζn−1) Δt

2

where the parameters β and E introduce FB-feedback during both stages, while γ controls the type

of corrector scheme. Without FB-feedback, we have:

β = E = 0 ⇒
 

⎧

⎪⎨

⎪⎩

√ γ = 0 ⇒ LF-TR αmax = 2

γ = 1/12 ⇒ LF-AM3 αmax = 1.5874

γ = 0.0804 ⇒ max stability αmax = 1.5876

Keeping γ = 1/12 maintains third-order accuracy. The most accurate choices for β and E are

β = 17/120 and E = 11/20, which yields a fourth-order scheme with low numerical dispersion and

diffusion and αmax = 1.851640 [Shchepetkin and McWilliams, 2009b].

A.7 Generalized FB with an AB3-AM4 Step

One drawback of the previous two schemes is the need to evaluate the right-hand-side (r.h.s.) terms

twice per time step. The original forward-backward scheme manages to achieve αmax = 2 while

only evaluating each r.h.s. term once. It is possible to contruct a robust scheme using a combination

of a three-time AB3-like step for ζ with a four-time AM4-like step for u:

3 1 ζn+1 = ζn + 2 + β F(un) − 2 + 2β F(un−1) + βF(un−2) Δt

(259) 1 1 n+1 n u = u + + γ + 2E G(ζn+1) + 2 − 2γ − 3E G(ζn) + γG(ζn−1) + EG(ζn−2) Δt

2

This is second-order accurate for any set of values for β, γ, and E. It is third-order accurate if

β = 5/12. However, it has a wider stability range for β = 0.281105. [Shchepetkin and McWilliams,

2009b] choose to use this scheme for the barotropic mode in their model with β = 0.281105,

γ = 0.0880, and E = 0.013, to obtain αmax = 1.7802, close to the value of 2 for pure FB, but with

better stability properties for the combination of waves, advection, and Coriolis terms.

133
 
 

 

 

B The vertical σ-coordinate
 

ROMS now supports several vertical coordinate systems as fully described on the ROMS wiki.

See also Shchepetkin and McWilliams [2005]. The currently recommended values correspond to

Vtransform= 2 and Vstretching= 4, therefore that is what is described here.

B.1 Transform

The vertical coordinate corresponding to Vtransform= 2 is:

z(x, y, σ,t) = ζ(x, y,t) + [ζ(x, y,t) + h(x, y)] S(x, y, σ), (260)

σ + h(x, y) C(σ) S(x, y, σ) = hc	 (261) hc + h(x, y)

where hc is a (positive) thickness above which we wish to have more resolution and C(σ) is a

nondimensional, monotonic, vertical stretching function ranging from −1 ≤ C(σ) ≤ 0.

In an undisturbed ocean state, corresponding to zero free-surface, z = S(x, y, σ) so that in

our coordinate system, the depths are not affected by the displacements of the free-surface. This

ensures that the vertical mass fluxes generated by a purely barotropic motion will vanish at every

interface.

We find it convenient to define:

Hz ≡

∂z (262) ∂σ

We compute Hz discretely as Δz/Δσ since this leads to the vertical sum of Hz being exactly the

total water depth D.

Notice that, unlike our previous stretching function,

0, if σ = 0, C(σ) = 0, at the free-surface; S(x, y, σ) =	 (263) −1, if σ = −1, C(σ) = −1, at the ocean bottom;

The new stretching is superior in that:

•	 Regardless of the design of C(σ), it behaves like equally-spaced sigma-coordinates in shallow

regions, where h(x, y) � hc. This is advantageous because it avoids excessive resolution and

associated CFL limitation is such areas.

•	 Near-surface refinement behaves more or less like geopotential coordinates in deep regions

(level thicknesses, Hz, do not depend or weakly depend on bathymetry), while near-bottom

like sigma coordinates (Hz is roughly proportional to depth). This reduces the extreme

r-factors near the bottom and reduces pressure gradient errors.

•	 The “true” sigma-coordinate system is recovered as hc → ∞. This is useful when configuring applications with “flat” bathymetry and “uniform” level thickness. Practically, you can

achieve this by setting Tcline to 1.0d+16 in ocean.in. This will set hc = 1.0x1016. Although

not necessary, we also recommend that you set θS = 0 and θB = 0.

•	 In an undisturbed ocean state, ζ ≡ 0, transformation (2) yields the following unperturbed

depths, zˆ,

hc σ + h(x, y) C(σ) zˆ(x, y, σ) ≡ h(x, y) S(x, y, σ) = h(x, y)	 (264) hc + h(x, y)

and

dzˆ = dσ h(x, y) hc (265) hc + h(x, y)

As a consequence, the uppermost grid box retains very little dependency from bathymetry in

deep areas, where hc � h(x, y). For example, if hc = 250 m and h(x, y) changes from 2000 to

6000 m, the uppermost grid box changes only by a factor of 1.08 (less than 10%).

134
 
B.2 Stretching

In ROMS, we recommend Vstretching= 4. C(σ) is defined as a continuous, double stretching

function:

•	 Surface refinement function:
 

1 − cosh(θS σ) C(σ) =	 , for θS > 0, C(σ) = −σ2, for θS ≤ 0 (266) cosh(θS) − 1

•	 Bottom refinement function:
 

exp(θB C(σ)) − 1
 C(σ) = , for θB > 0	 (267) 1 − exp(−θB)

Notice that the bottom function (267) is the second stretching of an already stretched transform

(266). The resulting stretching function is continuous with respect to θS and θB as their values

approach zero. The range of meaningful values for θS and θB are:

0 ≤ θS ≤ 10 and 0 ≤ θB ≤ 4

However, users need to pay attention to extreme r-factor rx1 values near the bottom.

135
 
 

 

 

 

C Horizontal curvilinear coordinates
 

The requirement for a boundary-following coordinate system and for a laterally variable grid resolution can both be met (for suitably smooth domains) by introducing an appropriate orthogonal

coordinate transformation in the horizontal. Let the new coordinates be ξ(x, y) and η(x, y) where

the relationship of horizontal arc length to the differential distance is given by:

1 (ds)ξ = dξ (268) m

1 (ds)η = dη (269) n

Here, m(ξ, η) and n(ξ, η) are the scale factors which relate the differential distances (Δξ, Δη) to

the actual (physical) arc lengths.

It is helpful to write the equations in vector notation and to use the formulas for div, grad, and

curl in curvilinear coordinates [see Batchelor, 1967, Appendix 2]:

∂φ ∂φ 'φ = ˆξm + ˆηn (270) ∂ξ ∂η

∂ a ∂ b ' · -a = mn + (271) ∂ξ n ∂η m

' × -a = mn

 

 

 

 

 

 

 

 

ξ

ˆ

1

m

ξ

ˆ

2

n ˆ

k

∂ ∂ ∂

∂ξ ∂η ∂z

a

m

b

n c

 

 

 

 

 

 

 

 

(272)
 

∂ m ∂φ ∂ n ∂φ '2φ = ' · 'φ = mn + (273) ∂ξ n ∂ξ ∂η m ∂η

where φ is a scalar and -a is a vector with components a, b, and c.

136
 
 

D Viscosity and Diffusion

D.1 Horizontal viscosity

The horizontal viscosity and diffusion coefficients are scalars which are read in from ocean.in.

Several factors to consider when choosing these values are:

spindown time The spindown time on wavenumber k is 1 for the Laplacian operator and 1

k2ν2 k4ν4

for the biharmonic operator. The smallest wavenumber corresponds to the length 2Δx π and is k = , leading to Δx

Δ 2 4 x Δx Δt < tdamp = or (274)

π2ν2 π4ν4

This time should be short enough to damp out the numerical noise which is being generated but long enough on the larger scales to retain the features you are interested in.

This time should also be resolved by the model timestep.

boundary layer thickness The western boundary layer has a thickness proportional to

1 1

ν2 3 ν4 5

Δx < LBL = and (275) β β

for the Laplacian and biharmonic viscosity, respectively. We have found that the model

typically requires the boundary layer to be resolved with at least one grid cell. This leads

to coarse grids requiring large values of ν.

D.2 Horizontal Diffusion

We have chosen anything from zero to the value of the horizontal viscosity for the horizontal

diffusion coefficient. One common choice is an order of magnitude smaller than the viscosity.

D.3 Vertical Viscosity and Diffusion

ROMS stores the vertical mixing coefficients in arrays with three spatial dimensions called Akv

and Akt. Akt also has a fourth dimension specifying which tracer, so that temperature and salt

can have differing values. Both Akt and Akv are stored at w-points in the model; horizontal

averaging is done to obtain Akv at the horizontal u and v-points. The values for these coefficients

can be set in a number of ways, as described in §4.11.

137
 
 

E Radiant heat fluxes
 

As was seen in §5.2, the model thermodynamics requires fluxes of latent and sensible heat and

longwave and shortwave radiation. We follow the lead of Parkinson and Washington [1979] in

computing these terms.

E.1 Shortwave radiation

The Zillman equation for radiation under cloudless skies is:

S cos2 Z Qo = (276) (cosZ + 2.7)e × 10−5 + 1.085 cosZ + 0.10

where the variables are as in Table 10. The cosine of the zenith angle is computed using the formula:

cosZ = sin φ sin δ + cos φ cos δ cos HA. (277)

The declination is

δ = 23.44◦ × cos[(172 − day of year) × 2π/365] (278)

and the hour angle is

HA = (12 hours − solartime) × π/12. (279)

The correction for cloudiness is given by

SW↓= Qo(1 − 0.6c3). (280)

E.2 Longwave radiation

The clear sky formula for incoming longwave radiation is given by:

F ↓ = σT4 1 − 0.261 exp −7.77 × 10−4(273 − Ta)

2 (281) a

while the cloud correction is given by:

LW ↓ = (1 + 0.275c) F ↓ . (282)

E.3 Sensible heat

The sensible heat is given by the standard aerodynamic formula:

H ↓ = ρacpCHVwg(Ta − Tsfc). (283)

E.4 Latent heat

The latent heat depends on the vapor pressure and the saturation vapor pressure given by:

e = 611 × 10a(Td−273.16)/(Td−b) (284)

es = 611 × 10a(Tsfc−273.16)/(Tsfc−b) (285)

The vapor pressures are used to compute specific humidities according to:

= Ee

q10m (286) p − (1 − E)e

qs = Ees (287) p − (1 − E)es

The latent heat is also given by a standard aerodynamic formula:

LE ↓ = ρaLCEVwg(q10m − qs). (288)

Note that these need to be computed independently for the ice-covered and ice-free portions of each

gridbox since the empirical factors a and b and the factor L differ depending on the surface type.

138
 
Variable Value Description

(a, b) (9.5, 7.66) vapor pressure constants over ice

(a, b) (7.5, 35.86) vapor pressure constants over water

c cloud cover fraction

CE 1.75 × 10−3 transfer coefficient for latent heat

CH 1.75 × 10−3 transfer coefficient for sensible heat

cp 1004 J kg−1 K−1 specific heat of dry air

δ declination

e vapor pressure in pascals

es saturation vapor pressure

E 0.622 ratio of molecular weight of water to dry air

HA hour angle

L 2.5 × 106 J kg−1 latent heat of vaporization

L 2.834 × 106 J kg−1 latent heat of sublimation

φ latitude

Qo incoming radiation for cloudless skies

qs surface specific humidity

q10m 10 meter specific humidity

ρa air density

S 1353 W m−2 solar constant

σ 5.67 × 10−8 W m−2 K−4 Stefan-Boltzmann constant

Ta air temperature

Td dew point temperature

Tsfc surface temperature of the water/ice/snow

Vwg geostrophic wind speed

Z solar zenith angle

Table 10: Variables used in computing the incoming radiation and latent and sensible heat
 

139
 
F The C preprocessor

The C preprocessor, cpp, is a standalone program which comes with most C compilers. On older

UNIX systems it was not in the default path, but in /lib or in /usr/lib. Most modern systems have

some version of gcc which comes with a quite capable cpp. Just be sure to give it the -traditional

flag.

This Appendix describes the C preprocessor as used in ROMS with the Fortran language. A

more complete description is given by Kernighan and Ritchie [1988]. More practical advice on using

cpp is given by Hazard [1991].

F.1 File inclusion

Placing common blocks in smaller files, which are then included in each subroutine, is the easiest

way to make sure that the common blocks are declared consistently. Many Fortran compilers

support an include statement where the compiler replaces the line

include ’file.h’

with the contents of file.h; file.h is assumed to be in the current directory. The C preprocessor

has an equivalent include statement:

#include "file.h"

We are using the C preprocessor style of include because many of the ROMS include files are not

pure Fortran and must be processed by cpp.

F.2 Macro substitution

A macro definition has the form

#define name replacement text

where “name” would be replaced with “replacement text” throughout the rest of the file. This was

used in SCRUM as a reasonably portable way to get 64-bit precision:

#define BIGREAL real*8

It is customary to use uppercase for cpp macros—the C preprocessor is case sensitive.

It is also possible to define macros with arguments, as in

#define av2(a1,a2) (.5 * ((a1) + (a2)))

although this is riskier than the equivalent statement function

av2(a1,a2) = .5 * (a1 + a2)

The statement function is preferable because it allows the compiler to do type checking and because

you don’t have to be as careful about using enough parentheses.

The third form of macro has no replacement text at all:

#define MASKING

In this case, MASKING will evaluate to true in the conditional tests described below.

140
 
F.3 Conditional inclusion
 

It is possible to control which parts of the code are seen by the Fortran compiler by the use of

cpp’s conditional inclusion. For example, the statements

#ifdef MASKING
 

# include "mask.h"
 

#endif /* MASKING */
 

:
 

# ifdef MASKING
 

c
 

c Apply Land/Sea mask: slipperiness.
 

c
 

do j=1,M
 

do i=2,Lm
 

Uflux(i,j)=Uflux(i,j)*pmask(i,j)
 

enddo
 

enddo
 

# endif /* MASKING */
 

will not be in the Fortran source code if MASKING has not been defined. Likewise, #ifndef

tests for a macro being undefined:

#ifndef RMDOCINC
 

c rmask Mask at RHO-points (0=Land, 1=Sea).
 

c pmask Slipperiness mask at PSI-points (0=Land, 1=Sea,
 

c 1-gamma2=boundary).
 

c umask Mask at U-points (0=Land, 1=Sea).
 

c vmask Mask at V-points (0=Land, 1=Sea).
 

c
 

c=======================================================================
 

#endif
 

There are also #else and #elif (else if) statements. An example using #else and #elif is

shown:

#ifdef SOLITON

real(r8) :: g = 1.0_r8 ! non-dimensional

# elif defined WBC_1 || defined WBC_2 || defined WBC_3

real(r8) :: g = 9.8_r8 ! m/s2

# elif defined CIRCLE

real(r8) :: g = 3.92e-2_r8 ! m/s2

#else

real(r8) :: g = 9.81_r8 ! m/s2

#endif

Actually, #ifdef is a restricted version of the more general test

#if expression

where “expression” is a constant integer value. Zero evaluates to false and everything else is

considered true. Compound expressions may be built using the C logical operators:

&& logical and
 

|| logical or
 

! logical not
 

141
 
These symbols would be used as in the following example:

#if defined CANYON_A || defined CANYON_B
 

do j=0,M
 

do i=0,L

yc=c32000-c16000*(sin(pi*xr(i,j)/xl))**24

h(i,j)=c20+p5*(hmax-c20)*(c1+tanh((yr(i,j)-yc)/c10000))

enddo
 

enddo
 

#endif
 

F.4 C comments

The C preprocessor will also delete C language comments starting with /* and ending with */ as

in:

#endif /* MASKING */

When mixed with Fortran code, it is safer to use a Fortran comment.

F.5 A note on style

In ROMS, the style adopted is as shown above and here:

#define NEP5

#if defined NEP5 || defined BERING

:

# ifdef MASKING

:

# endif

#endif

The gnu community has instead adopted this style:

#define NEP5 1

#if NEP5 || BERING

:

# if MASKING

:

# endif

#endif

Both styles work and you should be aware that there’s more than one way to do it.

F.6 Potential problems

The use of the C preprocessor is not entirely free of problems, but many can be worked around or

avoided by using the gnu version of cpp with the -traditional flag.

1. Apostrophes in Fortran comments.	 cpp does not know that it is in a comment and some

versions will complain about unmatched apostrophes in the following:

!	 Some useful comment about Green’s functions.

142
 
2. C++ comments. Some of the newer versions of cpp will remove C++ comments which go from

’//’ to the end of the line. Some perfectly reasonable Fortran lines contain two consecutive

slashes, such as:

common // var1, var2
 

44 format(//)
 

and Fortran 90 string concatenation:

mystring = ’Hello, ’ // ’World!’

3. Macro replacement. One feature of cpp is that you can define macros and have it perform

replacements. The code:

#define REAL double precision
 

REAL really_long_variable, second_long_variable
 

becomes

double precision really_long_variable, second_long_variable

and you run the risk of creating lines which are longer than 72 characters in length.

Also, make sure that your macros will not be found anywhere else in the code. I used to

use #define DOUBLE for double precision until it was pointed out to me that DOUBLE

PRECISION is perfectly valid Fortran. Since a string that is simply “defined” becomes 1,

the macro processor would turn this into 1 PRECISION.

F.7 Modern Fortran

I started working on these ocean models before 1990, much less before Fortran 90 compilers were

generally available. Fortran 90’s kind feature as used in ROMS is a better way to handle the

BIGREAL type declarations. On the other hand, Fortran 90 does not include conditional compilation. However, it is deemed useful enough that the Fortran 2003 standard has the coco means of

conditional compilation. We might start using this in about ten years.

143
 
G Makefiles
 

One of the software development tools which comes with the UNIX operating system is called

make. Make has many uses, but is most commonly used to keep track of how a large program

should be compiled. ROMS now requires the gnu version of make, sometimes known as gmake

[Mechlenburg, 2005]. This appendix describes generic make, the gnu enhancements to make, as

well as describing the makefile structure used in ROMS. This material first appeared in the ARSC

HPC Newsletter in several segments and has been updated and restructured here.

G.1 Introduction to Portable make

Make gets its instructions from a description file, by default named makefile or Makefile. This

file is called the Makefile, but other files can be used by invoking make with the -f option:

make -f Makefile.yukon

The ancester to ROMS originally had a Makefile that looked something like:

model: main.o init.o plot.o
 

<TAB> f77 -o model main.o init.o plot.o
 

main.o: main.f
 

<TAB> f77 -c -O main.f
 

init.o: init.f
 

<TAB> f77 -c -O init.f
 

plot.o: plot.f
 

<TAB> f77 -c -O0 plot.f
 

clean:
 

<TAB> rm *.o core
 

The default thing to build is model, the first target. The syntax is:

target: dependencies

<TAB> command

<TAB> command

The target model depends on the object files, main.o and friends. They have to exist and be

up to date before model’s link command can be run. If the target is out of date according to

the timestamp on the file, then the commands will be run. Note that the tab is required on the

command lines.

The other targets tell make how to create the object files and that they in turn have dependencies. To compile model, simply type “make”. Make will look for the file makefile, read it, and

do whatever is necessary to make model up to date. If you edit init.f, that file will be newer than

init.o. Make would see that init.o is out of date and run the “f77 -c -O init.f” command. Now

init.o is newer than model, so the link command “f77 -o model main.o init.o plot.o” must be

executed.

To clean up, type “make clean” and the clean target will be brought up to date. The clean

target has no dependencies. What happens in that case is that the command (“rm *.o core”) will

always be executed.

The original version of this Makefile turned off optimization on plot.f due to a compiler bug,

but hopefully you won’t ever have to worry about that.

144
 
G.1.1 Macros
 

Make supports a simple string substitution macro. Set it with:

MY_MACRO = nothing today

and refer to it with:

$(MY_MACRO)

The convention is to put the macros near the top of your Makefile and to use upper case. Also,

use separate macros for the name of your compiler and the flags it needs:

F90 = f90

F90FLAGS = -O3

LIBDIR = /usr/local/lib

LIBS = -L$(LIBDIR) -lmylib

Let’s rewrite our Makefile using macros:

#

# IBM version

#

F90 = xlf90_r

F90FLAGS = -O3 -qstrict

LDFLAGS = -bmaxdata:0x40000000

model: main.o init.o plot.o

$(F90) $(LDFLAGS) -o model main.o init.o plot.o

main.o: main.f

$(F90) -c $(F90FLAGS) main.f

init.o: init.f

$(F90) -c $(F90FLAGS) init.f

plot.o: plot.f

$(F90) -c $(F90FLAGS) plot.f

clean:

rm *.o core

Now when we change computers, we only have to change the compiler name in one place.

Likewise, if we want to try a different optimization level, we only specify that in one place.

Notice that you can use comments by starting the line with a #.

G.1.2 Implicit Rules

Make has some rules already built in. For fortran, you might be able to get away with:

OBJS = main.o init.o plot.o

model: $(OBJS)

$(FC) $(LDFLAGS) -o model $(OBJS)

145
 
as your whole Makefile. Make will automatically invoke its default Fortran compiler, possibly

f77 or g77, with whatever default compile options it happens to have (FFLAGS). One built-in

rule often looks like:

.c.o:

$(CC) $(CFLAGS) -c $<

which says to compile .c files to .o files using the compiler CC and options CFLAGS. We can

write our own suffix rules in this same style. The only thing to watch for is that make by default

has a limited set of file extentions that it knows about. Let’s write our Makefile using a suffix

rule:

#

# Cray version

#

F90 = f90

F90FLAGS = -O3

LDFLAGS =

.f.o:

$(F90) $(F90FLAGS) -c $<

model: main.o init.o plot.o

$(F90) $(LDFLAGS) -o model main.o init.o plot.o

clean:

rm *.o core

G.1.3 Dependencies

There may be additional dependencies beyond the source->object ones. In our little example,

all our source files include a file called commons.h. If commons.h gets modified to add a new

variable, everything must be recompiled. Make won’t know that unless you tell it:

# include dependencies

main.o: commons.h

init.o: commons.h

plot.o: commons.h

Fortran 90 introduces module dependencies as well. See §H for how we automatically generate this

dependency information.

The most common newbie mistake is to forget that the commands after a target have to start

with a tab.

G.2 gnu make

Over the years, the community has moved from the stance of writing portable Makefiles to a

stance of just using a powerful, portable make. The previous section described a portable subset

of make features. We now delve into some of the more powerful tools available in gnu make.

146
 
G.2.1 Make rules

The core of make hasn’t changed in decades, but concentrating on gmake allows one to make use

of its nifty little extras designed by real programmers to help with real projects. The first change

that matters to my Makefiles is the change from suffix rules to pattern rules. I’ve always found

the .SUFFIXES list to be odd, especially since .f90 is not in the default list. Good riddance to

all of that! For a concrete example, the old way to provide a rule for going from file.f90 to file.o

is:

.SUFFIXES: .o .f90 .F .F90
 

.f90.o:
 

<TAB> $(FC) -c $(FFLAGS) $<
 

while the new way is:

%.o: %.f90
 

<TAB> $(FC) -c $(FFLAGS) $<
 

In fact, going to a uniform make means that we can figure out what symbols are defined and use

their standard values—in this case, $(FC) and $(FFLAGS) are the built-in default names for the

compiler and its options. If you have any questions about this, you can always run make with the

-p (or --print-data-base) option. This prints out all the rules make knows about, such as:

# default

COMPILE.f = $(FC) $(FFLAGS) $(TARGET_ARCH) -c

Printing the rules database will show variables that make is picking up from the environment,

from the Makefile, and from its built-in rules—and which of these sources is providing each value.

G.2.2 Assignments

In the old days, I only used one kind of assignment: = (equals sign). Gmake has several kinds of

assignment (other makes might as well, but I no longer have to know or care). An example of the

power of gnu make is shown by an example from my Cray X1 Makefile. There is a routine which

runs much more quickly if a short function in another file is inlined. The way to accomplish this is

through the -O inlinefrom=file directive to the compiler. I can’t add this option to FFLAGS,

since the inlined routine won’t compile with this directive—there is only the one file that needs it.

I had:

FFLAGS = -O 3,aggress -e I -e m

FFLAGS2 = -O 3,aggress -O inlinefrom=lmd_wscale.f90 -e I -e m

lmd_skpp.o:
 

<TAB> $(FC) -c $(FFLAGS2) $*.f90
 

The first change I can make to this using other assignments is:

FFLAGS := -O 3,aggress -e I -e m

FFLAGS2 := $(FFLAGS) -O inlinefrom=lmd_wscale.f90

The := assignment means to evaluate the right hand side immediately. In this case, there is no

reason not to, as long as the second assigment follows the first one (since it’s using the value of

$(FFLAGS)). For the plain equals, make doesn’t evaluate the right-hand side until its second

pass through the Makefile. However, gnu make supports an assignment which avoids the need

for FFLAGS2 entirely:

147
 
lmd_skpp.o: FFLAGS += -O inlinefrom=lmd_wscale.f90

What this means is that for the target lmd_skpp.o only, append the inlining directive to FFLAGS.

I think this is pretty cool!

One last kind of assignment is to set the value only if there is no value from somewhere else

(the environment, for instance):

FC ?= mpxlf90_r

If we used := or =, we would override the value from the environment.

G.2.3 Include and a Few Functions

One reasonably portable make feature is the include directive. It can be used to clean up the

Makefile by putting bits in an include file. The syntax is simply:

include file

and we use it liberally to keep the project information neat. We can also include a file with the

system/compiler information in it, assuming we have some way of deciding which file to include.

We can use uname -s to find out which operating system we’re using. We also need to know which

compiler we’re using.

One user-defined variable is called FORT, the name of the Fortran compiler. This value is

combined with the result of “uname -s” to provide a machine and compiler combination. For

instance, ftn on Linux is the Cray cross-compiler. This would link to a different copy of the

NetCDF library and use different compiler flags than the Intel compiler which might be on the

same system.

# The user sets FORT:

FORT ?= ftn

OS := $(shell uname -s | sed ’s/[\/ ]/-/g’)

include $(COMPILERS)/$(OS)-$(strip $(FORT)).mk

We pick one include file at compile time, here picking Linux-ftn.mk, containing the Cray crosscompiler information. The value Linux comes from the uname command, the ftn comes from the

user, and the two are concatenated. The sed command will turn the slash in UNICOS/mp into

a dash; the native Cray include file is UNICOS-mp-ftn.mk. Strip is a built-in function to strip

away any extra white space.

Another tricky system is CYGWIN, which puts a version number in the uname output, such

as CYGWIN_NT-5.1. Gnu make has quite a few built-in functions, one of which allows us to

do string substitution:

OS := $(patsubst CYGWIN_%,CYGWIN,$(OS))

In make, the % symbol is a sort of wild card, much like * in the shell. Here, we match CYGWIN

followed by an underscore and anything else, replacing the whole with simply CYGWIN. Another

example of a built-in function is the substitution we do in:

objects = $(subst .F,.o,$(sources))

where we build our list of objects from the list of sources. There are quite a few other functions,

plus the user can define their own. From Mechlenburg [2005]:

148
 
GNU make supports both built-in and user-defined functions. A function invocation

looks much like a variable reference, but includes one or more parameters separated

by commas. Most built-in functions expand to some value that is then assigned to a

variable or passed to a subshell. A user-defined function is stored in a variable or macro

and expects one or more parameters to be passed by the caller.

We will show some user-defined functions in §G.3.3.

G.2.4 Conditionals

We used to have way too many Makefiles, a separate one for each of the serial/MPI/OpenMP

versions on each system (if supported). For instance, the name of the IBM compiler changes when

using MPI; the options change for OpenMP. The compiler options also change when using 64-bit

addressing or for debugging. A better way to organize this is to have the user select 64-bit or not,

MPI or not, etc., then use conditionals. The complete list of user definable make variables is given

in §2.4.1.

Gnu make supports two kinds of if test, ifdef and ifeq (plus the negative versions ifndef, ifneq).

The example from the book is:

ifdef COMSPEC

# We are running Windows

else

# We are not on Windows

endif

An example from the IBM include file is:

ifdef USE_DEBUG

FFLAGS += -g -qfullpath

else

FFLAGS += -O3 -qstrict

endif

To test for equality, an example is:

ifeq ($(USE_MPI),on)

# Do MPI things

endif

or

ifeq "$(USE_MPI)" "on"

# Do MPI things

endif

The user has to set values for the USE_MPI, USE_DEBUG, and USE_LARGE switches in

the Makefile or the build script before the compiler-dependent piece is included:

USE_MPI ?= on
 

USE_DEBUG ?=
 

USE_LARGE ?=
 

The Makefile uses the conditional assign “?=” in case a build script is used to set them in the

environment. Be sure to leave the switches meant to be off set to an empty string—the string “off”

will test true on an ifdef test.

149
 
G.3 Multiple Source Directories the ROMS Way

There’s more than one way to divide your sources into separate directories. The choices we have

made include nonrecursive make and putting the temporary files in their own $(SCRATCH_DIR)

directory. These include the .f90 files which have been through the C preprocessor, object files,

module files, and libraries.

G.3.1 Directory Structure

The directory structure of the source code has the top directory, a Master directory, a ROMS

directory with a number of subdirectories, and several other directories. Master contains the main

program while the rest contain sources for libraries and other files. Note that the bulk of the source

code gets compiled into files that become libraries with the ar command, one library per directory.

There is also a Compilers directory for the system- and compiler-specific Makefile components.

G.3.2 Conditionally Including Components

The makefile will build the lists of libraries to create and source files to compile. They start out

empty at the top of the makefile:

sources :=

libraries :=

That’s simple enough, but the list of directories to search for these sources will depend on the options

chosen by the user, not just in the make options (§2.4.1), but inside the ROMS_HEADER file

as well. How does this happen? Once make knows how to find the ROMS_HEADER, it is

used by cpp to generate an include file telling make about these other options.

MAKE_MACROS := Compilers/make_macros.mk

MACROS := $(shell cpp -P $(ROMS_CPPFLAGS) Compilers/make_macros.h > \

$(MAKE_MACROS); $(CLEAN) $(MAKE_MACROS))

The make_macros.h file contains blocks such as:

#ifdef SWAN_COUPLING

USE_SWAN := on

#else

USE_SWAN :=

#endif

The resulting make_macros.mk file will simply end up with either

USE_SWAN := on

or

USE_SWAN :=

This file can then be included by the makefile and the variable USE_SWAN will have the correct

state for this particular compilation. We can now use it and all the similar flags to build a list of

directories.

We initialize two lists:

modules :=
 

includes := ROMS/Include
 

150
 
Add the adjoint bits:

ifdef USE_ADJOINT

modules += ROMS/Adjoint

endif

ifdef USE_ADJOINT

includes += ROMS/Adjoint

endif

Add the bits we’ll always need:

modules +=	 ROMS/Nonlinear \
 

ROMS/Functionals \
 

ROMS/Utility \
 

ROMS/Modules
 

includes +=	 ROMS/Nonlinear \
 

ROMS/Utility \
 

ROMS/Drivers
 

Then we add in some more:

ifdef USE_SWAN

modules += Waves/SWAN/Src

includes += Waves/SWAN/Src

endif

modules += Master
 

includes += Master Compilers
 

Now that our lists are complete, let’s put them to use:

vpath %.F $(modules)

vpath %.h $(includes)

vpath %.f90 $(SCRATCH_DIR)

vpath %.o $(SCRATCH_DIR)

include $(addsuffix /Module.mk,$(modules))

CPPFLAGS += $(patsubst %,-I%,$(includes))

1.	 vpath is a standard make feature for providing a list of directories for make to search for

files of different types. Here we are saying that *.F files can be found in the directories

provided in the $(modules) list, and so on for the others.

2. For each directory in the $(modules) list, make will include the file Module.mk that is

found there. More on these later.

3. For each directory in the $(includes) list, add that directory to the list searched by cpp

with the −I flag.

G.3.3 User-defined make Functions

The Module.mk fragments mentioned before call some user-defined functions. It’s time to show

these functions and talk about how they work. They get defined in the top Makefile:

151
 
#-------------------------------------------------------------------------­

# Make functions for putting the temporary files in $(SCRATCH_DIR)

#-------------------------------------------------------------------------­

# $(call source-dir-to-binary-dir, directory-list)

source-dir-to-binary-dir = $(addprefix $(SCRATCH_DIR)/, $(notdir $1))

# $(call source-to-object, source-file-list)

source-to-object = $(call source-dir-to-binary-dir, \

$(subst .F,.o,$(filter %.F,$1)))

# $(call f90-source, source-file-list)

f90-source = $(call source-dir-to-binary-dir, \

$(subst .F,.f90,$1))

# $(call make-library, library-name, source-file-list)

define make-library

libraries += $(SCRATCH_DIR)/$1

sources += $2

$(SCRATCH_DIR)/$1: $(call source-dir-to-binary-dir, \

$(subst .F,.o,$2))
 

$(AR) $(ARFLAGS) $$@ $$^
 

$(RANLIB) $$@
 

endef

# $(call one-compile-rule, binary-file, f90-file, source-files)

define one-compile-rule

$1: $2 $3

cd $$(SCRATCH_DIR); $$(FC) -c $$(FFLAGS) $(notdir $2)

$2: $3
 

$$(CPP) $$(CPPFLAGS) $$(MY_CPP_FLAGS) $$< > $$@
 

$$(CLEAN) $$@
 

endef

# $(compile-rules)

define compile-rules

$(foreach f, $(local_src), \
 

$(call one-compile-rule,$(call source-to-object,$f), \
 

$(call f90-source,$f),$f))
 

endef

1. We define a function to convert the path from the source directory to the Build directory,

called source-dir-to-binary-dir. Note that the Build directory is called $(SCRATCH_DIR)

here. All it does is strip off the leading directory with the the built-in function notdir, then

paste on the Build directory.

2. Next comes source-to-object, which calls the function above to return the object filename

when given the source filename. It assumes that all sources have a .F extension.

152
 
3. A similar function is f90-source, which returns the name of the intermediate source which

is created by cpp from our .F file.

4. The Module.mk fragment in each library source directory invokes make-library, which

takes the library name and the list of sources as its arguments. The function adds its library

to the global list of libraries and provides rules for building itself. The double dollar signs

are to delay the variable substitution. Note that we call source-dir-to-binary-dir instead

of source-to-object—this is a work-around for a make bug.

5. The next, one-compile-rule, takes three arguments: the .o filename, the .f90 filename, and

the .F filename. From these, it produces the make rules for running cpp and the compiler.

A note on directories: make uses vpath to find the source file where it resides. It would be

possible to compile from the top directory and put the .o file in Build with the appropriate

arguments, but I don’t know how to get the .mod file into Build short of a mv command.

Likewise, if we compile in the top directory, we need to know the compiler option to tell it

to look in Build for the .mod files it uses. Doing a cd to Build before compiling is just

simpler.

6. The last, compile-rules, is given a list of sources, then calls one-compile-rule once per

source file.

Again, you can invoke make -p to see how make internally transforms all this into actual

targets and rules.

G.3.4 Library Module.mk

In each library directory, there is a file called Module.mk which gets included by the top level

makefile. These Module.mk bits build onto the list of sources and libraries to be compiled and

built, respectively. These Module.mk files look something like:

local_sub := ROMS/Nonlinear

local_lib := libNLM.a

local_src := $(wildcard $(local_sub)/*.F)

$(eval $(call make-library,$(local_lib),$(local_src)))

$(eval $(compile-rules))

First, we provide the name of the current directory and the library to be built from the resident

sources. Next, we use the wildcard function to search the subdirectory for these sources. Note

that every .F file found will be compiled. If you have half-baked files that you don’t want used,

make sure they have a different extension.

Each subdirectory is resetting the local_src variable. That’s OK because we’re saving the

values in the global sources variable inside the make-library function, which also adds the local

library to the libraries list. The compile-rules function uses this local_src variable to generate

the rules for compiling each file, placing the resulting files in the Build directory.

G.3.5 Main Program

The main program is in a directory called Master and its Module.mk is similar to the library

one:

153
 
local_sub := Master
 

local_src := $(wildcard $(local_sub)/*.F)
 

local_objs := $(subst .F,.o,$(local_src))
 

local_objs := $(addprefix $(SCRATCH_DIR)/, $(notdir $(local_objs)))
 

sources += $(local_src)
 

ifdef LD_WINDOWS
 

$(BIN): $(libraries) $(local_objs)
 

$(LD) $(FFLAGS) $(local_objs) -o $@ $(libraries) $(LIBS_WIN32) $(LDFLAGS)

else

$(BIN): $(libraries) $(local_objs)

$(LD) $(FFLAGS) $(LDFLAGS) $(local_objs) -o $@ $(libraries) $(LIBS)

endif

$(eval $(compile-rules))

Instead of a rule for building a library, we have a rule for building a binary $(BIN). In this case,

the name of the ROMS binary is defined elsewhere. The binary depends on the libraries getting

compiled first, as well as the local sources. During the link, the $(libraries) are compiled from the

sources in the other directories, while $(LIBS) are exteral libraries such as NetCDF.

G.3.6 Top Level Makefile

Now we get to the glue that holds it all together. We’ve covered many things so far, but there’s

still a few bits which might be confusing:

1. There can be rare cases where you might have special code for some systems. You can check

which system you are on in the .F file with:

#ifdef X86_64
 

! special stuff
 

#endif
 

To be sure this is defined on each X86_64 system, it has to be passed to cpp:

CPPFLAGS += -D$(shell echo ${OS} | tr "-" "_" | tr [a-z] [A-Z])
 

CPPFLAGS += -D$(shell echo ${CPU} | tr "-" "_" | tr [a-z] [A-Z])
 

CPPFLAGS += -D$(shell echo ${FORT} | tr "-" "_" | tr [a-z] [A-Z])
 

CPPFLAGS += -D’ROOT_DIR="$(ROOTDIR)"’

ifdef ROMS_APPLICATION
 

CPPFLAGS += $(ROMS_CPPFLAGS)
 

CPPFLAGS += -DNestedGrids=$(NestedGrids)
 

MDEPFLAGS += -DROMS_HEADER="$(HEADER)"
 

endif

This guarantees that CPPFLAGS will have terms in it such as:

-DLINUX -DX86_64 -DPGI

-D’ROOT_DIR="/export/staffdata/kate/roms/trunk"’ -DSHOREFACE

-D’HEADER="shoreface.h"’ -D’ROMS_HEADER="shoreface.h"’

-DNestedGrids=1

154
 
2. For mod_strings.F, there is a special case:

$(SCRATCH_DIR)/mod_strings.f90: CPPFLAGS += -DMY_OS=’"$(OS)"’ \
 

-DMY_CPU=’"$(CPU)"’ -DMY_FORT=’"$(FORT)"’ \
 

-DMY_FC=’"$(FC)"’ -DMY_FFLAGS=’"$(FFLAGS)"’
 

allowing ROMS to report in its output:

Operating system : Linux
 

CPU/hardware : x86_64
 

Compiler system : pgi
 

Compiler command : pgf90
 

Compiler flags : -O3 -tp k8-64 -Mfree
 

Local Root : /export/staffdata/kate/roms/trunk
 

Header Dir : ./ROMS/Include
 

Header file : shoreface.h
 

Though this doesn’t seem to work on the Mac.

3. The very first makefile I showed had a set list of files to remove on make clean. We now

build a list, called clean_list:

clean_list := core *.ipo $(SCRATCH_DIR)

ifeq "$(strip $(SCRATCH_DIR))" "."
 

clean_list := core *.o *.oo *.mod *.f90 lib*.a *.bak
 

clean_list += $(CURDIR)/*.ipo
 

endif

In other words, we want to clean up the Build directory unless it happens to be the top level

directory, in which case we only want to remove specific files there.

4.	 “all” is the first target that gets seen by make, making it the default target. In this case,

we know there is only the one binary, whose name we know—Mechlenburg [2005] shows what

to do with more than one binary. Both “all” and “clean” are phony targets in that no files

of those names get generated—make has the .PHONY designation for such targets. Also,

the clean target doesn’t require any compiler information, so the compiler include doesn’t

happen if the target is “clean”:

ifneq "$(MAKECMDGOALS)" "clean"
 

include $(COMPILERS)/$(OS)-$(strip $(FORT)).mk
 

endif
 

$(MAKECMDGOALS) is a special variable containing the current make target.

5. We’ll be creating different executable names, depending on which options we’ve picked:

BIN := $(BINDIR)/oceanS
 

ifdef USE_DEBUG
 

BIN := $(BINDIR)/oceanG
 

else
 

155
 
ifdef USE_MPI

BIN := $(BINDIR)/oceanM
 

endif
 

ifdef USE_OpenMP
 

BIN := $(BINDIR)/oceanO
 

endif
 

endif
 

6. The NetCDF library gets included during the final link stage. However, we are now using

the Fortran 90 version of it which requires its module information as well. We just copy the

.mod files into the Build directory:

NETCDF_MODFILE := netcdf.mod
 

TYPESIZES_MODFILE := typesizes.mod
 

$(SCRATCH_DIR)/$(NETCDF_MODFILE): | $(SCRATCH_DIR)
 

cp -f $(NETCDF_INCDIR)/$(NETCDF_MODFILE) $(SCRATCH_DIR)
 

$(SCRATCH_DIR)/$(TYPESIZES_MODFILE): | $(SCRATCH_DIR)
 

cp -f $(NETCDF_INCDIR)/$(TYPESIZES_MODFILE) $(SCRATCH_DIR)
 

Old versions of NetCDF do not have the typesizes.mod file, in which case it has to be

removed from the following dependency list:

$(SCRATCH_DIR)/MakeDepend: makefile \
 

$(SCRATCH_DIR)/$(NETCDF_MODFILE) \
 

$(SCRATCH_DIR)/$(TYPESIZES_MODFILE) \
 

| $(SCRATCH_DIR)
 

7. Then there is MakeDepend itself. This file is created by the Perl script sfmakedepend.

MakeDepend gets created by “make depend” and included on make’s second pass through

the makefile:

depend: $(SCRATCH_DIR)

$(SFMAKEDEPEND) $(MDEPFLAGS) $(sources) > $(SCRATCH_DIR)/MakeDepend

ifneq "$(MAKECMDGOALS)" "clean"
 

-include $(SCRATCH_DIR)/MakeDepend
 

endif

The dash before the include tells make to ignore errors so that make depend will succeed

before the file exists. The MakeDepend file will contain the include and module dependencies for each source file, such as:

Build/mod_diags.o: tile.h cppdefs.h globaldefs.h shoreface.h

Build/mod_diags.f90: tile.h cppdefs.h globaldefs.h shoreface.h

Build/mod_diags.o: Build/mod_kinds.o Build/mod_param.o Build/mod_diags.f90

Note that the .h files are included by cpp, so that both the .f90 and .o files become out of

date when an include file is modified. Without the module dependencies, make would try to

build the sources in the wrong order and the compiler would fail with a complaint about not

finding mod_param.mod, for instance.

156
 
G.4 Final warnings

The cost of this nifty make stuff is:

1. We’re a little closer to the gnu make bugs here, and we need a newer version of gnu make

than before (version 3.81, 3.80 if you’re lucky). Hence this stuff at the top of the makefile:

NEED_VERSION := 3.80 3.81 3.82

$(if $(filter $(MAKE_VERSION),$(NEED_VERSION)),, \

$(error This makefile requires one of GNU make version $(NEED_VERSION).))

2. The Makefile dependencies get just a little trickier every change we make. Note that F90

has potentially both include and module use dependencies. The book example uses the C

compiler to produce its own dependencies for each source file into a corresponding .d file to

be included by make. Our Fortran compilers are not so smart. For these hairy compiles, it’s

critical to have accurate dependency information unless we’re willing to make clean between

compiles.

157
 
H sfmakedepend

One Perl script I use with Fortran creates the dependency information, much like the X11 program

makedepend. I originally wrote fmakedepend which was used with traditional Fortran include

statements. I later wrote a variant of it for use with the C preprocessor, called sfmakedepend.

The latest version of sfmakedepend does the job of both programs and also searches for the

dependencies introduced by Fortran 90 modules. It is used by the Makefiles described in §G.

It recursively searches for Fortran style includes, for instance if file.f has the statement:

include ’commons.h’

the line

file.o: commons.h

will be added to the bottom of the Makefile. This tells make that file.o depends on commons.h

as well as file.f, and to recompile file.f whenever commons.h is modified. It likewise searches

source files for C style includes such as

#include "commons.h"

and adds the corresponding dependencies to the Makefile. It has several options, including -s,

required for Fortran compilers which will not invoke the C preprocessor for you. In this case the

above dependency line would become

file.o: commons.h

file.f: commons.h

letting make know that the C preprocessor must be rerun on file.F whenever commons.h is

updated.

When using the C preprocessor, you can ask it to search directories other than the current

directory. Likewise, sfmakedepend can be instructed to search other directories with -I dir

options. Note that it is legal to have more than one -I dir option as in:

sfmakedepend -I /usr/local/include -I /home/me/include *.F

Fortran 90 introduces some interesting dependencies. Two compilers I have access to (NAG f90

and IBM xlf) produce a private my_module.mod file if you define module My_Module in

file mod.f90. This file is used by the compiler when you use the module as a consistency check

(type-safe programming). If foo.f90 uses that module, you will need the following dependency

information:

foo.o: my_module.mod

my_module.mod: mod.o

This says that before compiling foo.f90 we need to have the file my_module.mod. This file in

turn depends on mod.o, so that mod.f90 must be compiled before foo.f90. The sgi is similar

except that it uses the file MY_MODULE.kmo to store the private module information. Use

sfmakedepend -g on the SGI.

Rather than creating extra module files, the Cray and Parasoft compilers store the module

information in the object file and then files which use the modules need to be compiled with extra

flags pointing to the module object files. For instance, if foo.f90 uses My_Module which was

defined in mod.f90, then you will need to compile mod.f90 first and provide the Cray compiler with

the extra option -p mod.o when compiling foo.f90. When using the Cray, use sfmakedepend -c

to get the dependency information:

158
 
foo.o: mod.o
 

$(FC) $(FFLAGS) -c -p mod.o foo.f90
 

$(FC) and $(FFLAGS) are assumed to be previously defined as the name of the compiler and

the compiler options, respectively.

Note: Gnu make is robust enough to handle these chained dependencies. Some hasn’t always

been true of other old (very old now) versions of make.

sfmakedepend assumes that all the files using and defining modules are in the same directory

and are all in the list of files to be searched. It seems that the industry has not settled on a practical

way to deal with a separate modules directory, anyway.

I sometimes include non-existent files as a compile time consistency check:

#ifndef PLOTS
 

#include "must_define_PLOTS" /* bogus include */
 

#endif
 

This program warns about include files it can’t find, but not if there is a “bogus” on the same line.

See the comments at the top of sfmakedepend for up-to-date information on the options.

Compiler vendors have managed to more nearly standardize the Fortran module business and just

about all work with these options to sfmakedepend:

--cpp --fext=f90 --file=- --objdir=$(SCRATCH_DIR)

159
 
I Subversion

The ROMS source code is distributed using Subversion (svn). There are svn clients available for

nearly every operating system and many resources available, including an O’Reilly book [CollinsSussman et al., 2008]. I’ll just cover a few basics here, taken in part from the ROMS wiki.

If you wish to maintain your own copy of ROMS in a source code repository, you may want to

investigate git, which has the ability to download from an svn server. There is an excellent book

about git which is online and free—really, do yourself a favor and read it [Chacon and Straub,

2014]. I have rambled at length about git on the ROMS wiki because I truly believe that svn is

not the best tool for most ROMS users (those who can’t check back in). My ramblings about the

shortcomings of svn have driven one ROMS user to adopt Mercurial instead, another valid option.

I.1 Overview

Subversion is a tool for managing software development that keeps track of who modified what and

allows the return to a previous version if changes don’t work as expected. All the ROMS/TOMS

files are stored in a svn repository on www.myroms.org with access controlled by requiring authentication with the same ROMS Username/Password combination assigned to registered users of the

ROMS Forum.

This svn repository is the official version of the code which only the developers are allowed to

change. Users should download the ROMS code to their local machines using an svn client. Don’t

attempt to use a regular web browser to browse or download files from the svn repository - there

are much better tools for interacting with the code repository.

We strongly recommend that users always check out the current trunk version since this has

the most recent updates and bug fixes. The tags version is kept largely as an historical record of

stable releases at the conclusion of major code upgrades.

Below is a general description of how subversion works. Please look at the svn book [CollinsSussman et al., 2008] for more detailed information and the wiki for more on some available GUI

clients.

I.2 Checking out the code

WARNING: It is strongly suggested that you checkout the ROMS source code using the same

operating system you wish to compile and run ROMS on. If you download the code on a Windows

machine and wish to run it on a non-Windows machine you will need convert the line endings with

a utility like dos2unix or recode. Even with these utilities you may still have problems compiling

ROMS.

In order download source code from a Subversion repository, the svn client software must be

installed on your local machine. If you are compiling subversion on your own be sure to build

it with SSL support or you will not be able to download the ROMS source code. Most Linux

distributions come with subversion (the command name is svn), so shell commands may be used

without installing additional software. If your username on your local computer is not the same as

your ROMS username you will need to pass the –username <username> option to svn; an example

is given below. The general form of subversion commands is:

svn action from to {optional_qualifiers}

To check-out the files from the ROMS repository trunk:

svn checkout https://www.myroms.org/svn/src/trunk MyDir

where MyDir is the destination directory on your local computer. Note the https rather than http.

If your username on your local computer is not the same as your ROMS username you will need

to pass the –username option to svn:

160
 
svn checkout --username joe_roms https://www.myroms.org/svn/src/trunk MyDir

You only check out once, after that, a hidden directory called .svn exists to keep track of the source,

destination and a bunch of other information. Your username and password will also be saved.

I.3 Updates

Now and again, you might feel the urge to get up to speed with the latest changes that have been

made to the ROMS repository. When that happens, simply go to the directory that was “MyDir”

above and type:

svn update

Subversion will remember where you checked out from before and see if a newer revision exists. If

so, it will download and apply all the relevant changes.

I.4 Code changes

As you use ROMS, you may find yourself adding new files and new chunks of code to existing files.

Unless you are a developer with write access to the repository at www.myroms.org, you have no

easy way to save your changes within the svn framework, since any one directory can only point

to one repository. To keep getting updates from the trunk, you must keep using the svn server at

myroms.org. At the very least, saving a tarball before fetching major updates is a prudent step.

I.5 Conflicts

Sometimes when you make changes to your copy of the ROMS code, those changes will conflict

with changes made to the repository. This means that the changes from the server overlapped with

your own, and now you have to manually choose between them.

Whenever a conflict occurs, three things typically occur to assist you in resolving that conflict:

•	 Subversion halts during the update, offering you several choices, and remembers that the file

is in a state of conflict if you don’t clear it right then.

•	 If Subversion considers the file to be mergeable, it places conflict markers (special strings of

text which delimit the “sides” of the conflict, usually “<” and “>” characters) into the file

to visibly demonstrate the overlapping areas.

•	 For every conflicted file, Subversion places three extra unversioned (not under Subversion

control) files in your working copy:

filename.mine : This is your file as it existed in your working copy (local copy) before

you updated your working copy. This file has only your latest changes in it. (If

Subversion considers the file to be unmergeable, then the .mine file isn’t created,

since it would be identical to the working file.)

filename.rOLDREV : This is the file that was the BASE revision before you updated your

working copy. That is, the file that you checked out before you made your latest

edits.

filename.rNEWREV : This is the file that your Subversion client just received from the

server when you updated your working copy. This file corresponds to the HEAD

(latest) revision of the repository.

161
 
For example, let’s say you checked out revision 280 and made some changes to a file, for instance

User/Functionals/ana_hmixcoef.h. Now you want to update your ROMS source code to take

advantage of a new algorithm but when you run svn update your ana_hmixcoef.h is now in

conflict with the new version in the repository:

> svn update
 

U Version
 

U ROMS/Modules/mod_mixing.F
 

U ROMS/Functionals/ana_hmixcoef.h
 

C User/Functionals/ana_hmixcoef.h
 

Updated to revision 291.
 

>

The above is with an older version of svn. The latest, greatest does this:

Conflict discovered in ’ROMS/Utility/inp_par.F’.
 

Select: (p) postpone, (df) diff-full, (e) edit,
 

(mc) mine-conflict, (tc) theirs-conflict,
 

(s) show all options:

Selecting (p) will behave as the old version.

If you get a conflict, you need to do one of three things:

•	 Merge the conflicted text “by hand” by examining and editing the conflict markers within

the file.

•	 Copy one of the temporary files on top of your working file.

•	 Run svn revert <filename> to throw away all of your local changes.

Once you’ve resolved the conflict, you need to let Subversion know by running “svn resolved”. This

removes the three temporary files and Subversion no longer considers the file to be in a state of

conflict. More on this below.

I.5.1 Merging conflicts by hand

To merge your changes with those from the latest revision in the repository, open ana_hmixcoef.h

in your favorite editor and look for a string of “<” characters. You should see something like this:

<<<<<<< .mine
 

#ifndef DISTRIBUTE
 

IF (Lanafile.and.(tile.eq.0)) THEN
 

#else
 

IF (Lanafile) THEN
 

#endif
 

=======
 

#ifdef DISTRIBUTE
 

IF (Lanafile) THEN
 

#else
 

IF (Lanafile.and.(tile.eq.0)) THEN
 

#endif
 

>>>>>>> .r291
 

162
 
After comparing the two code blocks (separated by the “=” symbols), you decide that you prefer

the logic from the repository so you remove the conflict markers and your code so the section now

looks like this:

#ifdef DISTRIBUTE

IF (Lanafile) THEN

#else

IF (Lanafile.and.(tile.eq.0)) THEN

#endif

Now you can save the file and let Subversion know that you have resolved the conflict:

> svn resolved User/Functionals/ana_hmixcoef.h

Resolved conflicted state of ’User/Functionals/ana_hmixcoef.h’

I.5.2 Copying a file onto your working file

If you get a conflict and decide that you want to throw out your changes, you can merely copy one

of the temporary files created by Subversion over the file in your working copy. Let’s say you want

to use the new revision from the repository:

> cd User/Functionals

> ls ana_hmixcoef.h*

ana_hmixcoef.h ana_hmixcoef.h.mine ana_hmixcoef.h.r280

ana_hmixcoef.h.r291

> cp ana_hmixcoef.h.r291 ana_hmixcoef.h

> svn resolved ana_hmixcoef.h

Resolved conflicted state of ’ana_hmixcoef.h’

I.5.3 Punting: Using svn revert

If you get a conflict, and upon examination decide that you want to throw out your changes and

start your edits again, just revert your changes:

> cd User/Functionals

> svn revert ana_hmixcoef.h

Reverted ’ana_hmixcoef.h’

Note that when you revert a conflicted file, you don’t have to run “svn resolved”.

163
 
References

J. S. Allen, P. A. Newberger, and J. Federiuk.	 Upwelling circulation on the oregon continental

shelf. part i: Response to idealized forcing. J. Phys. Oceanogr., 25:1843–1866, 1995.

A. Arakawa and V. R. Lamb.	 Methods of computational physics, volume 17, pages 174–265. Academic Press, 1977.

G. K. Batchelor. An introduction to fluid dynamics. Cambridge University Press, 1967.

J. P. Beamer, D. F. Hill, A. Arendt, and G. E. Liston. High-resolution modeling of coastal freshwater

discharge and glacier mass balance in the gulf of alaska watershed. Water Resour. Res., 52:3888–

âĂŞ3909, 2016. doi:10.1002/2015WR018457.

A. Beckmann and D. B. Haidvogel. Numerical simulation of flow around a tall, isolated seamount.

part i: Problem formulation and model accuracy. J. Phys. Oceanogr., 23:1736–1753, 1993.

W.P. Budgell.	 Numerical simulation of ice-ocean variability in the barents sea region: Towards

dynamical downscaling. Ocean Dynamics, 2005. doi:10.1007/s10236-005-0008-3.

J. A. Carton, B. S. Giese, and S. A. Grodsky.	 Sea level rise and the warming of the oceans in the

soda ocean reanalysis. J. Geophys. Res., 110, 2005. doi:10.1029/2004JC002817.

S. Chacon and B. Straub.	 Pro Git. Apress, second edition, 2014. ISBN 978-1484200773.

https://progit.org/.

D. C. Chapman. Numerical treatment of cross-shelf open boundaries in a barotropic coastal ocean

model. J. Phys. Oceanogr., 15:1060–1075, 1985.

Ben Collins-Sussman, Brian W. Fitzpatrick, and C. Michael Pilato. Version Control with Subversion. O’Reilly & Associates, Inc., Sebastopol, CA, second edition, 2008. http://svnbook.redbean.com/.

A. Dai, T. Qian, K. E. Trenberth, and J. D Milliman. Changes in continental freshwater discharge

from 1948–2004. J. CLimate, 22:2773–2791, 2009. doi:10.1175/2008JCLI2592.1.

E. E. Ebert and J. A. Curry.	 An intermediate one-dimensional thermodynamic sea ice model for

investigating ice-atmosphere interactions. J. Geophys. Res., 98:10085–10109, 1993.

C. W. Fairall, E. F. Bradley, J. E. Hare, A. A. Grachev, and J. B. Edson. Bulk parameterization

of air-sea fluxes: Updates and verification for the coare algorithm. J. Climate, 16:571–591, 2003.

K. N. Fedorov. Layer thicknesses and effective diffusivities in the diffusive thermocline convection

in the ocean. In J. C. J. Nihoul and B. M. Jamart, editors, Small-scale turbulence and mixing

in the ocean, pages 471–479. Elsevier, New York, 1988.

K. Fennel, J. Wilkin, J. Levin, J. Moisan, J. O’Reilly, and D. Haidvogel. Nitrogen cycling in

the mid atlantic bight and implications for the north atlantic nitrogen budget: Results from a

three-dimensional model. Global Biogeochem. Cycles, 20, 2006. doi:10.1029/2005GB002456.

R. A. Flather. A tidal model of the northwest european continental shelf.	 Memoires de la Société

Royale de Sciences de Liège, 6:141–164, 1976.

M. G. G. Foreman.	 Manual for tidal heights analysis and prediction. Technical Report 77-10,

Institute of Ocean Sciences, Patricia Bay, 1996a. Pacific Marine Science Report.

164
 
M. G. G. Foreman.	 Manual for tidal currents analysis and prediction. Technical Report 77-10,

Institute of Ocean Sciences, Patricia Bay, 1996b. Pacific Marine Science Report.

P. J. S. Franks, J. S. Wroblewski, and G. R. Flierl.	 Behavior of a simple plankton model with

food-level acclimation by herbivores. Mar. Biol., 91:121–129, 1986.

N. G. Freeman, A. M. Hale, and M. B. Danard.	 A modified sigma equations’ approach to the

numerical modeling of great lake hydrodynamics. J. Geophys. Res., 77(6):1050–1060, 1972.

B. Galperin, L. H. Kantha, S. Hassid, and A. Rosati. A quasi-equilibrium turbulent energy model

for geophysical flows. J. Atmos. Sci., 45:55–62, 1988.

S.M. Griffies and R.W. Hallberg. Biharmonic friction with a smagorinsky-like viscosity for use in

large-scale eddy-permitting ocean models. Mon. Wea. Rev., 128:2935–2946, 2000.

S.M. Griffies, A. Gnanadesikan, R.C. Pacanowski, V. Larichev, J.K. Dukowicz, and R.D. Smith.

Isoneutral diffusion in a z-coordinate ocean model. J. Phys. Oceanogr., 28:805–830, 1998.

D. B. Haidvogel and A. Beckmann. Numerical Ocean Circulation Modeling. Imperial College Press,

1999.

D. B. Haidvogel, H. G. Arango, K. Hedstrom, A. Beckmann, P.	 Malanotte-Rizzoli, and A. F.

Shchepetkin. Model evaluation experiments in the north atlantic basin: Simulations in nonlinear

terrain-following coordinates. Dyn. Atmos. Ocean., 32:239–281, 2000.

D. B. Haidvogel, H. G. Arango, W. P. Budgell, B. D. Cornuelle, E. Curchitser, E. Di Lorenzo,

K. Fennel, W. R. Geyer, A. J. Hermann, L. Lanerolle, J. Levin, J. C. McWilliams, A. J.

Miller, A. M. Moore, T. M. Powell, A. F. Shchepetkin, C. R. Sherwood, R. P. Signell, J. C.

Warner, and J. Wilkin. Ocean forecasting in terrain-following coordinates: formulation and

skill assessment of the regional ocean modeling system. J. Comp. Phys., 227:3429–3430, 2007.

doi:10.1016/j.jcp.2007.01.016.

W. P. Hazard. Using cpp to aid portability. Computer Language, 8(11):49–54, 1991.

K. S. Hedstrom. Technical manual for a coupled sea-ice/ocean circulation model (version 2). Technical report, Institute of Marine and Coastal Sciences, New Brunswick, NJ, June 2000. OCS

Study MMS 2000-047.

W. D. Hibler, III. A dynamic thermodynamic sea ice model. J. Phys. Oceanogr., 9:815–846, 1979.

S. Hinckley, K. O. Coyle, G. Gibson, A. J. Hermann, and E. L. Dobbins. A biophysical npz model

with iron for the gulf of alaska: Reproducing the differences between an oceanic hnlc ecosystem

and a classical northern temperate shelf ecosystem. Deep Sea Res. II, 2009. in press.

W. R. Holland, J. C. Chow, and F. O. Bryan. Application of a third-order upwind scheme in the

ncar ocean model. J. Climate, 11:1487–1493, 1998.

E. C. Hunke. Viscous-plastic sea ice dynamics with the evp model: linearization issues.	 J. Comp.

Phys., 170:18–38, 2001.

E. C. Hunke and J. K. Dukowicz. An elastic-viscous-plastic model for sea ice dynamics.	 J. Phys.

Oceanogr., 27:1849–1868, 1997.

E. C. Hunke, W. H. Lipscomb, A. K. Turner, N. Jeffery, and S. Elliott. Cice: the los alamos sea

ice model documentation and software user’s manual, version 5.0. Technical report, LANL, Los

Alamos, NM, 2013. LA-CC-06-012.

165
 
D. R. Jackett and T. J. McDougall. Stabilization of hydrographic data.	 J. Atmos. Ocean. Tech.,

12:381–389, 1995.

Y. Kanarska, A. F. Shchepetkin, and J. C. McWilliams. Algorithm for non-hydrostatic dynamics

in roms. Ocean Modeling, 18:143–174, 2007.

R. F. Keeling, B. B. Stephens, R. G. Najjar, S. C. Doney, D. Archer, and M. Heimann. Seasonal

variations in the atmospherice o2/n2 ratio in relation to kinetics of air-sea gas exchange. Global

Biogeochem. Cycles, 12:141–163, 1998.

B. W. Kernighan and D. M. Ritchie.	 The C Programming Language. Prentice Hall, Englewood

Cliffs, New Jersey 07632, second edition, 1988.

M. J. Kishi, M. Kashiwai, D. M. Ware, B. A. Megrey, D. L. Eslinger, F. E. Werner, M. Noguchi-Aita,

T. Azumaya, M. Fujii, S. Hashimoto, D. Huang, H. Iizumi, Y. Ishida, S. Kang, G. A. Kantakov,

H.-C. Kim, K. Komatsu, V. V. Navrotsky, S. Lan Smith, K. Tadokoro, A. Tsuda, O. Yamamura,

Y. Yamanaka, K. Yokouchi, N. Yoshie, J. Zhang, Y. I. Zuenko, and V. I. Zvalinsky. Nemuro—a

lower trophic level model for the north pacific marine ecosystem. Ecological Modelling, 2002:

12–25, 2007.

H. Lamb. Hydrodynamcis. Cambridge University Press, 6 edition, 1932. (1945 Dover Publications

reproduction).

W. G. Large. Modeling and parameterization ocean planetary boundary layers. In E. P. Chassignet

and J. Verron, editors, Ocean Modeling and Parameterization, pages 81–120. Kluwer Academic

Publishers, 1998.

W. G. Large and S. G. Yeager. The global climatology of an interannually varying air-sea flux data

set. Clim. Dyn., 33:341–364, 2009. DOI 10.1007/s00382-008-0441-3.

W. G. Large, J. C. McWilliams, and S. C. Doney.	 Oceanic vertical mixing: a review and a model

with a nonlocal boundary layer parameterization. Rev. Geophys., 32:363–403, 1994.

J. R. Ledwell, A. J. Wilson, and C. S. Low. Evidence for slow mixing across the pycnocline from

an open-ocean tracer-release experiment. Nature, 364:701–703, 1993.

J.-F. Lemieux, B. Tremblay, F. Dupont, M. Plante, G. C. Smith, and D. Dumont. A basal

stress parameterization for modeling landfast ice. J. Geophys. Res., 120:3157–3173, 2015.

doi:10.1002/2014JC010678.

B. P. Leonard. A stable and accurate convective modelling procedure based on quadratic upstream

interpolation. Comput. Method Appl. Mech. Eng., 19:59–98, 1979.

S.-J. Lin. A finite volume integration method for computing pressure gradient force in general

vertical coordinates. Quart. J. R. Met. Soc., 123:1749–1762, 1997.

A. Macks and J. Middleton. Numerical modelling of wind-driven upwelling and downwelling. University of New South Wales, 1993.

J. Mailhôt and R. Benoit. A finite-element model of the atmospheric boundary layer suitable for

use with numerical weather prediction models. J. Atmos. Sci., 39:2249–2266, 1982.

P. Marchesiello, J. C. McWilliams, and A. Shchepetkin. Open boundary conditions for long-term

integration of regional oceanic models. Ocean Modelling, 3:1–20, 2001.

L. Margolin and P. K. Smolarkievicz.	 Antidiffusive velocities for multipass donor cell advection.

SIAM J. Sci. Comput., pages 907–929, 1998.

166
 
E. Mason, J. Molemaker, A. F. Shcheptkin, F. Colas, J. C. McWilliams, and P. Sangrà. Procedures

for offline grid nesting in regional ocean models. Ocean Modelling, 35:1–15, 2010.

G. A. Maykut and N. Untersteiner. Some results from a time-dependent thermodynamic model of

sea ice. J. Geophys. Res., 76:1550–1575, 1971.

John D. McCalpin. A comparison of second-order and fourth-order pressure gradient algorithms in

a σ-coordinate ocean model. Int. J. Num. Meth. Fluids, 18:361–383, 1994.

R. Mechlenburg. Managing Projects with GNU Make. O’Reilly & Associates, Inc., Sebastopol, CA,

2005.

G. L. Mellor. The three-dimensional current and surface wave equations.	 J. Phys. Oceanogr., 33:

1978–1989, 2003.

G. L. Mellor. Some consequences of the three-dimensional currents and surface wave equation.	 J.

Phys. Oceanogr., 35:2291–2298, 2005.

G. L. Mellor. The depth-dependent current and wave interaction equations: a revision.	 J. Phys.

Oceanogr., 38:2586–2596, 2008.

G. L. Mellor and L. Kantha.	 An ice-ocean coupled model. J. Geophys. Res., 94:10,937–10,954,

1989.

G. L. Mellor and T. Yamada. A hierarchy of turbulence closure models for planetary boundary

layers. J. Atmos. Sci., 31:1791–1806, 1974.

G. L. Mellor and T. Yamada.	 Development of a turbulence closure model for geophysical fluid

problems. Rev. Geophys. Space Phys., 20:851–875, 1982.

W. A. Oost, G. J. Komen, C. M. J. Jacobs, and C. van Oort. New evidence for a relation between

wind stress and wave age from measurements during asgamage. Bound.-Layer Meteor., 103:

409–438, 2002.

I. Orlanski. A simple boundary condition for unbounded hyperbolic flows. J. Comp. Phys., 21(3):

251–269, July 1976.

J. E. Overland and C. H. Pease.	 Modeling ice dynamics of coastal seas. J. Geophys. Res., 93:

15,619–15,637, 1988.

C. L. Parkinson and W. M. Washington. A large-scale numerical model of sea ice.	 J. Geophys.

Res., 84:6565–6575, 1979.

P. Penven, L. Debreu, P. Marchesiello, and J. C. McWilliams. Evaluation and application of the

roms 1-way embedding procedure to the central california upwelling system. Ocean Modeling,

12:157–187, 2006.

H. Peters, M. C. Gregg, and J. M. Toole. On the parameterization of equatorial turbulence.	 J.

Geophys. Res., 93:1199–1218, 1988.

N. A. Phillips. A coordinate system having some special advantages for numerical forecasting.	 J.

Meteorology, 14(2):184–185, 1957.

T. M. Powell, C. V. W. Lewis, E. N. Curchitser, D. B. Haidvogel, A. J. Hermann, and E. L.

Dobbins. Results from a three-dimensional, nested biological-physical model of the california

current system and comparisons with statistics from satellite imagery. J. Geophys. Res., 111

(C07018), 2006. doi:10.1029/2004JC002506.

167
 
W. H. Press, B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling.	 Numerical Recipes, The Art

of Scientific Computing. Cambridge University Press, 1986.

P. J. Rasch. Conservative shape-preserving two-dimensional transport on a spherical reduced grid.

Mon. Wea. Rev., 122:1337–1350, 1994.

W. H. Raymond and H. L. Kuo.	 A radiation boundary condition for multi-dimensional flows.

Quart. J. R. Met. Soc., 110:535–551, 1984.

R. Rew, G. Davis, S. Emmerson, and H. Davies.	 NetCDF User’s Guide. Unidata, University

Corporation for Atmospheric Research, Boulder, Colorado, 1996. Version 2.4.

M. M. Rienecker, M. J. Suarez, R. Gelaro, R. Todling, J. Bacmeister, E. Liu, M. G. Bosilovich,

S. D. Schubert, L. Takacs, G.-K. Kim, S. Bloom, J. Chen, D. Collins, A. Conaty, and A. da Silva.

Merra: Nasa’s modern-era retrospective analysis for research and applications. J. Climate, 24:

3624–3648, 2011. doi:10.1175/JCLI-D-11-00015.1.

R. Sadourny and K. Maynard.	 Formulations of lateral diffusion in geophysical fluid dynamics

models. In C.A. Lin, R. Laprise, and H. Ritchie, editors, Numerical Methods of Atmospheric and

Oceanic Modelling, pages 547–556. NRC Research Press, 1997.

A. J. Semtner, Jr. A model for the thermodynamic growth of sea ice in numerical investigations of

climate. J. Phys. Oceanogr., 6:379–389, 1976.

A. F. Shchepetkin and J. C. McWilliams.	 Quasi-monotone advection schemes based on explicit

locally adaptive dissipation. Mon. Wea. Rev., 126:1541–1580, 1998.

A. F. Shchepetkin and J. C. McWilliams. A method for computing horizontal pressure-gradient

force in an oceanic model with a nonaligned vertical coordinate. J. Geophys. Res., 108:1–34,

2003. doi:10.1029/2001JC001047.

A. F. Shchepetkin and J. C. McWilliams. The regional ocean modeling system (roms): A splitexplicit, free-surface, topography-following coordinates oceanic model. Ocean Modeling, 9:347–

404, 2005.

A. F. Shchepetkin and J. C. McWilliams.	 A correction note for “ocean forecasting in terrainfollowing coordinates: formulation and skill assessment of the regional ocean modeling system”.

J. Comp. Phys., 228:8985–9000, 2009a.

A. F. Shchepetkin and J. C. McWilliams. Computational kernel algorithms for fine-scale, multiprocess, long-time oceanic simulations. In R. Temam and J. Tribbia, editors, Handbook of Numerical Analysis: Computational Methods for the Ocean and the Atmosphere, volume 14, pages

121–183. Elsevier Science, 2009b. published.

A. F. Shchepetkin and J. C. McWilliams. An accurate boussinesq oceanic model with a practical,

“stiffened” equation of state. Ocean Modeling, 38:41–70, 2011.

J. Smagorinsky.	 General circulation experiments with the primitive equations. i. the basic experiment. Mon. Wea. Rev., 91:99–164, 1963.

P.	 K. Smolarkiewicz and W. W. Grabowski. The multidimensional positive definite advection

transport algorithm: non-oscillatory option. J. Comp. Phys., 86:355–375, 1990.

Y. Song.	 A general pressure gradient formulation for ocean models. part i: Scheme design and

diagnostic analysis. Mon. Wea. Rev., 126:3213–3230, 1998.

168
 
M. Steele, G. L. Mellor, and M. G. McPhee.	 Role of the molecular sublayer in the melting or

freezing of sea ice. J. Phys. Oceanogr., 19:139–147, 1989.

P.	 K. Taylor and M. A. Yelland. The dependence of sea surface roughness on the height and

steepness of the waves. J. Phys. Oceanogr., 31:572–590, 2001.

J. Thuburn. Multidimensional flux-limited advection schemes. J. Comp. Phys., 123:74–83, 1996.

I. B. Troen and L. Mahrt. A simple model of the atmospheric boundary layer; sensitivity to surface

evaporation. Boundary-Layer Meteor., 37:129–148, 1986.

L. Umlauf and H. Burchard. A generic length-scale equation for geophysical turbulence models. J.

Marine Res., 61:235–265, 2003.

R. C. Wajsowicz. A consistent formulation of the anisotropic stress tensor for use in models of the

large-scale ocean circulation. J. Comp. Phys., 105:333–338, 1993.

J. C. Warner, C. R. Sherwood, H. G. Arango, and R. P. Signell. Performance of four turbulence

closure models implemented using a generic length scale method. Ocean Modelling, 8:81–113,

2005.

J. C. Warner, C. R. Sherwood, R. P. Signell, C. K. Harris, and H. G. Arango. Development of a

three-dimensional, regional, coupled wave, current, and sediment-transport model. Computers

& Geosciences, 34:1284–1306, 2008.

D. J. Webb, B. A. De Cuevas, and C. S. Richmond. Improved advection schemes for ocean models.

J. Atmos. Ocean. Tech., 15:1171–1187, 1998.

Jonathan Whitefield, Peter Winsor, James W. McClelland, and Dimitris Menemenlis. A new river

discharge and river temperature climatology data set for the pan-arctic region. Ocean Modelling,

2015. doi:10.1016/j.ocemod.2014.12.012.

J. Wilkin and K. Hedstrom. User’s manual for an orthogonal curvilinear grid-generation package.

Institute for Naval Oceanography, 1991.

P. Xiu and F. Chai. Spatial and temporal variability in phytoplankton carbon, chlorophyll, and

nitrogen in the north pacific. J. Geophys. Res., 117, 2012. doi:10.1029/2012JC008067.

P. Xiu and F. Chai. Connections between physical, optical and biogeochemical processes in the

pacific ocean. Prog. in Oceanogr., 122:30–53, 2014. doi:10.1016/j.pocean.2013.11.008.

A. M. Yaglom and B. A. Kader. Heat and mass transfer between a rough wall and turbulent fluid

at high reynolds and peclet numbers. J. Fluid. Mech., 62:601–623, 1974.

169
 
www.boem.gov

The Department of the Interior Mission

As the Nation's principal conservation agency, the Department of the Interior has

responsibility for most of our nationally owned public lands and natural resources. This

includes fostering the sound use of our land and water resources, protecting our fish,

wildlife and biological diversity; preserving the environmental and cultural values of

our national parks and historical places; and providing for the enjoyment of life through

outdoor recreation. The Department assesses our energy and mineral resources and

works to ensure that their development is in the best interests of all our people by

encouraging stewardship and citizen participation in their care. The Department also

has a major responsibility for Americal Indian reservation communities and for people

who live in island communities.

The Bureau of Ocean Energy Management

The Bureau of Ocean Energy Management (BOEM) works to manage the exploration

and development of the nation's offshore resources in a way that appropriately balances

economic development, energy independence, and environmental protection through oil

and gas leases, renewable energy development and environmental reviews and studies.