# Presentation: 533671

## Framework for Modernization and Componentization of Fusion Modules (FMCFM): Status and Vision DE-FG-02-05ER84192

- Framework for Modernization and Componentization of Fusion Modules (FMCFM): Status and Vision DE-FG-02-05ER84192

**Ovsei Volberg** (PI)
**Svetlana Shasharina and Johan Carlsson**

## Acknowledgements

- We would like to express our gratitude for the help, advice, comments and encouragement to the following people (in alphabetical order):
**Glenn Bateman, John R. Cary, Stephen Jardin, Jon Kinsey and Arnold Kritz **
- All the people of the NTCC project.
- US DOE for the Phase 1 SBIR award.
- Finally I personally would like to thank all of you for inviting me to this seminar and for coming to listen to my talk.

- Tech-X Corporation

## Project Goal

**Assist  Fusion Integrated  Modeling projects by developing a framework that will contain a *****set of tools, standards, and techniques***** for **_**modernization**_** of established fusion codes with subsequent **_**conversion  of these modules to components**_** with clear defined interfaces in accordance with modern software engineering practices.**
**The project addresses the following problems relevant for the FSP****:**
**Modernization of Fortran modules to avoid their reimplementation, protect investment in software development, and increase performance and safety.**
**Conversion of modernized modules components with well-defined interfaces to facilitate their coupling.**
**Solve the language interoperability problem – use in the simulation of components written in different programming languages.**
**Gain experience in components coupling.**

- Tech-X Corporation

## DOE SBIR Program

- Contains 2 development phases
- _Phase I_:  100K, prototype development, project’s concretization
- _Phase II_:  $750, full development
- FMCFM is currently  a Phase I project

- Tech-X Corporation

## Goals in Phase 1

**Evaluate NTCC standards and procedures. **
**Compare them with other community-wide efforts in other areas of science. **
**Extend them based on this comparison. **
**Use the new standards and Fortran 90 modernization techniques to modernize MMM95 and GLF23 transport modules.**
**Define simplified interfaces between models and solvers**
**Investigate the feasibility of  using Common Component Architecture for converting fusion transport  modules and solvers into components**
**Evaluate CCA CCaffiene framework for  component coupling and simulation.**
***This will result in a prototype framework for fusion code modernization and componentization***

- Tech-X Corporation

## Accomplishments to date

- Modernization
**We produced drafts of the additional (to NTCC Standards)  development documents  to be included in the FMCFM Developer’s Guide:**
**Fortran Coding Rules**
**Naming Convention**
**Policy for Interoperability  Document that incorporates modified NTCC Standards as requirements**
**Provided a flexible  and portable build system  based on GNU Autotools (*****de facto *****standard for open source development)  for the framework  that could benefit other community projects as well.**
**By using modern software engineering concepts available in Fortran 90/95 we  created Fortran 90 wrappers to modernize two NTCC transport modules**** **
**MMM95 - Multi-Mode transport model**
**GLF23 - Gyro-Landau-Fluid drift wave model**
- Incorporated a new transport equation solver implemented in C
**A combination of a relaxation scheme with a quasi-Newton scheme, well-suited for block-diagonal problems, like transport equations **
- FMCFM is work in progress, it is not finished yet.

- Tech-X Corporation

## Outline for the rest of the talk

- Overview of Phase 1 modernization efforts
**What do we like in the NTCC ?**
**Comparison of NTCC with other community wide efforts**
**New development documentation**
**Modernization of MMM95 and GLF23**
- Componentization and coupling by using CCA tools and techniques
**Simplified simulation**
**Driver, Transport and Solver CCA Components**
**Development of Ccaffeine coupled simulation**
- Conclusions

- Tech-X Corporation

## Phase 1 FMCMF  is a prototype:

- Phase I project is limited in scope:
**Centered around the NTCC Modules Library and NTCC Standards **
**Focused on modernization of just 2 transport modules:**
**MMM95 - Multi-Mode transport model**
**GLF23 - Gyro-Landau-Fluid drift wave model**
**Uses only CCA tools for componentization**
- But the FMCFM will help us to answer the following questions important for fusion integrated modeling:
**Which tools and techniques could be used for modernization?**
**What are the benefits and difficulties  of using the CCA tools for componentization? **
**How does one make a component out of existing code**?
**What kind of interfaces are needed for coupling?**

- Tech-X Corporation

## What do we like  about the NTCC project?

- NTCC Standards - a good set of requirements
- Rob Jacob, one of the designers of the CCSM Flux Coupler: “I wish that the Climate community had such a set of requirements before we started”
- Thanks to the strict acceptance requirements, NTCC modules are modified already:
**Carries out a specific task (transport, heating and etc.)**
**Isolated with an interface such as an argument list of a procedure**
**Contains the driver program for testing, test cases, and documentation**
- We like how the maintenance issues are solved:
**Peer Review Process**
**Modern coding practices encouraged (for the new modules)**
**Web-based access to module download and documentation**

- Tech-X Corporation

**Notes:**
Short answer : a lot
Rob Jacobs: I wish the Climate Community would have such standards before starting CCSM
Significant step towards the integrated modeling.

## We evaluated several community-wide projects  and their development standards

**Sun-Earth System Modelling and Space Weather Prediction**
**Space Weather Modelling Framework (SWMF**) **http://csem.engin.umich.edu/SWMF**
**Center  for Integrated Space Weather Modelling (CISM) http://www.bu.edu/cism**
**Earth Climate and Weather Modelling and Forecasting**
**Earth System Modelling Framework (ESMF) http://www.esmf.ucar.edu**
**European Program for Integrated Earth System Model (PRISM) http://prism.enes.org**
**Community Climate System Model (CCSM) **_[http://www.ccsm.ucar.edu](http://www.ccsm.ucar.edu/)_
**Aerodynamics Analysis and Design**
**NASA Report TM-2003-212421**
**Meteorology**
**European Standards for Writing and Documenting Exchangeable Fortran 90 Code**
**Common Components Architecture**** **
**http://www.cca-forum.org**

- Tech-X Corporation

## Results: Evaluation and comparison with the NTCC 

**We elevated several “NTCC goals” to standards, for example:**
**Avoid using hard-wired I/O unit numbers.  Allow informational output to be switched on or off.  Provide a method for rerouting warning or error message output to a user specified file or I/O unit number**
**We enhanced the NTCC Standards by  the “Parallelization Standards” to accommodate requirements of high-performance computing.**
**We developed additional software engineering documents for the future FMCFM Developer’s Guide:**
**Fortran 90 Coding Rules**
**Naming Convention (both for C/C++ and Fortran)**
**Interoperability Specification (the enhanced  and modified NTCC Standards became a section on Requirements in this document)**
***The documents have been published on the Tech-X website via Confluence software:( ***_[http://](http://grid.txcorp.com/)__[grid.txcorp.com](http://grid.txcorp.com/)_*** and go to Fusion Framework Space) We will publish number of evaluated documents on this web site as well.***
**We use the portable build system based on GNU Autotools similar to many community-wide projects, for example CCA and ESMF.  We reinforced the build system by the Tech-X config module**.

- Tech-X Corporation

## Results (cont.): Parallelization Standards

**Standard:** The parallelization mechanism (if any) should employ the MPI standard for inter-module communications.
**Standard:** If the MPI standard is used for intra-module communications, the physics module should be able to use an arbitrary MPI communicator.
**Standard: **The module needs to be compiled as a library in addition to its standard mode when it is compiled and linked into a stand-alone executable.
**Standard: **The module should read input from and write output to files. No hard-coded unit numbers are allowed.
**Goal: **Module should be able to write its state to restart file(s) and continue its run after a stop of execution using restart file(s) with bit-to bit compatibility on the same platform-compiler combination.

- Tech-X Corporation

## Our Fortran 90 Modernization Approach: Abstraction, Encapsulation and Modularity

**MODULE**: modules encapsulate data and routines; modules are containers for grouping type definitions, interface blocks, data and procedures. **USE** gives the access to module’s content.
**Allocatable arrays/pointers**: support for dynamic data structures.
**TYPE**: the user defined data types support abstraction.
**INTERFACE **and** MODULE PROCEDURE**: support method overloading: the same method name , different argument lists.
**PRIVATE** and **PUBLIC**: support data hiding and public access.
**Explicit interface blocks**: enforce argument types verification in procedure calls.
**Renaming**:  prevent method name clashes between modules.
**Array syntax**: whole array operations

- Tech-X Corporation

## Fortran features to be avoided (from our Fortran Coding Rules document)

- COMMON blocks – use the declaration of module instead.
- real*8 - use kind  statement instead
- EQUIVALENCE - use  pointer or derived data types instead to form data structures.
- Assigned and computed GOTO - use the case construct instead.
- Arithmetic IF statements - use the block **if, else, else if , end if **or** select case** construct instead. Labeled DO constructs - use unlabeled **end do** instead.
- PAUSE
- DATA and BLOCK DATA.

- Tech-X Corporation

## Modernization of the NTCC Fortran 77 Modules 

- Each transport module main program is broken into 2 Fortran 90 (F90)  modules: a data definition module  and a module with procedures that operate on this data.
- Each COMMON block is substituted by a F90 module
- No hard-wired I/O unit numbers, all I/O operations to and from file; these files are associated with unit numbers at run-time via **ModIoUtil** F90 module.
**ModKind** F90 module is used
**ModTransportInterfaceBlocks** that includes explicit interface blocks for **callglf2d** and **mmm95** routines
- Each transport module is built into a library: we link it with the test program if we need a stand alone version or we use it to create a CCA component with ***ports***.

- Tech-X Corporation

## Example: two main F90 modules for each transport module

- Data F90 module
- module ModMMM95
-   use ModUnitNumbers
-   implicit none
-   save
- .....
- real :: zdx, zxfact, zepslon
-   ! the dimension of arrays below <= kr
-   real, dimension(:),allocatable :: zxb , zrminor , zrmajor , zelong , zdense
-   real, dimension(:),allocatable :: zdensi , zdensh , zdensz , zdensf , zdensfe
- ...
- end module ModMMM95

- Procedure F90 module
- module ModWrapMMM95
-   use ModMMM95
-   use ModTranspInterfaceBlocks
-   implicit none
-  !  private   ! except
-  !PUBLIC MEMBER FUNCTIONS
-   public :: init_io
-   public :: set_defaults
-  public :: read_input_namelist
-  public :: verify_data
-   public :: setup
-   public :: compute
-   public :: write_output
-   public :: finalize
- contains
- ......
- end ModWrapMMM95

- Tech-X Corporation

## Example: ModIoUnit

- Checks for I/O units that are free (unused).
- Associates files named ***input, temp*** and **output** with unused I/O units and opens them.
- Associates any named file with an unused I/O unit number and opens it.
- Closes opened files.
- An additional set of subroutines in a separate file allows to interface module procedures from Fortran 77 when USE statement  cannot be used.

- Tech-X Corporation

## Example: ModTranspInterfaceBlocks

- ModTranspInterfaceBlocks contains the explicit interface blocks for subroutines callglf2d and mmm95 that perform calculations for the transport modules  GLF23 and MMM95. If this module is accessed by a USE statement in any program unit that makes calls to any of those subroutines, then those calls would be checked by compiler for errors in the number or type of arguments.
- An explicit interface for a computational routine of any new transport module will be added to this module

- Tech-X Corporation

## Evaluation of the CCA is in progress 

- Our current/future use of the CCA tools:
- The Ccaffeine execution environment for assembling coupled simulation from the models converted to components
- A peer-wise communication pattern via ports (common interfaces):
**Provide ports that are implemented by a component**
**Uses ports, where c component makes calls to ports provided by another  component**
- Scientific Interface Definition Language (SIDL) that works with Babel language interoperability tool to address the language interoperability between Fortran transport components and the solver written in C.
- Our current work is concentrated on :
**We prototyped the ****provide/uses ports**** (CCA interfaces) for a ****Driver Component, Transport Component  ****and ****Solver Component**
**At the current step, we have started to work with the CCA language interoperability tools (SIDL/Babel**) **to create interoperable  components**

- Tech-X Corporation

## Simplified coupling as a test

- For the initial coupling we consider densities and electron temperature and its gradient constant
- We assume that only ion temperature ***T******i*** and its gradient ***grad T******i*** constitute a state vector at the moment ***t**** in the simplified simulation (not to produce good physics, but to learn coupling in Ccaffeine)*
- The initial input parameters for a transport component are taken from its test cases downloaded from the NTCC web site.
- Set and Get methods for the state vector
- The new evaluate method that includes set methods  and  compute method.

- Tech-X Corporation

## Sketch for Ccaffeine Coupling of FMCFM Components

- Tech-X Corporation

## We are ready for CCA:

- Specifying CCA provide and use ports for the **driver component**, **transport component** and **solver component**
- Using **SIDL/Babel** to address the language interoperability.
- Using **Ccaffeine** framework as an environment to couple components
**“****Receiver makes right”** approach to insure that components get data in the form they need it.

- Tech-X Corporation

## Conclusions

- We gained valuable experience in NTCC module modernization, standards development and documentation.
- We created a portable build system that has been successfully used in our project.
- We developed Fortran modules and collected tools that can assist the community in the modernization  efforts.
- We incorporated a new solver into the FMCFM
- We are working to formulate a coherent approach to how to build CCA components from Fortran legacy codes.
- We need to spend more time to determine common interfaces  for coupling Driver with Transport Components (problems input/output transformations)

- Tech-X Corporation

## Questions to you

**What suggestions do you have for our work with standards and development documentation?**
**What improvements could you recommend to our Fortran code modernization process?**
**Suggestions for finishing the Phase 1 project and contents of the final report?**
**Could you recommend what kind of role we could play in modernization and componentization which would help existing FSP efforts?**
**Other modules?**

- Tech-X Corporation