# Presentation: 385700

## FRErator – the Bridge between FRE and Curator DB

## Uniform, Economical  Approach

- DB makes easy the reusability of blocks needed for building and configuring new model and running  experiments.
- Metadata giving full description of models, platforms, and runs helps in analysis and making decision.
- Metadata written in Curator DB during experiment is used for Data Portal support.
- Automation saves user time in registering new experiment and making it published on Data Portal
- Metadata about current experiments runs updated in real-time informs users about status of their jobs

## Role of Curator DB on different stages of modeling process

- Role of Curator DB on different stages of modeling process

- Metadata and Configuring Data - Curator DB

- Component Building

- Model Assembling

- Experiment Preparation

- Postprocessing

- Data Portal Service

**FMS Runtime Environment**

**in development**
**currently**

**Notes:**
Eventually, Curator DB will serve as global GFDL metadata provider on all stages of modeling process starting from Component Building through Modeling Assembling, Experiment Preparation, Postprocessing and finishing on Publishing data on Data Portal.
Currently, it’s already used for publishing purpose and we are developing new version of FRE where these ideas are being implemented.

## Modes of working with database

**Research mode** - modeler introduces new algorithmizations and new components from newly developed modules for future usage in coupled models. New components are to be described in database manually (not from rts.xml). The model runs conducted for this developed purpose are not to be recorded in DB excepting final ones proving physical correctness of new approach.
**Production mode** – experimenter composes coupled model from available components described in database, builds scenario, postprocessing plan using GUI and runs experiment. All this activity is recorded in database automatically.
**Automatic mode** – applications fill metadata and config data into database grabbing it from rts.xml file (created manually) during running FRE tools.

**Notes:**
During design DB it’s vital to keep in mind at least basic scenarios how users will work with DB. It can be selected 3 main modes of working with DB:
Research mode ...
Production mode ...
Automatic mode ...

## Current usage of Curator DB

- DB serves for GFDL Data Portal web site for data discovery and navigation: _[IPCC CM2.1](http://nomads.gfdl.noaa.gov/CM2.X/CM2.1/available_data.html)__. The daemon screens Data Portal storage seeking newly put data files and records metadata extracted from files and system information about them into DB. _
- DB is used for bringing metadata consistency data files on Data Portal with accepted standards (described in DB). The application accesses to DB for metadata standard for given file and compares/fixes it in the file.
- DB contains aggregation datasets and used by automatic tool for configuring THREDDS Aggregation Server.
- Curator DB is a backend DB for GFDL LAS (LASurator)
**And perspective usage – FRE backend DB and metadata/configdata resource in modeling process workflow**.

## What we have now

- Designed Curator DB intended not only for model configuring data but also for metadata describing models.
- Automatic tool FRErator mapping and writing rts.xml into Curator DB. It’s highly configurable to handle wide range of xml schemas. It will be invoked (by user’s desire) from FRE tools. (See life demo)
- Pilot version of web _[interface](http://nomads.gfdl.noaa.gov:8080/modelDB/)__[ ](http://nomads.gfdl.noaa.gov:8080/modelDB/)__for viewing model metadata/configdata from DB._

## FRErator is easy configurable tool working with arbitrary xml schema.
Configuration file rts.xsm (xml schema mapping) has xml schema syntax expanded with mapping attributes: map, mapGroup, mapPrimKey, mapFKey, mapGlbFld, mapRule, Group, mapGroup.
The fragment of rts.xsm:
_____________________________________________________________________________________________________________________________________________________________________________________________________________________________

	 <element name="parameterGroup">
<complexType mixed="true">
                 <all Group="parameter" minOccurs="0">
                        <element ref="parameter“           
                          mapGroup="ParamsGroups.param_group_content_id|ParamGroup_Contents.param_group_content_id"/> 
                 </all>
                 <attribute name="name" map="ParamsGroups.param_group_name"   
                         mapPrimKey="ParamGroup_Contents.param_group_content_id, ParamsGroups.param_group_id"
		mapRule="RANDOM_ID" /> 
                 <text map="ControlledParams.param_prog_name, ParamValues.param_value"
                                   mapPrimKey="ControlledParams.param_id, ParamValues.param_value_id"
                          mapFKey="ParamGroup_Contents.param_id|ControlledParams, ParamGroup_Contents.param_value_id|ParamValues"
                          mapRule="SEMICOLON_SPLIT,EQUAL_SPLIT,RANDOM_ID,GROUP" /> 
             </complexType>
        </element>

        <element name="parameter" maxOccurs="unbounded">
             <complexType mixed="true">
                 <attribute name="name" map="ControlledParams.param_prog_name"        
                                 mapPrimKey="ControlledParams.param_id" 
                                 mapFKey="ParamGroup_Contents.param_id" mapRule="RANDOM_ID" /> 
                 <text map="ParamValues.param_value" mapPrimKey="ParamValues.param_value_id" 
                          mapFKey="ParamGroup_Contents.param_value_id" 
                          mapRule="RANDOM_ID" /> 
             </complexType>
       </element>
__________________________________________________________________________________________________

- FRErator is easy configurable tool working with arbitrary xml schema.
- Configuration file rts.xsm (xml schema mapping) has xml schema syntax expanded with mapping attributes: *map, mapGroup, mapPrimKey, mapFKey, mapGlbFld, mapRule, Group, mapGroup**.*
- The fragment of rts.xsm:
- _____________________________________________________________________________________________________________________________________________________________________________________________________________________________
- 	 <element name="**parameterGroup**">
- <complexType mixed="true">
-                  <all *Group*="**parameter**" minOccurs="0">
**         **               <element ref="**parameter**“
-                           *mapGroup*="**ParamsGroups.param_group_content_id|ParamGroup_Contents.param_group_content_id**"/>
-                  </all>
-                  <attribute name="**name**" *map*="**ParamsGroups.param_group_name**"
-                          *mapPrimKey*="**ParamGroup_Contents.param_group_content_id, ParamsGroups.param_group_id**"
- 		*mapRule*="RANDOM_ID" />
-                  <text *map*="**ControlledParams.param_prog_name, ParamValues.param_value**"
-                                    *mapPrimKey*="**ControlledParams.param_id, ParamValues.param_value_id**"
-                           *mapFKey*="**ParamGroup_Contents.param_id|ControlledParams, ParamGroup_Contents.param_value_id|ParamValues**"
-                           *mapRule*="SEMICOLON_SPLIT,EQUAL_SPLIT,RANDOM_ID,GROUP" />
**             **</complexType>
**       ** </element>
-         <element name="**parameter**" maxOccurs="unbounded">
-              <complexType mixed="true">
-                  <attribute name="**name**" *map*="**ControlledParams.param_prog_name**"
-                                  *mapPrimKey*="**ControlledParams.param_id**"
-                                  *mapFKey*="**ParamGroup_Contents.param_id**" *mapRule*="RANDOM_ID" />
-                  <text *map*="**ParamValues.param_value**" *mapPrimKey*="**ParamValues.param_value_id**"
-                           *mapFKey*="**ParamGroup_Contents.param_value_id**"
-                           *mapRule*="RANDOM_ID" />
-              </complexType>
-        </element>
- __________________________________________________________________________________________________

- FRErator design

## Will map the such tags: 

<parameterGroup name="diag_manager_nml"> 
	max_input_fields = 400; 
	max_output_fields = 500; 
	mix_snapshot_average_fields=.true. 
    ....
</parameterGroup>

<parameterGroup name="aerosolrad_package_nml">
      <parameter name="volcanic_dataset_entry">1, 1, 1, 0, 0</parameter>
      <parameter name="using_volcanic_lw_files">.false.</parameter>
      <parameter name="lw_ext_filename">"bogus"</parameter>
      <parameter name="lw_ext_root">" bogus"</parameter>
	 <parameter name="lw_asy_filename">" bogus"</parameter>
	 <parameter name="lw_asy_root">"bogus "</parameter> 
	 <parameter name="lw_ssa_filename">" bogus"</parameter>
	 <parameter name="lw_ssa_root">"bogus "</parameter>
     .....
</parameterGroup>

- <parameterGroup name="diag_manager_nml">
- 	max_input_fields = 400;
- 	max_output_fields = 500;
- 	mix_snapshot_average_fields=.true.
-     ....
- </parameterGroup>
- <parameterGroup name="aerosolrad_package_nml">
-       <parameter name="volcanic_dataset_entry">1, 1, 1, 0, 0</parameter>
-       <parameter name="using_volcanic_lw_files">.false.</parameter>
-       <parameter name="lw_ext_filename">"bogus"</parameter>
-       <parameter name="lw_ext_root">" bogus"</parameter>
- 	 <parameter name="lw_asy_filename">" bogus"</parameter>
- 	 <parameter name="lw_asy_root">"bogus "</parameter>
- 	 <parameter name="lw_ssa_filename">" bogus"</parameter>
- 	 <parameter name="lw_ssa_root">"bogus "</parameter>
-      .....
- </parameterGroup>

## What we need else

- Automatic Tool generating canonical rts.xml from DB
- Editing GUI allowing user to select needed preexisting components and configurations from DB, edit them on a fly write into DB and generate new rts.xml.