**Latest CCR Affecting this Ticket:** (for database administrator use
only)\
**Latest CCR DATE Affecting this Ticket:** (for database administrator
use only)

Ticket: BE_S3_05 (With L4 Mappings)

EDG Access to Data Pools

+-------------------------------+----+---------------------------------+
| **Launch Criticality:**       |    | **Ticket CCR Number^\*^:** (for |
| Synergy III                   |    | database administrator use      |
|                               |    | only**)**                       |
| **Review:** (subsystems       |    |                                 |
| affected) DMS (V0GTWY), DPL,  |    | **Ticket CCR Date^\*^:** (for   |
| CSS                           |    | database administrator use      |
|                               |    | only)                           |
| **Priority:** TBD             |    |                                 |
|                               |    | \* Represents changes to        |
| **Number of Tracked Ticket    |    | information from Ticket table   |
| Changes:** (for database      |    | only. Does not include          |
| administrator use only)       |    | information linked in from      |
|                               |    | other tables.                   |
+-------------------------------+----+---------------------------------+

  -----------------------------------------------------------------------
  NOTE: The number of tracked changes (above) represents the number of
  changes to this particular Ticket. Whenever the data appearing in this
  Ticket changes this number is incremented by 1.

  -----------------------------------------------------------------------

**External Interface Dependency:**

EDG:

This ticket defines capabilities in V0GTWY to extend its inventory
search capability to include in its result to the EDG the ftp URL of
granules if such granules are present in the local Data Pool.

It will be necessary to update the ICD between the EDG and V0GTWY
(505-41-30 ICD) in order to capture changes in the ODL message returned
to the EDG by the V0GTWY (Inventory Search Result).

**Subsystem Dependency:**

DPL, CSS (Registry will store parameters to connect to DPL db).

**Preconditions:**

The Data Pool must include at least 1,000 granules covering 10 different
ESDT and 5 different data collection groups. The granules must include
large data files with file sizes of several megabytes. Note: The need
for larger numbers of granules, ESDTs and collection groups is only
important in the PVC where performance testing is conducted. The exact
number of granules, ESDTs and collection groups should be derived from
the test criteria.

**Differences from Previous Releases:**

Expansion of the Inventory Search Result to include an optional group
with ftp URL and expiration date for granules that are located in a Data
Pool The Gateway CI will conduct a search within the Data Pool database
for each granule returned from SDSRV CI in an inventory search result.
That search will attempt to return URLs and expiration of URL
information for each granule retrieved. Results of these searches will
be rendered into an additional element within the inventory search
result on a granule by granule basis.

**Operations Concept:**

Most Synergy users interface with ECS using EDG and are likely to
continue to do so. The EDG provides a graphical interface that allows
users to search ECS holdings based on different search criteria.

The Data Pool is a large online cache of ECS data. Data is inserted into
the Data Pool via Spatial Subscription Server subscriptions placed on
ECS insert events. Configuration settings in the Data Pool database are
used to control which data types are eligible for insertion into the
Data Pool. Data Pool insert subscriptions also specify the length of
time that data will reside in the Data Pool before it expires and is
eligible for cleanup. At the time of insertion, Data Pool creates an
associated metadata file. Thus, the Data Pool, will contain (one or
more) science granules, metadata files and possibly zero or more browse
image files, all of which are cross referenced via the granule id in the
Data Pool database. A single browse image can be associated with
multiple granules.

When TBD EDG releases are available, this capability will allow users to
query ECS Data Pool holdings and open a direct connection and download
those products.

Until that time, there will be no change in ECS operations.

**Operational Impact:**

No ECS operational impact other than increased time in Inventory Search
due to the interface with Data Pool. There could also be potential
degradation of DPL database performance due to additional searches.

**Transition Approach:**

Treated as a patch install done during regular DAAC maintenance periods.
Though too early in the software life cycle to know for sure, it is
reasonable to expect that there will also be a database patch for the
DPL db (generated by development of this capability or other in Synergy
III) and a configuration patch.

**Comments:**

  ---------- ---------------- -------------------------------- -----------
                                                               

                                                               
  ---------- ---------------- -------------------------------- -----------

**Level 3 Requirement(s):**

+---------+-------------+--------------+--------+--------+----------+
| **L3    | **L3 Text** | **Cla        | **Cate | **Rel  | **CCR    |
| ID**    |             | rification** | gory** | ease** | Num**    |
+---------+-------------+--------------+--------+--------+----------+
| D       | The ECS     |              | S      | SY2    | 02-0979  |
| ADS2676 | shall       |              | ynergy | SY3    |          |
|         | provide an  |              |        |        |          |
|         | on-line     |              |        |        |          |
|         | cache of    |              |        |        |          |
|         | data        |              |        |        |          |
|         | products    |              |        |        |          |
|         | that are    |              |        |        |          |
|         | accessible  |              |        |        |          |
|         | by end      |              |        |        |          |
|         | users via   |              |        |        |          |
|         | the Web and |              |        |        |          |
|         | FTP. The    |              |        |        |          |
|         | minimal     |              |        |        |          |
|         | capacity of |              |        |        |          |
|         | these       |              |        |        |          |
|         | caches      |              |        |        |          |
|         | shall be as |              |        |        |          |
|         | follows:    |              |        |        |          |
|         |             |              |        |        |          |
|         | -   EDC     |              |        |        |          |
|         |     DAAC:   |              |        |        |          |
|         |     40      |              |        |        |          |
|         |             |              |        |        |          |
|         |   Terabytes |              |        |        |          |
|         |             |              |        |        |          |
|         | -   GSFC    |              |        |        |          |
|         |     DAAC:   |              |        |        |          |
|         |     60      |              |        |        |          |
|         |             |              |        |        |          |
|         |   Terabytes |              |        |        |          |
|         |             |              |        |        |          |
|         | -   LaRC    |              |        |        |          |
|         |     DAAC:   |              |        |        |          |
|         |     20      |              |        |        |          |
|         |             |              |        |        |          |
|         |   Terabytes |              |        |        |          |
|         |             |              |        |        |          |
|         | ```{=html}  |              |        |        |          |
|         | <!-- -->    |              |        |        |          |
|         | ```         |              |        |        |          |
|         | -   NSIDC   |              |        |        |          |
|         |     DAAC: 1 |              |        |        |          |
|         |             |              |        |        |          |
|         |    Terabyte |              |        |        |          |
+---------+-------------+--------------+--------+--------+----------+

**IRD Requirement(s):**\
None

  --------- ------------------------------------------------------- -----
                                                                    

                                                                    
  --------- ------------------------------------------------------- -----

**Level 4 Requirement(s):**

+------------+-------------------------------------+---------+--------+
| **L4 ID**  | **L4 Text**                         | **Re    | **CCR  |
|            |                                     | lease** | Num**  |
+------------+-------------------------------------+---------+--------+
| S          | The GTWAY CI Inventory Search       | SY3     |        |
| -DMS-50010 | Result shall include an optional    |         |        |
|            | group containing the URL location   |         |        |
|            | for science granules residing in    |         |        |
|            | the Data Pool.                      |         |        |
+------------+-------------------------------------+---------+--------+
| S          | The GTWAY CI shall have the ability | SY3     |        |
| -DMS-50020 | to provide the URL location of      |         |        |
|            | metadata files associated with any  |         |        |
|            | granules in an inventory search     |         |        |
|            | result that reside in the Data      |         |        |
|            | Pool.                               |         |        |
+------------+-------------------------------------+---------+--------+
| S          | The GTWAY CI shall have the ability | SY3     |        |
| -DMS-50025 | to provide all necessary URL        |         |        |
|            | location information to handle      |         |        |
|            | multifile granules seamlessly.      |         |        |
+------------+-------------------------------------+---------+--------+
| S          | The GTWAY CI shall have the ability | SY3     |        |
| -DMS-50030 | to provide the URL location of any  |         |        |
|            | existing browse images that reside  |         |        |
|            | in the Data Pool that are           |         |        |
|            | associated with granules in an      |         |        |
|            | inventory search result.            |         |        |
+------------+-------------------------------------+---------+--------+
| S          | The GTWAY CI Inventory Search       | SY3     |        |
| -DMS-50040 | Result optional group shall contain |         |        |
|            | the expiration date of the science  |         |        |
|            | granules located in a Data Pool.    |         |        |
|            |                                     |         |        |
|            | Note: In this context, the          |         |        |
|            | expiration date refers as to the    |         |        |
|            | time at which the granule (and its  |         |        |
|            | associated files) should be removed |         |        |
|            | from the Data Pool.                 |         |        |
+------------+-------------------------------------+---------+--------+
| S          | The GTWAY CI Inventory Search       | SY3     |        |
| -DMS-50050 | Result optional group shall not     |         |        |
|            | include Data Pool granule URL       |         |        |
|            | locations for products whose        |         |        |
|            | expiration date is in the past.     |         |        |
+------------+-------------------------------------+---------+--------+
| S          | The GTWAY CI shall not fail or halt | SY3     |        |
| -DMS-50070 | the inventory search result if      |         |        |
|            | unable to obtain the URL location   |         |        |
|            | for either                          |         |        |
|            |                                     |         |        |
|            | a)  Science granule file(s)         |         |        |
|            |                                     |         |        |
|            | b)  metadata file                   |         |        |
|            |                                     |         |        |
|            | c)  browse image (if available)     |         |        |
|            |                                     |         |        |
|            | or the expiration time for a        |         |        |
|            | particular granule in the inventory |         |        |
|            | search.                             |         |        |
+------------+-------------------------------------+---------+--------+
| S          | The GTWAY CI shall log an error if  | SY3     |        |
| -DMS-50080 | an incomplete set of results is     |         |        |
|            | returned from DPL.                  |         |        |
|            |                                     |         |        |
|            | A complete set of result consists   |         |        |
|            | of                                  |         |        |
|            |                                     |         |        |
|            | a\) metadata file (one)             |         |        |
|            |                                     |         |        |
|            | b)  Zero or more science granule    |         |        |
|            |     data files                      |         |        |
|            |                                     |         |        |
|            | c)  Zero or more browse images      |         |        |
|            |                                     |         |        |
|            | d\) One expiration date             |         |        |
+------------+-------------------------------------+---------+--------+
| S          | The GTWAY CI shall not degrade its  | SY3     |        |
| -DMS-50090 | performance in Inventory Search     |         |        |
|            | Result generation by more than 10%. |         |        |
+------------+-------------------------------------+---------+--------+
| S          | The GTWAY CI shall be able to turn  | SY3     |        |
| -DMS-50100 | on and off inventory searches into  |         |        |
|            | the data pool database.             |         |        |
|            |                                     |         |        |
|            | Note: This is to prevent            |         |        |
|            | unnecessary performance degradation |         |        |
|            | in DAACs that do not have a data    |         |        |
|            | pool.                               |         |        |
+------------+-------------------------------------+---------+--------+
| S          | The GTWAY CI shall format the       | SY3     |        |
| -DMS-50110 | expiration date for a data pool URL |         |        |
|            | in universal time code (UTC).       |         |        |
+------------+-------------------------------------+---------+--------+
| S          | The GTWAY CI shall indicate when    | SY3     |        |
| -DMS-50120 | the Data Pool Inventory is offline  |         |        |
|            | in its inventory result to the EDG. |         |        |
+------------+-------------------------------------+---------+--------+

**L4 to L3 Mappings:**

  ----------------------------- --------------------- -------------------
  **L4 ID**                     **L3 ID**             **CCR Num**

  S-DMS-50010                   DADS2676              

  S-DMS-50020                   DADS2676              

  S-DMS-50025                   DADS2676              

  S-DMS-50030                   DADS2676              

  S-DMS-50040                   DADS2676              

  S-DMS-50050                   DADS2676              

  S-DMS-50070                   DADS2676              

  S-DMS-50080                   DADS2676              

  S-DMS-50090                   DADS2676              

  S-DMS-50100                   DADS2676              

  S-DMS-50110                   DADS2676              

  S-DMS-50120                   DADS2676              
  ----------------------------- --------------------- -------------------

**L4 to IRD Mappings:**\
None

  ------------------------------ -------------------- --------------------
                                                      

                                                      

                                                      
  ------------------------------ -------------------- --------------------

**Design Directive(s):**

Design must allow for backwards compatibility with the EDG.

Another design driver is to isolate as much as possible V0GTWY changes
from other Synergy III development. V0GTWY architect must carefully
coordinate design for all Synergy III capabilities that include V0GTWY
changes.

Finally, design should avoid excessive db queries that will negatively
impact system performance.

**Criteria:**

+----------+------+---------------------------------+--------+--------+
| **       | **   | **Criteria Text**               | **     | **CCR  |
| Criteria | Crit |                                 | Type** | Num**  |
| Key**    | eria |                                 |        |        |
|          | ID** |                                 |        |        |
+----------+------+---------------------------------+--------+--------+
|          | 10   | Via the EDG client or a test    | FC     |        |
|          |      | driver simulating V0 protocols, |        |        |
|          |      | submit an inventory search      |        |        |
|          |      | containing a mixture of science |        |        |
|          |      | data types, all of which will   |        |        |
|          |      | find a match in the archive and |        |        |
|          |      | in the Data Pool.               |        |        |
|          |      |                                 |        |        |
|          |      | Visually inspect the Inventory  |        |        |
|          |      | Search Result to verify that    |        |        |
|          |      | the information for each        |        |        |
|          |      | granule found in a Data Pool,   |        |        |
|          |      | contains the ftp URL for the    |        |        |
|          |      | granule and all of the          |        |        |
|          |      | associated files (metadata and  |        |        |
|          |      | browse if available).           |        |        |
|          |      |                                 |        |        |
|          |      | If using a test driver start a  |        |        |
|          |      | web browser and from the        |        |        |
|          |      | Inventory Search Result, cut    |        |        |
|          |      | and paste into the browser one  |        |        |
|          |      | at the time, the ftp URLs found |        |        |
|          |      | in the Inventory Search Result. |        |        |
|          |      |                                 |        |        |
|          |      | If using the EDG, select the    |        |        |
|          |      | displayed URLs in the Inventory |        |        |
|          |      | Search Result.                  |        |        |
|          |      |                                 |        |        |
|          |      | Verify that each product        |        |        |
|          |      | downloads correctly.            |        |        |
+----------+------+---------------------------------+--------+--------+
|          | 20   | Via the EDG client or a test    | FC     |        |
|          |      | driver simulating V0 protocols, |        |        |
|          |      | submit an inventory search that |        |        |
|          |      | will match a group of granules  |        |        |
|          |      | in the Data Pool composed as    |        |        |
|          |      | follows:                        |        |        |
|          |      |                                 |        |        |
|          |      | 1.  One match that has a        |        |        |
|          |      |     granule file, metadata file |        |        |
|          |      |     and browse file.            |        |        |
|          |      |                                 |        |        |
|          |      | 2.  Another match that has a    |        |        |
|          |      |     granule file, metadata      |        |        |
|          |      |     file, but no browse file    |        |        |
|          |      |                                 |        |        |
|          |      | 3.  Another match that has      |        |        |
|          |      |     metadata file, browse, but  |        |        |
|          |      |     no science data file.       |        |        |
|          |      |                                 |        |        |
|          |      | 4.  Another match that has only |        |        |
|          |      |     a metadata file.            |        |        |
|          |      |                                 |        |        |
|          |      | Inspect visually the generated  |        |        |
|          |      | Inventory Search Result to see  |        |        |
|          |      | entries for granules 1 through  |        |        |
|          |      | 3 and their corresponding       |        |        |
|          |      | associated files.               |        |        |
|          |      |                                 |        |        |
|          |      | If using a test driver, cut and |        |        |
|          |      | paste into a browser all ftp    |        |        |
|          |      | URLs and verify that they       |        |        |
|          |      | download correctly.             |        |        |
|          |      |                                 |        |        |
|          |      | If using the EDG, select the    |        |        |
|          |      | displayed URLs in the Inventory |        |        |
|          |      | Search Result and verify that   |        |        |
|          |      | they download correctly.        |        |        |
+----------+------+---------------------------------+--------+--------+
|          | 30   | Via the EDG client or a test    | FC     |        |
|          |      | driver simulating V0 protocols, |        |        |
|          |      | submit an inventory search      |        |        |
|          |      | containing criteria that will   |        |        |
|          |      | match a multifile granule.      |        |        |
|          |      |                                 |        |        |
|          |      | If using a test driver, cut and |        |        |
|          |      | paste the found link in the     |        |        |
|          |      | Inventory Search Result and     |        |        |
|          |      | verify that all the files that  |        |        |
|          |      | make up the granule are         |        |        |
|          |      | downloaded correctly.           |        |        |
|          |      |                                 |        |        |
|          |      | If using the EDG, select the    |        |        |
|          |      | displayed URLs in the Inventory |        |        |
|          |      | Search Result and verify that   |        |        |
|          |      | they download correctly.        |        |        |
+----------+------+---------------------------------+--------+--------+
|          | 50   | Via the EDG client or a test    | EC     |        |
|          |      | driver simulating V0 protocols, |        |        |
|          |      | submit an inventory search      |        |        |
|          |      | containing criteria that will   |        |        |
|          |      | match a product without a       |        |        |
|          |      | metadata file entry in the DPL  |        |        |
|          |      | database (remove it manually).  |        |        |
|          |      |                                 |        |        |
|          |      | Verify that Inventory Search    |        |        |
|          |      | does not halt.                  |        |        |
|          |      |                                 |        |        |
|          |      | Verify that the log has an      |        |        |
|          |      | entry noting the metadata file  |        |        |
|          |      | unavailability.                 |        |        |
+----------+------+---------------------------------+--------+--------+
|          | 70   | Make the Data Pool database     | EC     |        |
|          |      | inaccessible.                   |        |        |
|          |      |                                 |        |        |
|          |      | Via the EDG client or a test    |        |        |
|          |      | driver simulating EDG           |        |        |
|          |      | protocols, submit an inventory  |        |        |
|          |      | search containing criterion     |        |        |
|          |      | that will match a granule in    |        |        |
|          |      | the Data Pool.                  |        |        |
|          |      |                                 |        |        |
|          |      | Verify that the Inventory       |        |        |
|          |      | Search Result generation does   |        |        |
|          |      | not halt, but logs an error for |        |        |
|          |      | the Data Pool search.           |        |        |
|          |      |                                 |        |        |
|          |      | Verify that the V0 GWY sends    |        |        |
|          |      | sufficient status information   |        |        |
|          |      | to the EDG, in order for the    |        |        |
|          |      | EDG to display a message        |        |        |
|          |      | indicating that the search of   |        |        |
|          |      | the cache inventory could not   |        |        |
|          |      | be completed.                   |        |        |
+----------+------+---------------------------------+--------+--------+
|          | 80   | Verify that the V0GTY is        | PC, FC |        |
|          |      | configured such that inventory  |        |        |
|          |      | searches into the data pool     |        |        |
|          |      | inventory are enabled. Via the  |        |        |
|          |      | EDG client or a test driver     |        |        |
|          |      | simulating the V0 Protcols,     |        |        |
|          |      | submit an inventory search      |        |        |
|          |      | containing criteria that will   |        |        |
|          |      | return 20 granules from 2       |        |        |
|          |      | different collections. All of   |        |        |
|          |      | these granules should be        |        |        |
|          |      | available in the data pool      |        |        |
|          |      | inventory for access. Note the  |        |        |
|          |      | elapsed time from the point at  |        |        |
|          |      | which the inventory search      |        |        |
|          |      | request is submitted to the     |        |        |
|          |      | V0GTY to the point that the     |        |        |
|          |      | inventory search result is      |        |        |
|          |      | returned from the V0GTY. Verify |        |        |
|          |      | that all expected granules are  |        |        |
|          |      | returned in the inventory       |        |        |
|          |      | search result.                  |        |        |
|          |      |                                 |        |        |
|          |      | Disable the V0GTY searches of   |        |        |
|          |      | the data pool inventory.        |        |        |
|          |      | Reissue the same search that    |        |        |
|          |      | was issued before. Note again   |        |        |
|          |      | the time required to receive    |        |        |
|          |      | the inventory search result.    |        |        |
|          |      | Verify that the time required   |        |        |
|          |      | to complete the first search is |        |        |
|          |      | not more than 10% greater than  |        |        |
|          |      | the time to complete the second |        |        |
|          |      | search. Verify that all         |        |        |
|          |      | expected granules are returned  |        |        |
|          |      | in the inventory search result. |        |        |
+----------+------+---------------------------------+--------+--------+