# Presentation: 232722

## FIS Metadata Catalog 

- FIS Metadata Catalog

- Karen L. Sender, Doug Turnbull,
- Jan Pappas, Tom Kagehiro, Joseph Dane, Pauli Salmu, Tina Chang
- NOAA Fisheries Service, Pacific Islands Fisheries Science Center
- NOAA Tech 2006
- 3 Nov 2005

**Notes:**
The National Fisheries Information System is a Virtual System.  Its mission is to provide summary fishery dependent statistics to stakeholders in management planning, trade analysis, industry – economics and other social sciences, congress, academe, and a myriad of other known and unknown users.

To realize that promise it is mandatory that an understanding and management of data be complete and robust.

InPort is a big piece of the toolset developed or in development under the FIS umbrella to accomplish this. 

InPort is more than a simple data catalog. 

We have been on a mission to educate our data community on the full scope of data management – what you so appropriately call end-to-end data management.

As we like to drive home the point.... (change slide)

## Slide 2

**Notes:**
...data management is not just data archiving.

Actually, InPort was born from our recognition of the complexity of data management that was the original impetus for the design of InPort.  Trying to find the details of the many parts of our data systems was all too often an exercise is frustration, detective work, and, too often, misinformation.

## What is InPort?

**Notes:**
The InPort Metadata Catalog has been identified as a foundational tool to support FIS goals...

InPort – A Tool forUnderstanding Our Data

Scientists and managers can search and retrieve catalog information to answer...

What data do we have (species, measurements, etc.)?
Who has the data?  How to contact them?
When were the data collected?
Where and how were the data collected?
How was the data processed?
Where do we have data gaps?

## Data Catalog Benefits

- Searchable, national catalog of fisheries data
- Available to scientists and managers
- Supports analysis and re-engineering of data collection programs
- Paves the way for integrating data sets
- Promotes best practices
- Facilitate the sharing of data and information

**Notes:**
Gives NOAA Fisheries a searchable, national catalog of fisheries data
Available to scientists and managers from all regions
Supports analysis and re-engineering of data collection programs
Paves the way for integrating data sets 
Promotes best practices in Fisheries data management

And, of great importance, facilitate the sharing of data and information

Now I would like to focus on a few of the major capabilities of InPort <change slide>

## InPort Capabilities

- Inventory of Fisheries data holdings
- Detailed documentation of Fisheries data
- Provide metadata for data integration analysis and planning
- Share metadata

**Notes:**

For our FIS projects, I would like to list the kinds of information objects that we feel we need to catalog

...the level of detail that we are capable of maintaining

...how this would support data integration and standardization

And

...how we intend to interface InPort to other metadata systems

## Data Inventory to Support FIS

- Projects, programs, data collections
- Databases, data sets, spreadsheets, files
- Tables and Columns
- Documents
- Reports
- Statistical Models
- Data Processing flow

**Notes:**
Remember, this is what InPort stores.  
  More than a data dictionary
  More than a document archive
  More than a design repository

## Metadata Analysis

- Create and promote data standards
- Design data integration schemes
- Share tools, expertise, technologies
- Promote data reuse
- Identify data gaps and redundancies

**Notes:**
Identify opportunities to

Create data standards by analyzing commonalities in data collections and methods.

Have the detailed information required to develop data translation tools for data integration and aggregation

Save time and money by identifying what tools are already developed.  Recognize what technologies are being used in the data community and who might have knowledge or expertise that that could be shared.

Having a searchable catalog of data sets would promote the re-use of existing data sets.  This, of course, also requires complete documentation of those data sets.

And, of course, it is very important to be able to recognize where we have data gaps and where we might find opportunities to eliminate redundant data collections.

## Sharing metadata

**Notes:**
While metadata can be searched and accessed via the InPort web site, xml exports from the InPort repository can feed into other metadata servers...

Another FIS project is developing a separate web front end to serve as a virtual library of their project’s information, tools, standards, and best practices.

Other special purpose user interfaces can be developed and all of these merged into the FIS Data Portal

NBILL  = National Biological Information Infrastructure
CoRIS = Coral Reef Information System 
FGDC  = Federal Geographic Data Committee 

## InPort Features

- Create and manage InPort system users and access roles
- Create, edit, manage hierarchical metadata “libraries”
- Import from data dictionaries
- Establish relationships between catalog items
- Publish metadata (public vs. restricted)
- Search metadata
- Export metadata

**Notes:**
“Simple to use, complex enough to capture detailed documentation”

<Features stuff>

## Catalog Hierarchy

- Example Data System Hierarchy

**Notes:**
InPort follows the Library catalog metaphor

Information is organized in a Hierarchical manner.  Each contributing organization has it’s Metadata Library under which it creates metadata folders (called projects) that can contain sub-folders or catalog items.

In this example, our PIFSC Metadata Library contains a data collection program where we are documenting the many parts of one of its data systems.

This data system shows
             a document,
             a database with table and columns,
             a data collection item with its procedure
             a folder containing the data system’s data collection forms
   and    a folder documenting the data system’s web pages

## Catalog Workflow

**Notes:**
Catalog Control States for Editing, Publishing, Withdrawing, and Revising Metadata

The publication control state helps determine what operations may be performed on the catalog item. For example, the “Published Externally” state means that users (NOAA & public) can search for and view the catalog item. The “Draft” state means that the catalog item is viewable only by users who have been granted access to view or edit that catalog item.

An important concern with serving information is its currency.  The InPort system sets review dates at publication (currently one year from publication).  If the review date passes, the metadata is considered stale and is withdrawn to the Review/Revise state and must be re-approved and published to be made publicly available again.

## Query and Structure

## Searching InPort

- You can search published catalog items.
- InPort lists search results, summaries, and details.

**Notes:**
InPort will publish a catalog of fisheries data. The catalog contains information needed to use the fisheries data. Examples: how will we link logbook data? Do we have enough data to support our statistical analysis? Are we missing any data needed to monitor a habitat?

InPort will support necessary changes to our approach to data management. Fisheries management is changing and our information management approach must change, too.

## Search results

- Listing of catalog items with “library” in title, abstract, or description

## Summary of a Library

- Each NOAA Fisheries Service Partner may have their own library or libraries.

## Search: observer longline

- You can search with more specific terms.
- Example: observer longline

## Search results

- Catalog items matching “observer” and “longline”

## Item Summary: project

- Each catalog item will have a title, abstract, and other descriptive information

## Details about a project

- InPort lists details about the project: area, procedures, faqs
- This project includes a data set and some documentation

## Details about a data set

- Abstract about the LODS data set or database

## Typical fishery data

- Much of fisheries data is time-series information stored in tables or files.
- The tables record details about activities relevant to fishery performance.

## Fisheries table – A set

- Example: A set record describes activity that starts with putting fishing lines in the water and ends with pulling them out.

## Details -- table columns

- InPort stores table columns (attributes)

## Details about columns

- InPort stores details about each column or attribute

## Lists, summaries, details

- You can search all published catalog items.
- InPort shows search results, summaries and details.

## Technical Environment

**Notes:**
InPort should run within the FIS architecture.
-- We’ll use FIS security policies, user access policies.
-- We’ll interface to other FIS systems with the FIS architecture to the extent that it supports these things.
For access outside of the FIS system, we’ll use a public web-site and internet protocols (e.g., http or ftp access to client sites)
Technical environment: Unix, App Server 10g, Oracle9i (9.2.0.4)
Not shown: firewalls and load balancers

## FIS Architecture