# Presentation: 196171

## Slide 1

## CCT Outline

**Goals of Exercise**
**Test Strategy**
**Evaluating Group**
**Instructions to Participants**
**Evaluation Form**
**Results**
**Evaluators Comments**
**Response**
**Summary**
**Backup **

## Goal of Exercise

- Develop a tool to obtain realistic evaluations of the PDS data query/access/delivery tool set
- Identify a limited group of effective  evaluators
- Summarize their responses and identify needed modifications of the evaluation  tool and in the PDS data query/access/delivery tool set
- Identify future opportunities for effective application of tool

## Test Strategy

- We designed 9 data user scenarios that addressed current problems and spanned a range of planetary science disciplines.
- The scenarios were designed to motivate realistic data searches and allow participants to work in their field of specialization.
- In most scenarios, a motivating factor was characterizing a future mission where the user would benefit from knowledge of both the extent and the science content of prior data.
- Examples of the scenarios follow.

## Topic 1

**TOPIC**:  Icy Satellites (Developer: Reta Beebe, Atmospheres Node)
**SCENARIO**: The Jupiter Icy Moons Orbiter (JIMO) mission will orbit Ganymede and Europa, utilizing an ion drive engine to migrate from one moon to the next, with the specific purpose of expanding our understanding of the surface and subsurface characteristics of these icy moons.
**TASK**: Locate useful (you define this) Europa data from past exploration missions to motivate defining goals and planning observations and sequences for a JIMO-type mission at Europa.

## Topic 2

**TOPIC**:  Venus Atmosphere- (Developer: Nancy Chanover, Atmospheres Node)
**SCENARIO**: You want to develop a high spectral resolution near-IR imager to be flown on a Venus orbiter (e.g. Venus Express or an as-yet-unnamed Discovery or New Frontiers mission).  One of the science objectives of such an instrument is to characterize the wind field in the lowest portion of Venus' thick cloud deck.  The UV imaging data is sensitive to winds near 100 km altitude, but to probe the lower cloud deck (~ 50-70 km altitude), you need to use near-IR imaging of the night side.  Low-opacity regions of the clouds are manifested as bright spots through which Venus' hot surface thermal radiation escapes.  By tracking these "hot spots" over time, one can determine the zonal and meridional wind fields. Prior to developing plans for the spacecraft's orbits, you would want to know what has already been done in terms of near-IR space-based imaging of Venus.
**TASK**: Determine the spectral, spatial, and temporal coverage that previous spacecraft-borne near-IR instruments have provided at Venus

## Topic  5

**TOPIC**:  Comparison of Mars Landing Site Characteristics (Developer: Ray Arvidson and Tom Stein, Geosciences node)
**SCENARIO**: Five Martian landing sites have been explored by US missions. The high resolution detail at these landing sites provides data for a number of geologic studies among the sites.
**TASK**: Identify, locate, and retrieve lander or rover imaging and other data that allow for a comparative study of aeolian features at the landing sites, and also rock abundance characteristic (which could be correlated with orbiter data of the same regions).

## Topic  7

**TOPIC**:  Asteroid Characterization (Developer: Reta Beebe, Atmospheres Node and Mark Sykes, Univ. Arizona)
**SCENARIO**: You are to be involved with the DAWN mission to explore the asteroids Ceres and Vesta.  You want to gather all the available data about these and other asteroids in order to determine the extent of the available coverage [wavelength, phase angle, etc.]
**TASK**: Locate and obtain all PDS data pertaining to asteroids, specifically Ceres and Vesta; determine the overlap in wavelength coverage available as well as wavelength gaps that apply to past observations of these bodies

## Topic  9

**TOPIC**:  Fields and Particles – (Developers: Reta Beebe, Atmospheres and Ray Walker, PPI)
**SCENARIO**: You are going to propose an instrument to JIMO. You need to know the particle and field environment near Europa, which is the icy moon closest to Jupiter and in the harshest radiation environment. You will need to know the energetic particle environment that your instrument will encounter so you can design one to survive in this hostile environment.
**TASK**: Locate data in Europa’s circum-Jovian environment (8 RJ to 12 RJ) from the energetic particle experiments and the magnetometer on Galileo. Someone might also be interested in a range of local times, rather than radii (Suggested by Frank Crary).

## Evaluating Group

- We deliberately selected young investigators and graduate students whose planetary careers had focused in large part on a single mission.
- Computer literate
- Critical and informed
- Little previous experience with the PDS
- Originally we had identified a larger group of initial participants but delays of release of the test eliminated use of classes where professors had agreed to use the test as a classroom exercise.
- This assessment included 22 contributors who seriously addressed the tasks and provided excellent comments.

## Instruction to Participants

- Select a topic and review the evaluation sheet.
- log on to the PDS Central Node web site: http://pds.jpl.nasa.gov
- Initiate your effort to locate data relevant to your selected **SCENARIO** and **TASK, **using the tools available from the Central Node’s web page
- As you proceed through the data query system, keep track of how well the system performs for you as you dig in to the archive to locate and acquire the data that are of interest to your **SCENARIO** and **TASK**
- When you have identified and located the data set or data sets of interest, retrieve the data. If you become stymied in your data search activity with the Central Node tools, go to the web site of the Discipline Node that seems most logical for the data you seek.

## Instruction to Participants (cont)

- Look through any data retrieved; determine if the ancillary files that accompany the data provide enough information for you to determine if the ways in which you would envision working with the data (creating maps; isolating or extracting specific locations and/or times [time of day, season, etc.] ) are doable with the data you have obtained
- Using the notes you wrote down during the above procedure(s), fill-in the _EVALUATION FORM_ send to the following two addresses:   _[murphy](mailto:murphy@nmsu.edu)__[@](mailto:murphy@nmsu.edu)__[nmsu](mailto:murphy@nmsu.edu)__[.](mailto:murphy@nmsu.edu)__[edu](mailto:murphy@nmsu.edu)_**,  **_[rbeebe](mailto:rbeebe@nmsu.edu)__[@](mailto:rbeebe@nmsu.edu)__[nmsu](mailto:rbeebe@nmsu.edu)__[.](mailto:rbeebe@nmsu.edu)__[edu](mailto:rbeebe@nmsu.edu)_**.**
- If If you choose, register as a SUBSCRIBER to the PDS to be notified when data sets of interest to you become available. You can access this capability from the PDS Central Node home page.

## Evaluation Form 

**PDS DATA ACCESS TOOLS EVALUATION FORM**
- (please return to _[murphy](mailto:murphy@nmsu.edu)__[@](mailto:murphy@nmsu.edu)__[nmsu](mailto:murphy@nmsu.edu)__[.](mailto:murphy@nmsu.edu)__[edu](mailto:murphy@nmsu.edu)_ or _[rbeebe](mailto:rbeebe@nmsu.edu)__[@](mailto:rbeebe@nmsu.edu)__[nmsu](mailto:rbeebe@nmsu.edu)__[.](mailto:rbeebe@nmsu.edu)__[edu](mailto:rbeebe@nmsu.edu)_ no later than June 04, 2004)
- (if you prefer, please FAX to: 505-646-1602)
**PDS Central Node Data Query Tools Test**
**May 17- June 04, 2004**
**NAME**: ______________	**your email** ____________________
**INSTITUTION**: _________________________________________________
**TOPIC** you selected _____________________________________________
- Date(s)/time you conducted your test _______________________________...

## Evaluation Form (cont) 

- Please use a numerical rating from **1** to **5** to provide feedback about the various aspects of the PDS Central Node data access system for the categories that follow. The numerical rankings we wish you to employ are defined as follows:
**1**: Excellent (I can’t envision any changes being needed)
**2**: Very good (It generally performed well, but I could suggest a change or two)
**3**: Good (I was able to get what I wanted, but it was not an error free process)
**4**: Not good (this aspect was difficult to work with and did not immediately provide what I was expecting)
**5**: Poor (I could not determine how to use this piece of the system, and was unable to determine what ‘next step’ to take)

## Evaluation Form (cont) 

- _PLEASE RATE THE FOLLOWING ASPECTS OF YOUR EXPERIENCE_
- _______A. The initial view of the PDS Central Node Web Page
- _______B.  Being able to determine where to initially click to begin a data query session from the PDS Central Node web page
- _______C.  The assistance provided on-line to help you determine which type of  search to initiate
- _______ D.  The number and types of ‘filter’ options available to create your data query
- _______ E. The response time of the system in processing your query requests

## Evaluation Form (cont) 

- _________ F.  The information provided about data set size, and what options in addition to very long download times are available
- _______ G.  The length of time required for data to be downloaded, or delivered in a hard-media form
- _______ H.  The availability of and usefulness of on-line HELP options as you  proceed through the system
- _______  I.  The system’s ability in assisting you to identify data of interest
- _______ J. The system’s ability in assisting you to locate data of interest
- _______K. The usefulness of ancillary information (files containing information describing the structure of the overall archive and the specific data file(s) ) you obtained along with your data set(s)

## Evaluation Form Results (cont) 

## Evaluation Form Results (cont) 

## Evaluation Form Results (cont) 

## Evaluators Comments 

- Comments were significant and very useful for continuing improvement of system
- They reflected needed improvements of whole system--many of which are in the discipline node 5-year proposal plans
- They reveal need for continuing development of finer-grain search capability.
- Also revealed the lack of data in specific areas

## Response 

- Central Node is in the preliminary stages of analyzing the comments from the evaluators
- Many of the comments are a part of the development queue for FY05:
- Enhancements to search functions, e.g. text based search (“Google”)
- Enhanced help for inexperienced users
- Data sets originally noted as offline are continually being added to the online system
- Problems noted with incorrect urls were fixed in FY04
- Requests for more analysis/display software will be routed to the  Discipline Nodes, as have comments related to specific DN web sites.

## Summary 

- Users had no difficulty in using evaluation tool.
- The selected group of computer-savvy, motivated evaluators returned significant criticisms and suggestions.
- The central node has reviewed this initial test and planned to integrate identified modifications and improvements
- This approach of tailored scenarios focuses attention on areas where critical input is needed.
- The question remains--When and how do we use this approach?
- Evaluating specific system improvements?
- “Farming” the DPS agenda?

## Backup Material

## An Evaluator’s Response  

- I have some comments which are specific to the questions above, which should help to explain the assessment I gave:
*____2__A_The initial view of the PDS Central Node Web Page * Generally good, clear, can find everything I might need to get going. I particularly like the toolbar across the top, with automatic dropdown descriptions of each item. Would be nice if the nodes along the bottom toolbar had similar descriptions.
*____3___B Being able to determine where to initially click to begin a data query session from the PDS Central Node web page   *It is confusing that there are 2 data search boxes on the webpage, and I had to go to both to determine which to use. It might be easier if the one on the left were labeled “full data search” and one on the right “rapid data search” or something like it.
*____2___C The assistance provided on-line to help you determine which type of search to initiate.  *Once I got to the right data search box, it was pretty straightforward.

## An Evaluator’s Response  

*____4___D  The number and types of ‘filter’ options available to create your data query*
- I love this feature. However, one problem I had with filtering before searching was when I put in the following:
- Mission: Galileo
- Target name: Europa
- Instrument: Solid State Imager
- After hitting “Go”, I was directed to a page where the only option returned for “data products and related files” was to “Search for products with Ring node search tool”. Shouldn’t I have been sent straight to the Imaging node tool (or the Atlas)?

## An Evaluator’s Response  

*____2__E_ The response time of the system in processing your query requests*
- Pretty fast response to data queries directly within node; not always so fast for some ancillary data.
*____3__F_  The information provided about data set size, and what options in *
*        **addition to very long download times are available*
- I could have used more information about what some of the technical terms actually mean when selecting different data for sorting and downloading. I worked it out after a while, but it wasn’t always obvious

## An Evaluator’s Response  

*___1____G The length of time required for data to be downloaded, or delivered in a hard-media form*
- Data download was pretty instantaneous
*___5____H  The availability of and usefulness of on-line HELP options as you *
*        **proceed through the system*
- When scrolling through the SSI dataset from the imaging node, I was unable to get to the “technical help” section at all – I kept getting a “not found” error message. I could only access this page from the initial data search page.

## An Evaluator’s Response  

*___4____I  The system’s ability in assisting you to identify data of interest*
- The most useful thing that could be changed in order for me to complete the assigned task, is to be able to grab a set of files, say from a particular orbit, then sort them –with the browse images – in terms of a given parameter, such as lat lon, or resolution, and then sort that set and so on. Right now it appears that I can generate a list of images with the data I am seeking, but I can’t generate a list that includes the browse images, neither can I sort that list to see whether the existing data covers something I need. For example, right now if I want to find out what the highest resolution coverage of the Europa data is, I have to go through a lot of steps (or lists) to find out. Maybe this is a tool that is beyond the mandate of the PDS, but it would be very useful if this capability was available.

## An Evaluator’s Response  

- It would also be very useful to be able to save a specific search for a short period of time, in case I wanted to change one or two of the search parameters. Right now, I can only download the data. Again, this is probably beyond what the PDS is supposed to offer, but it would be useful nevertheless.
- On a minor note, for general searches, it might be more useful to have the search go to the Atlas start page first (when you click on “Search for products with the Atlas”), rather than straight into the data file search (so that you would have the option of searching the CDs if you knew what you were looking for).
*___3___J The system’s ability in assisting you to locate data of interest*
- The system is able, but it’s not always obvious how the system works when you are down in the weeds. There is generally some amount of trial and error to get what I want (which I generally do in the end).

## An Evaluator’s Response  

*___2____K The usefulness of ancillary information (files containing information describing the structure of the overall archive and the specific data file(s) ) you obtained along with your data set(s)*
- The information about the labels is fine, and was useful. It would be handy to have some links to, say, the Galileo web page so that I could get specific instrument information to help narrow my search. It seems that one needs a certain amount of preexisting knowledge in order to use the PDS to its fullest. While I can get away with this for Galileo, I don’t know that I would have got on so well with one of the other tasks.
- In general, most of the problems I have seem to be due to my inexperience (or impatience!), rather than any failing on the part of the PDS, and I generally get there in the end. Maybe the PDS could run open training sessions at AGU or something similar.

## An Evaluator’s Response  

- Final comment: I wanted to mention that I had a problem retrieving Galileo data a few months back and emailed Elizabeth about it. Not only did she tell me straight away what the problem was (a server crashed, I believe), but she kept me informed of the progress in fixing it and told me when I could get to the data again. I was really impressed with her service.

## Topic 3

**TOPIC**:  Venus Surface Properties– (Developer: Ray Arvidson & Tom Stein, Geosciences node)
**SCENARIO**: Investigate the idea that precipitation of metallic materials at higher altitudes is a possible explanation of the high elevation, high dielectric constant correlation that has been observed to exist for the Venusian surface.
**TASK**: Use available global map data to identify and locate areas with high elevation and high dielectric constant.  Focus on these areas to find higher resolution data sets of surface characteristics (e.g., bistatic radar, altimetry and radiometry data), along with derived surface characteristics from individual orbits.

## Topic  8

**TOPIC**:  Saturn Atmosphere – (Developer: Reta Beebe, Atmospheres node)
**SCENARIO**: Analysis of HST imaging has suggested that the Saturnian equatorial velocity has declined significantly over the past 20 years. In preparation for the data to be returned from Cassini, you want to familiarize yourself with all of the previous data that can provide information about Saturn’s equatorial motions.
**TASK**: Locate data available within the PDS that will allow you to characterize to the greatest possible extent Saturn’s equatorial velocity and variation in cloud morphology,during this period of time.