# Presentation: 162941

## The Development and Evaluation of the Climate Time Line Information Tool

**Communicating Climate Systems Variability with the Powers of Ten**

**Notes:**
Greetings.
My name is Mark McCaffrey and I’m a Science Communications Specialist working for NOAA’s Paleoclimatology Program in Boulder, and I actually work through CIRES, the Cooperative Institute for Research in Environmental Sciences at the University of Colorado. 

Today I would like to share with you some of the background on how a group of us developed and assessed the potential for a proto-type called the Climate Time Line Information Tool....or CTL for short. 

The primary goal of the Climate Time Line is to communicate in an interdisciplinary manner background information about the variability of climate systems over a variety of time scales......and hopefully to do so in an engaging and effective fashion. 

## Public Understanding of Climate Change

- 88% of the public has heard of global warming

- Data from the 2001 NSF survey

- Of those, 77% believed that “increased carbon dioxide and other gases released into the atmosphere will, if unchecked, lead to global warming and an increase in average temperatures.”

- 86% of all respondents feel it should be treated as serious or very serious problem.

**Notes:**
First a little background on the general public’s understanding of climate change.  
According to a recent NSF study on public attitudes towards science and technology, 88% of the public has heard of global warming, and of those, more than three quarters believe the increased carbon dioxide and other greenhosue gases will lead to global warming and the increase of average temperatures. So it sounds like Al Gore and the IPCC and EPA are doing a good job of getting the word out.  By far, everyone agrees that global warming should be regarded as a serious or very serious problem. 

So far, so good. 

## “The issue of climate change respects no border.  Its effects cannot be reined in by an army nor advanced by any ideology.  Climate change, with its potential to impact every corner of the world, is an issue that must be addressed by the world........”

- “The issue of climate change respects no border.  Its effects cannot be reined in by an army nor advanced by any ideology.  Climate change, with its potential to impact every corner of the world, is an issue that must be addressed by the world........”

- President George W. BushRemarks from June 11, 2001 speech

- “(But) we do not know how much effect natural fluctuations in climate may have had on warming.  We do not know how much our climate could, or will change in the future.  We do not know how fast change will occur, or even how some of our actions could impact it.”

**Notes:**
There is almost universal consensus that climate change-- specifically anthropogenic forcing from the combustion of fossil fuels causing increases in the greenhouse effect-- is happening.

But what there isn’t full consensus about is what we really know about climate change-- how much of it is within the natural variability of climate systems, how fast change can occur.

What we do know is that most people don’t know much about climate change and its potential socio-ecological impacts. 

## Public Understanding of Climate Change

- 70% lack understanding of basic scientific processes

- 49% don’t know that it takes the Earth one year to orbit the sun

- Environment not a major concern-- ranked 16th

- Only 31 percent knew that drinking water is the primary method that chemicals and minerals enter the human body (Roper)

- 13 percent were aware that less than 1 percent of all the water in the world is fresh (Roper)

- ....but agree it will be #1 in 25 years (Gallup)

**Notes:**
The NSF study I mentioned a moment ago showed that 70% of the public lack basic understanding of scientific processes, which confirms previous studies that show that less than 10% of the population is scientifically literate. 

49% of people don’t know what a year is.....in other words they don’t know that that’s how long it takes for Earth to orbit the sun.
Currently the environment is not a major concern-- a recent Gallup poll indicated concern for the environment ranked 16th in a laundry list of top worries...but everyone agreed that the environment will be the #1 problem in 25 years.

These other stats don’t directly address understanding of climate change, but they are indicators of people’s overall understanding -- or lack thereof-- of how humans fit into the Earth’s environmental systems. 

## How to Improve Understanding of Climate Variability?

- CIRES Innovative Research Grant as opportunity to experiment

- Provide interdisciplinary approach to climate change information

- NOAA is a treasure trove of climate and environmental information

- Looking for ways to integrate paleoclimate research and time scales with modern research and time scales

- Use “Powers of Ten” exponents for temporal scaling

**Notes:**
So, assuming we don’t want a public that believes in global warming based solely on their blind faith in what politicians and the media convey in sound bites, what can we do to improve our collective understanding of climate variability?
Well, we know that NOAA is a treasure trove of climate and related environmental information.  If we simply correlated NOAA resources-- research and websites-- with National Science Standards, that would big a major win in itself.
We also know that we need to provide an interdisciplinary approach to discussing climate change information so that it isn’t just a matter of putting out the output of models or listening to talking head experts. Jim White, who is, among other things, the head of the Environmental Studies program at CU says there is a crying need for interdisciplinary materials, books and resources, especially about climate change...in part because the students are already there-- they already think in a cross disciplinary manner. 
So another CIRES colleague -- Dan Kowal-- and I found out about a CIRES innovative research grant, which normally goes towards funding off the beaten track experiments and has only once previously funded anything that is related to outreach/education.  So we wrote a proposal since this was beyond our NOAA job descriptions. 
And we had the idea that we wanted to use the “powers of ten” to allow us to zip through time in a way that normal linear timelines simply don’t. 

## NOAA Education & Outreach

- Goal:  Help meet NOAA’s commitment to Education and Outreach and to improve scientific/climate literacy

- Education Committee now deciding on fixed percentage (like NASA) that will be allocated for EPO

**56.NOAA should require that a fixed percentage of program funds be dedicated to the development of effective education and outreach strategies.**

- i.e. Education & Public Outreach

**Notes:**
We had some idealistic notions of helping NOAA meet its commitment to Education and Outreach--- both of us were new to NOAA and didn’t realize all the issues and politics involved-- but as it turns out NOAA now has a very strong mandate for Education and Outreach-- or at least Outreach. The new NOAA Program Review Report recommends that NOAA should require a fixed percentage of program funds be dedicated to development of effective education and outreach. 
The NOAA Administrator-- Vice Admiral Lautenbaucher agreed with the recommendations, and right now the Education Committee is deciding what that fixed percentage should be.  They are using the NASA model which allocates a fixed percentage for what they call EPO, or Education and Public Outreach.
Incidentally, there is a myth that NASA has a legislated mandate for Edcuation and Outreach, but the Education Committee has looked into this and found out that there in fact isn’t one... they’ve just acted as if they did.
Apparently, as they has gone up on the hill to explore the mechanism that would allow this to happen, they are getting feedback that “education” really is not appropriate to NOAA’s mission, but Outreach is OK, so things are still being worked out. 

## CTL Development Timeline

- 2001

- 2002

- Develop Proposal for CIRES IRG- Submit April 2

- Waiting...

- Develop & Iterate Conceptual Framework

- Develop & Iterate Technical Infrastructureand Content

- Poster Session at AGU- December

- Develop & Iterate Technical Infrastructure and Content

- Conduct Evaluation- April-May

- Evaluation Analysis- June-July

- Develop Evaluation Plan & Receive OK

- Receive award!

- Developing and Testing the CTL Prototype

- Content

- Tech

- Content

- Tech

**X**You are here

**Notes:**
Any project that talks about a timeline obviously needs a timeline to organize its development....so here we are now and here’s the main steps in developing and testing the proto-type...
Developing the proposal--- trying to think through the process and goals and boiling them down into two pages including budget.....then waiting for the word and finding out that we received a grant award.
We’ll go through the the other steps in more detail shortly, but much of the technical and content development went on separately but in parallel.  The AGU conference in San Francisco last December when we gave a poster session was a major milestone since we wanted to get feedback from potential users sooner rather than later.
Then through much of last spring we continued to fine tune the website with feedback from within NOAA as well as from a few experts outside of NOAA.
The evaluation was planned in the late spring and then performed mainly in May, and if we could do it over again we’d make sure we had more time since we were racing to finish our interviews and observations with students before school finished up. 

## Conceptual Development

- Exponential Scaling- Log 10

- _[From FSU Molecular Expressions](http://www.microscopy.fsu.edu/primer/java/scienceopticsu/powersof10/index.html)_

- But really only focus on first 105 years

**Notes:**
Using log ten scaling obviously works well for spatial scaling.  Many of you have probably seen that book The Powers of Ten that came out in 1984: A Book About the Relative Size of Things in the Universe and the Effect of Adding Another Zero....or maybe you’ve seen some of the websites that have been inspired by this approach, like the FSU Molecular Expressions website that goes from 10 to the 23rd meters to 10 to the minus 15th meters. 
We thought we go from one hour-- which is roughly 10 to the minus 4th year, to 10 to the 10th or 10 billion years.....but that was a bit too ambitious, and we ended up settling for going from one day to the hundred thousand year scale, just to keep things focused. 

## Powers of TenTemporal Scaling

- J. Murray Mitchell, Jr.

- An Overview of Climatic Variability and Its Causal Mechanisms - Quaternary Research (1976)

**Notes:**
Log ten temporal scaling is obviously not something we invented-- it’s used in research all the time.  We revisited a study that was done by J. Murray Mitchell back in 1976 called “An Overview of Climatic Variability and Its Causal Mechanisms”. 

## Varying Time Scales

- Concept- developed by Dr. David M. Anderson

- One Day

- One Year

- Ten Years

- 100 Years

- 1,000 Years

- 1 Million Years

- 1 Billion Years

- 4.5 Billion Years Ago Earth Formed

- Interactive Web version - developed by Dan Kowal

**Notes:**
Dave Anderson, who is one of the NOAA scientists we work with, developed his own version of Mitchell’s timeline, and Dan Kowal took the design and made an interactive version of it that’s available on the website. 
By the way, here’s when the first day and then the first year occurred. We take the length of a day as a universal measure of time that we then break down into minutes and seconds.  We talk about the universe being 14 billion years old....but we’re using units of measure that didn’t exist until the Earth starting rotating and orbiting. 
This axis is a measure of relative variance.  Notice that annual solar forcing is the big one, and then the diurnal cycle. 

## Technical Development

- Headed by Dan Kowal- National Geophysical Data Center/CIRES

**Notes:**
Dan Kowal was a master at the web development.  I wish he could be here today, but he’s on the beach at Cape Cod today.  We used a variety of techniques -- including Hypernews-- to get organized and develop our initial iterations, and Dan kept challenging himself to learn new tools and try out different approaches. 

## Content Development

- Need to separate the linear (past events) from the non-linear (cycles, patterns and the science involved).

- Use matrix to frame content, then develop and review content.

**Notes:**
I worked on developing most of the content, using a matrix with four main themes across the top-- overview, climate science, climate history and resources-- and timescales down the side. 

We found fairly early on that it was important to separate the linear events-- what has happened in the past ten year, or past 10,000 years-- from cycles, patterns or processes that seem to occur at particular time scales.  We haven’t done this perfectly by any means but we felt it was important to try to pull these things apart. 

## Tour of Climate Time Line Information Tool

- _[http://www.](http://www.ngdc.noaa.gov/paleo/ctl/)__[ngdc](http://www.ngdc.noaa.gov/paleo/ctl/)__[.](http://www.ngdc.noaa.gov/paleo/ctl/)__[noaa](http://www.ngdc.noaa.gov/paleo/ctl/)__[.](http://www.ngdc.noaa.gov/paleo/ctl/)__[gov](http://www.ngdc.noaa.gov/paleo/ctl/)__[/](http://www.ngdc.noaa.gov/paleo/ctl/)__[paleo](http://www.ngdc.noaa.gov/paleo/ctl/)__[/](http://www.ngdc.noaa.gov/paleo/ctl/)__[ctl](http://www.ngdc.noaa.gov/paleo/ctl/)__[/](http://www.ngdc.noaa.gov/paleo/ctl/)_

**Notes:**
So let’s take a quick tour of what we put together.

## Getting the Word Out

- (Marketing)

- June 14, 2002

- Use list serves and informal networks to invite feedback of proto-type.

**Notes:**
This spring we began to informally ask for feedback from the climate community and educators.  It wasn’t really full blown marketing since we’re testing the waters, so to speak, but we did want to cast a big enough net to get a wide range of feedback. 
This past June we were reviewed by Science Magazine, and received feedback from people in India and New Zealand who basically liked the site but felt it didn’t have enough information about global warming. 

## Evaluation

- Audiences
- Undergraduates
- High School
- Middle School
- Experts

- Instruments & Protocols
- Surveys (online and paper)
- Interviews
- Observations
- Comments/Feedback

- University of Colorado Grad Students from CLEAR Team
- Abide by CU Human Research Committee guidelines

**Notes:**
And we also had enough money left in our budget to hire a team of CU Graduate Students-- Elizabeth Grassi and Heather Smith-- who were able to help us do a formal evaluation.  
Because Dan and I are technically CU faculty and the funding for the project came from CIRES, we had to abide by CU’s Human Research Committee guidelines, which was an interesting process and slowed us down a bit.
We originally wanted to focus on undergraduates, but ran out of time to do full blown interviews and observations with many of them, but we did end up interviewing and observing some high school and middle school students, and we received feedback from a wide range of experts.
The various instruments and protocols we developed are on the CTL website if any of you are interested in the actual materials we used. 

## Evaluation Results

- Accessibility good in general-- some problems with low bandwidth users (high school computer labs).

- Increased understanding of climate science, climate and human history, the forces that drive climate variability, and the relationship between climate and human history.

- High school and middle school students say they learned “nothing” during short visit, but test scores improve between pre and post.

- 100% of the undergraduate quiz scores increased average of 42%. Well suited for this audience.

**Notes:**
So what did we find out about the useability and content of the website?In general the site was accessible by almost everyone, although we had some problems when the high school students were trying to log onto the online quiz and their computers kept crashing. We’ve tried to work on the platform and browser compatibility issues, but the site is graphics intensive and so for folks with limited bandwidth might have problems.  We intend to have a low-bandwidth version, but that is on the backburner now. 
The interviews and pre and post quiz results indicate that students understanding of climate variability and climate events does increase....although when asked what they learned they say “nothing”. 
The undergraduate students-- who we thought we our original target audience-- did very well.  They all had high bandwidth and fast computers, they were all in the Environmental Studies program at CU and had some background on climate change....and their scores all increased between before and after visiting the site by an average of 42%. 
Many of the students treated the quiz as a treasure hunt-- and they’d search through the website looking for answers to the questions.  One middle school teacher thought we could develop learning activities that used this approach since it can be fun and engaging.....but we’d also need to make sure actual learning is happening and that they aren’t just memorizing trivia. 
We did learn that the site may actually work at the high school level, and, even though the concepts are somewhat advanced, even middle school students seemed to get something out of the site. Again, more details of this are on the CTL website. 

## Evaluation Results

- Experts found the site to be clear and organized, a useful resource, but were split as to whether the site was easy to use and navigate and whether the links were helpful.

- Experts identified several areas that need more information or coverage, esp. global warming

- Overall message: On the right track... and room for improvement!

- “The CTL website appears to be an excellent learning tool and certainly provides in-depth information on our climate.  It is evident that a lot of thought and work went into creating this website, and the effort to provide this kind of tool to high school and undergraduate students, and the public as well, is commendable.” Vice Admiral Conrad C. Lautenbacher

- “All of the suggestions below include ways you could make the content a bit more comprehensive...” (EPA)

**Notes:**
The experts we were in touch with found the site was clear and organized, although there was some debate on content and navigation of the site.  Several disagreed with our inclusion of research about a flood in the Black Sea some 7600 years back that Ryan and Pittman have proposed was the Deluge that is found in many world mythologies. 
Many thought we needed additional content-- some wanted more on longer time scales-- hundreds of millions of years-- and several wanted a stronger dimension on global warming.  Because there are a number of existing sites that focus exclusively on global warming, I wanted to focus primarily on natural variability, but we plan to add some additional information to better correlate population growth with CO2 and other greenhouse gas emissions. 
With a few notable exceptions, most of the feedback was very positive and confirmed our sense that the proto-type was on the right track....and there is plenty of room for improvement. 

## Instructional Design Assessment

- Conducted by Lucie Sommer

- A "database" of climate related resources, organized by a power of ten time scale. The learning context in which this database might be employed has not been defined nor has the intended audience.

- Need to better define teaching/learning audience, contexts and objectives. Some organizational confusion between “science” and “history”.

- Revising the site's content organization system and clarifying learning objectives important. Parallel database and guided inquiry activities.

- Graphic design issues interfere with "learning”. (True in 99% of faculty driven projects.) Solution: hire graphic design expert.

**Notes:**
We also hired an instructional design expert- Lucie Sommer-- to evaluate the project in terms of its potential for serving as a centerpiece for a curriculum project. In thinking about next steps, we considered proposing to the National Science Foundation that we build on the CTL pilot project to develop curriculum for undergraduate education that could use the basic framework to help faculty, particularly at non-research universities, communicate climate systems variability to their students. 
Lucie reported back that while she thought the CTL was an interesting “database” of climate related resources, we didn’t have a clear sense of our audience or the learning context for the project.  If we want to move forward with a next generation of CTL, we need better define the audiences, contexts and objectives involved. She also suggested some organizational confusion between the “linear” climate history and “nonlinear” climate science, which is true. 
From her perspective, we need to revise the site’s organization and get clear about the learning objectives.  One idea would be to keep the “database” approach but add parallel learning activities.  She also felt that the graphics interfered with “learning,” but added that this is true in 99% of project that are developed or driven by faculty, who may know content but don’t have the background or skills of graphic design experts. 

## Next Steps

- Surveying and analyzing teaching/learning needs and assets
- Surveying and analyzing technology systems and support
- Discussing and writing overall goals for project
- Drafting a revised content outline for "database"
- Collecting diverse feedback on draft outline
- Researching and compiling shared learning objectives for topics within content database
- Creating content outlines for topics, defining subtopics
- Identifying shared teaching/learning needs (specific to content topics)
- Writing a list of desired teaching/learning outcomes

- DESIGN TASKS

**Notes:**
That was somewhat discouraging, but Lucie outlined exactly what we’d need to do if we wanted to really turn the Climate Time Line into a robust tool with strong pedagogy and appropriate design elements. 
First, it would be necessary to survey and analyze the needs and assests as well as the technology systems involved.  We’d need to discuss and get down on paper our goals, and draft a revised outline for the content of a climate database.  We’d need to collect feedback on the draft outline, and come up with learning objectives for specific topics within the database.
Then we’d need to create content outlines and subtopics, identify the learning and teaching needs for the content topics, and write a list of outcomes that we’d like for a number of demonstration topics. 
This is all during the design phase.

## Next Steps

- Identifying individuals to develop teaching/learning activities
- Selecting appropriate teaching/learning methods & strategies
- Drafting design plans & collecting feedback on draft designs
- Adding additional content topics as needed
- Finalizing content outline for database content
- Redesigning navigation/organization system for project
- Collecting feedback on nav/org system revisions
- Programming navigation/organization system for project
- Programming demonstration templates
- Writing discipline related content
- Entering original and borrowed content into project

- DEVELOPMENT TASKS

**Notes:**
The next stage would be development, starting with pulling together a team to develop the activities, deciding on appropriate methods and strategies, and drafting design plans.
Again, getting feedback of draft ideas is important, and additional content topics might be added based on the feedback.  The content outline should then be finalized, and the navigation and organization of the project will need to be revisited and likely redesigned. More feedback...Then working on programming the overall navigation system and developing some demonstration templates. 
The content will need to be written up, and if outside materials are used, written consent is required.....and all the content needs to be woven into the fabric of the project. 

## Next Steps

- Designing and programming page layout/color scheme for screens
- Creating/editing original still and animated graphics
- Researching/obtaining relevant graphics from outside sources
- Updating group members on project needs/status, coordinating design/development efforts
- Testing for technical flaws, troubleshooting technical difficulties
- Revising/maintaining project in future-- what’s needed

- DEVELOPMENT TASKS

**Notes:**
Finally, the actual screens need to be designed and programmed, original graphics need to be created, and if they can’t be created they should be obtained from outside sources. 
Throughout the process the development team needs to be updated to make certain the design and development efforts are coordinated.
The site needs to be tested for technical flaws, and plans need to be made on how to handle future revisions and maintainence of the project. 

## http://www.ngdc.noaa.gov/paleo/ctl/

- _http://www.ngdc.noaa.gov/paleo/ctl/_

**Notes:**
So where do we go from here?  We’re exploring with EPA the potential for collaborating with their efforts to improve understanding of climate change. We may apply for an NSF grant to develop undergraduate curriculum since there seems to be a need for such an interdisciplinary approach that can help integrate research and education.

But I’d love to hear any ideas you might have on the merit of this project or where we might go with it in the future.