Engaging Scientists in Metadata
Ownership: Framing theQuestions
International Symposium on Knowledge Communities 2012
December 13 – 15, 2012
Research Center for Knowledge Communities
University of Tsukuba

Jane Greenberg,
Metadata Research Center <MRC>
SILS/UNC-CH

Outline
1. Assumptions
2. Motivation
3. Overriding goals and objectives
 Dryad HIVE DataONE—PAMWG work

4. Conclusions and framing questions
5. Q&A

Assumptions
Prevailing metadata generation methods result inadvantages and limitations+-

Automatic

Efficient, consistent

Disambiguation
challenges**Manual (info.
professional)
Manual (author)

Able to disambiguate

Costly, inefficient, not as
intimate w/subject

Intimate, …

Quality limitations

Collaborative/combinatory

Best of all worlds

Challenges magnified,costly to find rightcombination

Social/annotation

Cheap, additional views Inconsistent*Formalized, standard schemes, vetted on some level.

Assumptions
1. Prevailing metadata generation methods
result in advantages and limitations
2. More than one way to skin a cat Complementary, alternative approaches
 Social technology

3. Ownership appeal
 Empowerment and sustainability

Motivation
1. Metadata Generation Research; AMeGA 2001-2005,
Metadata bottleneck
27 scientists; don’t touch my metadata

2. COPD (Chronic Obstruction Pulmonary Disease)
ontology ~ NIH
Need + attract domain experts; sustainability

3. Dublin Core / proliferation of metadata schemes(Riley, 2009-2010; Willis, et al, 2012)

Long tail 
4. Interoperability and data reuse
Dryad, DataONE, …

Outline
1. Assumptions
2. Motivation
3. Overriding goals and objectives
 Dryad HIVE DataONE—PAMWG work
4. Conclusions and framing questions
5. Q&ADryadSlides: 7••••DSpace repository software (open source)
DOIs via California Digital Library/DataCite
CCZero (CC0)
Dryad DCAP (Dublin Core ApplicationProfile), ver. 3.0

Dryad’s goals

Dryad’s Goals
 One-stop deposition/access for dataobjects supporting publishedresearch…
 Acquisition, preservation, discovery,
and reuse of heterogeneous digital
datasets
 Allow journals and societies to pool
 their resources

DATA 
SYNTHESISPRESERVATION

Dryad development and governance Dryad development - a joint project of NESCent, the UNC
Metadata Research Center, and a growing number of partnerorganizations.
 Stakeholders: journals, publishers and scientific societies, andresearchers

 Governance Dryad is a nonprofit organization
 Governed by member organizations, including journals, publishers,scientific societies, funding agencies, and other stakeholders.
 Board: Sets policy and long-term strategic goals– Reps from science, journals, societies, OCLC, MS, etc.

Charter Dryad members
















The American Naturalist (Am. Soc. of Naturalists)
BMJ Open (British Medical Association)

The Biological Journal of the Linnean
Society (Linnean Society of London)
BioMed Central

Ecology Letters (Recherche Scientifique)Evolution (Society for the Study of Evolution)
Evolutionary Applications
Heredity (The Genetics Society)British Ecological Society
Journal of Evolutionary Biology (European Society 
for Evolutionary Biology)
Journal of Fish and Wildlife Management
Journal of Heredity (The American GeneticAssociation)

Partner repositories: Knowledge Network forBiocomplexity, NCBI GenBank, TreeBASE, DataONE

Journal ofPaleontology and Paleobiology(Paleontological Society)Molecular Biology andEvolution (Society for MolecularBiology and Evolution)
Molecular Ecology and Molecular
Ecology Resources
Molecular Phylogenetics andEvolution
Oikos (Nordic Society Oikos)
Oxford University Press
Pensoft Publishers
Public Library of ScienceScience (American Association for
the Advancement of Science)
Systematic Biology (Society for
Systematic Biology)
Wiley-Blackwell

Joint Data Archiving Policy(http://datadryad.org/jdap)<< Journal >> requires, as a condition for publication, that data
supporting the results in the paper should be archived in an
appropriate public archive, such as << list of approved archives here>>. Data are important products of the scientific enterprise, and theyshould be preserved and usable for decades in the future. Authors may
elect to have the data publicly available at time of publication, or, if thetechnology of the archive allows, may opt to embargo access to thedata for a period up to a year after publication. Exceptions may be
granted at the discretion of the editor, especially for sensitive
information such as human subject data or the location of endangered
species.
Whitlock, M. C., M. A. McPeek, M. D. Rausher, L. Rieseberg, and A. J. Moore. 2010. Data Archiving.
American Naturalist. 175(2):145-146. DOI:10.1086/650340

Dryad’s Workflow

Author submits
manuscript to journal

Journal reports
accepted manuscript
to Dryad; Dryad
creates provisional
recordJournal invites authorto submit data toDryad & provides linkto provisional record

Author submits data
to Dryad; curator
reviews, issues DOI

Dryad sends bothauthor & journal theDryad DOI

Dryad publishes data
files with link toarticle; Journal adds
Dryad DOI to all
forms of article

From: managing.editor@molecol.com
Date: April 19, 2011 3:09:22 PM EDT
To: Author
Cc: journal-submit@datadryad.org
Subject: Dryad entry for MEC-11-0140.R1

Dear Author

Many thanks for agreeing to participate in the Dryad project. To upload
your data, please click the link below- it will take you directly to yourentry in the Dryad database.
http://datadryad.org/submit?journalID=MolEcol&manu=223330<deleted text>

Once you have uploaded your data please include the Dryad identifierin your manuscript. Please let me know if you have any questions
about this process.
All the best,Tim Vines,
Managing Editor, Molecular Ecology

Pre-populated
metadata
field

Package metadata harvested from email
Pkg metadata (exactharvest)
Pkg metadata (someediting)Pkg metadata (notfrom email)Email metadata (notused)

DCContributorContr. 101 (gr. 99%, bl. 1%)

DDICorresp
DCDescription
DCTitleDCSubject

Subj. 177 (gr. 97%, rd. 2%, bl. 1%)

DCSpatial
Spat. 35
Temp. 2
DwCSci. 26

DCTemporal
DwCSci.Name
0204060

80

100

120

140160180200File metadata harvested from package metadata
Subj. 177 (gr.
97%, rd. 2%, bl.
1%)

Contr. 100 (gr. 93%, bl. 7%
DCContributor
Subj. 185 (gr. 83 %, or. 1%, red 4 %, bl. 12 %)DCSubject

DCSpatial

File metadata (inheritexactly)File metadata (someediting)
File metadata (created, notinherited)Pkg metadata not used forfile

DCTemporal

DwCSci.Name
050100150200

Data object relationships
DerivativeEquivalenceB(=dataset A annotated)AA(=samedata setin SAS)(=dataset inExcel)AA(=samedata seton paper)(=data set)C(=data setA revised)

Whole-partA(=data set)

Sequential

A1(=a subsetof A)

A1

A2(=part 1of a data set)(=part 2of a data set)

Instantiation; notion of “a work”− Bibliographic relationship (Tillett, 1992, 1992; Smiraglia, 1999, 2000+.;
Coleman, 2002)

Challenges
motivating challenges…
 Operational with ongoing development
R&D, metadata, and team logistics

 Different workflows
 Growth and sustainability

HIVESlides: 25Helping Interdisciplinary Vocabulary Engineering (HIVE) <AMG> approach for integrating discipline CVs
 Model addressing C V cost, interoperability, and usability
constraints (interdisciplinary environment)
Building, Sharing, Evaluation the HIVE….
12/12/2012

Titel (edit in slide master)26

José R. P. Agüera

HIVE TeamLee Richardson

Ryan Scherle

Lina Huang

Bob Losee

Hollie WhiteMadhura
MaratheJane Greenberg

Craig WillisVocabulary needs for Dryad Vocabulary analysis
 600 keywords, Dryad partner journals• Vocabularies: NBII Thesaurus, LCSH, the Getty’s TGN, ERIC
Thesaurus, Gene Ontology, IT IS (10 vocabularies)• Facets: taxon, geographic name, time period, topic, researchmethod, genotype, phenotype…

 Results
431 topical terms, exact matches
 NBII Thesaurus, 25%; MeSH, 18%531 terms (topical terms, research method and taxon)
 LCSH, 22% found exact matches, 25% partial

 Conclusion: Need multiple vocabularies

HIVE PartnersVocabulary Partners
 Library of Congress: LCSH
 the Getty Research Institute(GRI): TGN (Thesaurus ofGeographic Names ) United States Geological
Survey (USGS): NBII
Thesaurus, Integrated
Taxonomic Information
System (ITIS) National Library of Medicine
and the National
Agricultural Library

Advisory Board










Jim Balhoff, NESCent
Libby Dechman, LCSH
Mike Frame, USGS
Alistair Miles, Oxford, UK
William Moen, University ofNorth Texas
Eva Méndez Rodríguez,
University Carlos III of Madrid
Joseph Shubitowski, Getty
Research Institute
Ed Summers, LCSH
Barbara Tillett, Library ofCongressKathy Wisser, Simmons
Lisa Zolly, USGSWORKSHOPS HOSTS: Columbia Univ.;
Univ. of California, San Diego;
George Washington University;
Univ. of North Texas; Universidad
Carlos III de Madrid, Madrid, Spain

~~~~Amy Meet Amy Zanne. She is a botanist.
 Like every good scientist, she publishes,and she deposits data in Dryad.

Amy’s data

Usability
Formal usability study 4 biologist, 5
information professionals~ Tasks, usability ratings, satisfaction ranking
 Average time to search a concept:
Librarians: 6.53 minutes
Scientists: 3.82 minutes~ consistent w/research at NIEHS, 2 times as long

 Average time for automatic indexing sequence
Librarians: 1.91 minutes
Scientists: 2.1 minutes
Huang, 2010

System usability and flow metrics
25

Enjoyment
20Concentration
Enjoyment (20 pt.)
Libraries/Scientists•17/15.25

15

System Usability
Scale (SUS) (100pt.)
Libraries/Scientists•74.5/79.8

105Huang, 2010
0
user1

user2

user3

user4

user6

user8

Challenges
 Building vs. doing/analysis
 Interoperability = dumbing down
ontologies
 Proof-of-concept/ illustrate the differences between HIVE
and other vocabulary registries (NCBO and OBO Foundry) People wanting a service
 General large team logistics, and having people frommultiple disciplines (also the ++)DataONESlides: 38

Data Observation Network for Earth(DataONE)DataONE• Distributed framework for sustainable cyberinfrastructure• Science and society support ~ open, persistent, robust,and secure access to well-described, easily discovered
Earth observational data.

Overriding goals and objectives
Develop an approach supporting metadata
ownership; community driven(fairly applied)

Evaluate ownership impact:
1. on empowerment and sustainability
2. as a complement to predominant metadata
approaches
3. for DataONE interoperability and data reuse40

DataONE summer intern program
DataONE Preservation and Metadata WG (PAMWG)

 Components of Successful Metadata Registry
Frameworks (A. Murillo, J. Greenberg, & J. Boone,
MRC/SISL/UNC-CH, and J. Kunze, CDL) Multi-method approach
1. Literature review• confirm value; necessary for long tail science
2. Registry evaluation• Access and services most important• Challenge in access  limited use; workflow
3. Survey
41

Respondent demographics (n = 74)

42

Types of metadata created: descriptive, administrative, etc.
14 standardized schemes used, lots of in-house43

Metadata Vision One dictionary• Crowd sourced plus lightly supervised canon• Anyone can look up terms• Any part of “metadata speech”• Anyone can propose and refine their terms• Strong terms rise, weak terms decline
DataONE all hands Sept. 17-20, Albuquerque New Mexico
 Met Laughed, Talked, Cried, Hugged
 Conquered
44

Translating a vision to principles
Low barrier for contributions.
Transparency in the review process.
Collective review, with rotating responsibilities amongcommunity members (scientists, developers,organizations, curators, etc.)
Consideration of elders (experts) to guide the review
process and maintain thoughtful, balanced discussion.
Voting capacity of all users on the candidacy of terms
submitted and their use.
Collective ownership of any user or organization.
Stakeholder engagement in the design and review
process.
45

•••
•

Prototyping•••46

Collective ownership
VotingGood rises to the topTracks history

How to populate?
How to ensure + sustain ownership?
How to measure?

Support and contradictions
 Support
 Data on the Long-tail National and international data sharing policies

 Contradictions
 NASA scientists
 Global data meeting (US/EU)• Simple set  need detailed metadata47

Outline
1. Assumptions
2. Motivation
3. Overriding goals and objectives
 Dryad HIVE DataONE—PAMWG work

4. Conclusions and framing questions
5. Q&A

Assumptions
1. Prevailing metadata generation methods
result in advantages and limitations
2. More than one way to skin a cat Complementary, alternative approaches
 Social technology

3. Ownership appeal
 Empowerment and sustainability

The Metadata Universe

50

Jenn Riley, Metadata Universe
credit to John Kunze, CDL for this slide, and next 3

The Metadata Universe

Jenn Riley
51

The Metadata Universe

Jenn Riley
52

The Metadata Universe

Jenn Riley
53

Framing questions
 What next…?
 How can we encourage scientists to:• Generate metadata?
• Care about metadata quality?
• Follow standards?

 Is there a threshold of expectation?
 Where do automatic applications best fit?
 How can we study this topic more?
• Need to balance applied work and basic researchSlides: 54

Concluding comments
 A contribution, have to start somewhere…
• Good timing, the right discipline Confirmed success on some level Machine capabilities, eScience/data synthesis
 An educative commons, intellectuallyengaging

Acknowledgements all aroundSlides: 56Dryad Acknowledgments
 Dryad Consortium Board, journal partners, and data authors

 NESCent: Kevin Clarke, Hilmar Lapp, Heather Piwowar, Peggy
Schaeffer, Ryan Scherle, Todd Vision (PI) UNC-CH <Metadata Research Center>: Jose R. Pérez-Agüera,Sarah Carrier, Elena Feinstein, Lina Huang, Robert Losee, Hollie
White, Craig Willis
 U British Columbia: Michael Whitlock
 NCSU Digital Libraries: Kristin Antelman

 HIVE: Library of Congress, USGS, and The Getty ResearchInstitute; and workshop hosts
 Yale/TreeBASE: Youjun Guo, Bill Piel
 DataONE: Rebecca Koskela, Bill Michener, Dave Veiglais, andmany others
 British Library: Lee-Ann Coleman, Adam Farquhar, Brian Hole Oxford University: David Shotton

HIVE Acknowledgments
 HIVE Development Team Dryad Repository Team Former SILS Masters students: Lina Huang and JacquelynnSherman

58

PAMWG
Acknowledgments
U.S. National Science
Foundation (Grant #OCI0830944).

The Subpopulations and Intermediate
Outcomes in COPD Study(SPIROMICS) is funded by contractfrom the National Heart, Lung, andBlood Institute (NHLBI) to the
University of North Carolina(HHSN268200900020C).

59

Metadata Subgroup Members

 Co Chairs: J.Greenberg + J.Kunze
 Sarah Callaghan, British
Atmospheric Data Centre Greg Janee, Digital library research
specialist, Earth Research Institute,
University of California Santa
Barbara Nassib Nassar, Senior Research
Scientist, Senior Research Scientist
RENCI, University of North Carolina
Chapel Hill Angela Murillo, UNC-CH
 Karthik Ram, Environmental
Science, Policy & Management,
University of California Berkeley
 Jim Regetz, National Center for
Ecological Analysis and Synthesis(NCEAS) Tim Robertson, Global Biodiversity
Information Facility (GBIF)