[070718ssc1000]{.underline}

Beacher Wiggins:

Good morning, and welcome to another in our series of LC's "Digital
Future and You." We're glad to see such a hearty turnout this morning
for today's speaker. We are recording this, as we do many of our series
now, so we just want you to know that it's part of the proceedings this
morning. It's so, there are plenty of seats up front; if you don't find
any near the back, come down. This morning's speaker is Andrew Pace, and
it's my pleasure to introduce him to the Library of Congress and to you
this morning. Andrew is head of information technology at North Carolina
State University libraries, where he's participated in several
successful initiatives, including ILS \[integrated library system\]
migration, Web interface design, and served as a project manager for the
development of the library's electronic resources management system and
its Endeca-based faceted space browse online catalog; the focus of his
topic, this morning, to us.

Prior to going to NCSU libraries, Andrew was a product manager for
library vendor Innovative Interfaces. He's an at-large member of the
LITA \-- Library and Information Technology Association board, and he
has just been elected vice president, president-elect. Congratulations,
Andrew. He's a frequent speaker and writer on several library topics. In
fact, it was his speaking at the first open forum for the library's
working group on the future of bibliographic control that spurred me to
have Judy Cannon and Angela Kinney, the coordinators of LOC's "Digital
Future and You," to invite Andrew to come join us at his earliest
convenience. And as they are wont to do, they followed up on the
request, and we have Andrew with us today. You also may recognize him
from his "Technically Speaking" column in "American Libraries" magazine.
So, welcome to Andrew, and take the floor.

\[applause\]

Andrew Pace:

Thank you, Beacher. I'm always a little nervous when I get applause
before I say anything. I want to thank you, Beacher, and to Judith and
Angela and Cheryl and everybody else who made it possible for me to get
here and help with the technology and everything. I'm going to talk
somewhat rapidly \-- I talk somewhat rapidly, and so if I start to talk
too rapidly I want somebody to raise their hand and slow me down. I'm
going to give you a quick overview of what I'm going to talk about
today. You can see that I have an awful lot to cover, and I want to save
some time during my presentation to make as many irreverent comments as
I can about the state of libraries, and plenty of time at the end to
dance around your questions or feign expertise I do not possess, and
generally leave you with the impression that even if I don't know very
much at least I was moderately funny.

I also just wanted to point out \-- Beacher mentioned my history with
Innovative Interfaces, but I like to kind of give this as my 'street
cred,' that I've been doing catalogs since 1994. I started down the road
at Catholic University, where I did my graduate work converting the
Aladin catalog guides of NOTIS from paper to HTML. Some of you might
remember that. From '96 to '99 I was the product manager for WebPAC and
pretty much every product with the word Web in it at Innovative
Interfaces, which led to my ultimate departure and into the academic
arena where I became the DRA, then Sirsi, then SirsiDynix Web2
administrator. It's worth pointing out that actually the last project I
worked on at Innovative was the mockups of OPAC interfaces for the
Library of Congress when you all were shopping for an integrated library
system, so at some point I want that week of my life back.

\[laughter\]

In 2005-2006, I was the project manager for the NCSU Endeca catalog
launch, which is the focus of my talk today. And then in 2006, doing
presentations about the faceted catalog. I'm very glad that this is
being recorded. I've purposely tried to use as much time as I can to
create a dance mix version of this presentation

\[laughter\]

so that I can point to it as often as possible when people ask me to go
around the country looking under rocks for the three or four people who
haven't heard about our Endeca catalog yet. So a little bit about the
motivation for what we did, and my colleagues will tell you that I am
attempting to beat this metaphor into the ground, of the information
library systems puzzle. I'm a lover of metaphor, and I've been trying to
come up with a good one, a better one than this, but it's the one that I
continue to use.

This is my gross oversimplification of an attempt to describe the four
main pieces of the puzzle that my library, particularly IT \[information
technology\], have been charged with putting together: the catalog,
serials, abstracts and index, full-text databases, and my catchall, the
amorphous Web, my kitchen sink category for full text e-books, digitized
collections and the laundry list of content that we have out there. And
if you think of the catalog, for example, as one piece of the resource
discovery puzzle \-- and it is just one piece \-- and think of the
nonintegrated systems that we have now as a bunch of puzzle pieces, the
fact is that it is impossible to put these pieces together in the
current state of information technology. Rebuilding one of these things
without the larger puzzle in mind is like painting ourselves into a
corner. So you can see I love not only metaphor, but mixed metaphor.

This is sort of where my theory that I've been talking about for a long
time now of the disintegrated library system is understood. I do not
really believe that disintegration is what we are after; I do believe
that it's necessary to dismantle things and put them back together again
in order to do it better. So let me with one slide dismiss three of the
puzzle pieces, because we don't want to be here until sometime midday
tomorrow and just talk about all of these kinds of things. This list of
things that we're dealing with, with these other puzzle pieces that are
done against the backdrop of Library of Congress and OCLC \-- what I
euphemistically call the 800-pound gorilla and the elephant in the room.
You can decide which is which; I'll leave that to your imaginations.

So I'm going to concentrate today primarily on this catalog puzzle
piece. This is a great screen from my colleague Marshall Breeding, who
publishes library technology guides, and it's a right-to-left timeline
of the library automation marketplace, in which I think somewhere
between 60 and 70 corporate entities are represented. And you can see as
you move to the left that the mergers and acquisitions that have
happened over time \-- some belly-ups and things like that, that we have
a much different landscape. And this, again is part of this motivation.
So what we're left with is about 22 of that original 60; about 22 to 25
depending on how you count the open source folks. There are some vendors
that support the open source, so I put them in that vendor category. Of
these folks \-- now, I know the first thing that comes to people's minds
is, there are actually 20-something left? But there actually are, and
these are as much as I could represent of them on one screen.

And then, against my point that the traditional integrated library
system as we know it, the interest in which is waning among library
automation vendors, you start to see the plethora of products and
services that these companies are starting to provide, and again this is
only representative of my ability to download a .jpeg or .gif from their
corporate Web site. But this is our world, this is the library world,
and I want to thank my friends, one of these vendors, Talis, for
providing me a screen that gives us a better shot of the world of our
patrons and our users.

\[laughter\]

So, the rest of our motivation was that we found that that market space
that was out there was somewhat unresponsive. We'd been complaining
about the state of the online catalog for several years, something that
was near \-- it was a love/hate relationship I had with this thing.

So we didn't see a lot of response or promise of something coming down
the pipeline. With some reading and writing \-- I was writing an article
for "Library Journal" on the disintegrated library system and came
across some work of a guy named Mark Ludwig at SUNY Buffalo, who had
taken their data during a migration to Ex Libris and taken 70 gigabytes
of database data and translated it into seven gigabytes of XML data. And
he had this theory that you could actually build a better search engine
on top of these just flat files of data, and I was inspired by that, and
later that fall, published in the spring of that year, I wrote a little
article called "My Kingdom for an OPAC," in which I highlighted some of
the work like RedLightGreen (RLG) and Aquabrowser and this new company
that TLC, one of the vendors, was partnering with called Endeca.

It just so happened that that Winter the ALA Midwinter Meeting was in
Boston. Endeca is a company based in Cambridge; they invited us over for
a chat. We jokingly refer to this as the peanut butter and chocolate
meeting, where we realized that they knew a lot about search and
indexing. We knew our metadata; they weren't necessarily used to
customers who knew their metadata. So we had this great conversation
that was basically just a casual conversation about searching and online
systems, and then some more formal conversation after I came back from
that. It's worth pointing out that North Carolina State is an
organizational culture that inspires innovation. Okay, so we had \-- so
when we came back in January and took this to the administration and
said, "This is something that we want to try," it was embraced. And so
that was a very big part of this. And then we also did a very rapid
implementation. We started basically with the software and working with
Endeca in July, and we went live in January. We actually did not even
load the software on our servers until October of 2005, and it went live
in January 2006.

So the big picture that we were after was to improve the quality of the
catalog experience and exploit our existing infrastructure; make the
data that we had work a little bit harder, and build a more flexible
catalog tool that could be integrated with discovery tools of the
future. And I'll get to this at the end, because I think it's one of the
most important points.

What is Endeca? You might have heard of some of their smaller customers
that are here on the screen in front of you: Circuit City, Wal-Mart,
Barnes & Noble; we are now amongst this. They were in the \-- just in
the commercial space we were the first library user to bring up a
catalog powered by the Endeca engine, but we were preceded by many of
these sites. Since us, some others have followed: McMaster University in
Canada, the FCLA Consortium and CCLA Consortia in Florida, and Phoenix
Public Library. There's word that Chicago and Denver Public Library are
also pursuing an Endeca application. And at this annual meeting just
here in DC not too long ago we had our very first inaugural Endeca
library users group meeting at one of the hotels; a very impromptu and
hastily put-together meeting, but it felt like a critical mass in which
we could have some fruitful discussion.

Why Endeca? Because primarily the big thing that we're after, even
though we've gotten a lot of praise and discussion related to the
faceted catalog \-- the big thing we wanted was relevance ranking. We
wanted to be able to experiment with relevance ranking of bibliographic
data. We wanted better subject access by leveraging all of our metadata,
including the item-level metadata that we had in our catalog. We wanted
to improve response time; it's very hard to explain to a patron why
Google can search 5 billion Web pages faster than an online catalog can
search 2 million surrogate records. There are nice, good technical
explanations for that, but nothing that they should accept. We wanted to
enhance natural language searching through spell correction, and we
wanted the ability to do a true browse of the collection.

So, the context \-- and I've stolen this slide from a, some of my
colleagues who gave a presentation at ALA that I will reference at the
end of my presentation. But basically what we were looking at was this
context of having a significant investment in what we had built. RDA was
still slow in coming, and it was taking increasing heat. We didn't want
to necessarily wait for something to happen before we did something.
There were new communities of interest around subject access, so we knew
that as we were building this that some of the discussions were already
taking place. We take no credit for starting the discussions of the next
generation catalog, but we could see that this was something that was
going to come to a head.

And basically what we saw was that our integrated library system was a
maxed out inventory control system, and if you think about the history
of integrated library systems they were built to control acquisition and
circulation of materials; that the online catalog is an afterthought.
You can see that in a Web 2.0 world the library system as we know it is
nothing more than an inventory control system. Now, all of this said is
put much more succinctly by my colleague Roy Tennant who said, "Most
integrated library systems as they are currently configured and used
should be removed from public view." Okay. The saddest part about this
statement is that he said it four years ago.

So the other thing is that catalogs are hard to use. The vast majority
of libraries are still living with an OPAC that is bundled with their
ILS, that was created as an afterthought by vendors who didn't
necessarily specialize in search technology. Problems with these OPACs,
problems with these OPACs is effective search tools for today's library
patrons abound. At NCSU Libraries we saw lots of broad topical searches
performed in the online catalog search logs, but we didn't see a system
that supported this type of search very well. For example, simple
keyword searches often retrieved too many or too few results, which
leads to a general mistrust in the system among users who try to figure
out how to outsmart the system. Even when users make an attempt to use
authority headings, the browse list where they wind up is an A-Z browse
list, and it's often misunderstood.

Relevance ranking in online catalogs for the most part is in a genuinely
sad state. Our catalog provided only last-in, first-out relevance
ranking, which meant that the most relevant ranked records were by no
means those at the beginning of the results, especially after you load,
say, 50,000 government documents overnight, and those are the last
things in your, in your r record. And I don't mean to disparage
government documents; sometimes it's exactly what you're looking for.
Often the relevance of results is actually improved as you page through
the results set; this is unacceptable in a disintegrated world where we
hope to make search results available in other contexts, through Web
services. Most catalogs do not support features such as spell
correction, "did you mean?" automatic stemming, that are becoming more
and more common in other search contexts. And those that do support
spell correction are often just dictionary lookups redirecting you to
another possible spelling of the word not based on the actual spelling
of words in your dataset.

Sometimes students don't even realize when they are making spelling
mistakes, and it can lead to them walking away from a system thinking
that you don't own anything on a topic. A little bit more context \--
and some of this was as we were starting the project 18 months ago \--
but you can see that we were by no means the first to start thinking
about this problem. There were lots of things going on; WorldCat.org was
in beta, RedLightGreen, which was ultimately consumed by OCLC, was a
great interface that had not only a nice faceted display, but also a
Thurberized view of bibliographic records, FictionFinder, Vivisimo; the
list goes on. As things progressed the vendor community got into this as
well, in hopes that the second mouse gets the cheese, and then lots of
other open source projects were started, some of them even before ours.
And then of course you had the entire commercial Web that was out there
that was part of our motivation and some of the context with which we
were working.

I'd like to say just a few words about NextGen and 2.0. NextGen is not
really a phrase that I particularly care for because it's adjectives for
our libraries or for our systems, okay \-- this isn't actually our
current system,

\[laughter\]

or even our old one. But these are adjectives for the libraries and
systems that we're using; they are not adjectives for our patrons who
are already there. So in a lot of senses what we were doing was
catch-up. We wanted to make a system that was informed and enhanced by
search technologies that are being developed outside of the library
world, and things that are based on how our users know how to search,
not how we want them to search. These are just a couple of examples of
the kinds of interfaces that they know how to search, and also some
added incentive for us to design something a little bit better.

This is Clusty, a search engine that tries to take massive Web search
results and cluster them by topic. I'm going to go through these
relatively quickly. This is Amazon, who just recently added facets that
I think \-- and maybe somebody can even correct me on this \-- the
"narrow by category" that you see on the left I think are based on the
bisect -- bisection codes. This is one of our typical searches for
deforestation; I was always very curious what the relationship between
romance and deforestation was, so here it is. In case you were curious
you can always check out "Love at Calcutta Consulate," or even search
inside the book if you are so daring. That's not going to get me in
trouble, is it Beacher? Okay, I worked in National Archives, so I have
an FBI file if they just want to update it.

\[laughter\]

This is FictionFinder from OCLC, and one of the really nice things about
this is it has the number of editions and the number of libraries who
own the titles. I've been begging OCLC and continue to pressure OCLC to
release this data as a Web feed, as an RSS feed, because it would be
very helpful in determining either relevance or popularity of titles, so
it would be very nice to have that data. WorldCat \-- which you're
familiar with, and now also WorldCat local, which is being piloted at
the University of Washington among others \-- introduced this faceted
browsing. The Internet Archive just released a prototype of faceted
display of the catalog and better relevance ranking in their full text
resources, which is pretty interesting and fun to play with.

This is EBSCO, who partnered with Grokker to give a kind of a cloud or
grouping view, which is not my cup of tea, but I've seen other people
use it quite successfully and like this kind of interface. This is Koha
Zoom, which is an open source product that is live at Nelsonville Public
Library in Athens County, Ohio, which has faceted browsing; you can see
they're doing a lot of book jacket art and things like that. Here's
another one from the Georgia PINES Consortium, which you've probably
heard about. One of the complaints sometimes about open source software
is, does it scale? Well, Georgia Pine scales to the tune of 250
libraries, doing a heck of a lot more circulation than any academic
library is going to do in a given day. And they have a very nice
display. This is Project Blacklight from the University of Virginia
(UVA), so it's UVA plus Solr. The project's called Solr, so when you
have Solr plus UVA you get Blacklight, and so this is their project
there. I'm not sure when that's planning on being launched, but that's a
little screenshot for you.

Then you also have \-- the vendors are catching up in this space, too.
Ex Libris has a product called Primo. They're trying to kind of take
this to the next step by integrating discovery of other things, whether
it be meta-search or digital collections, things like that. Innovative
Interfaces has one called Encore. You can see they've got a word cloud
there on one side, and also the faceted navigation, pass-through
searching to Google Scholar, Masterfile, things like that. This is
Aquabrowser, which also has the word cloud on the left-hand side and
facets on the right. This is at Queens Borough Public Library, and this
is from AquaBrowser, which is now owned by Bowker, who is owned by
Cambridge Information Group. So, lots of interesting things going on
there between content and interface merging together. And this is Casey
Bisson's WordPress Catalog, where he took the contents of his online
catalog and put them into the WordPress blog software.

So the idea here was that you would enable the social tagging and
commenting and things like that, and that's a pretty neat one, too.
That's not a very exhaustive list, but it's kind of a good list if
you're interested in seeing some of these other things. One more thing
that I wanted to say about facets \-- and I mentioned this before, how
relevance ranking was extremely important to us. They are the icing and
not the cake, just like the metadata of our bibliographic data is the
bait and not the catch; the book is the catch. And so we saw the facets
and the experiments that we were doing with them, because this is
generally uncharted territory as kind of icing on the cake for us.

So, a little bit more about our local implementation and some of the
decisions that we had to make while we were doing this. As I mentioned,
we went live in January 2006. The system that we have works with a text
version of a daily snapshot of our library MARC data and other metadata
\-- the other metadata being primarily item-level metadata, which is not
in MARC format. It's used to improve the discovery portion of the online
catalog. It operates with our integrated library system, so we did not
replace our SirsiDynix system. This is purely a discovery tool. And the
Web2 interface that we have from SirsiDynix is still used for known item
searching; begins with phrase searching, and for our full record
display. And these were primarily timeline decisions, not necessarily
philosophical decisions, and things that we're still debating \-- about
the longevity and how long the old system will be around with us.

So the features that we were looking for were strength of search and
sort speeds, which we did not have in our old system; customizable
relevance ranking. It's a proprietary piece of software; it has many
relevant algorithms from which you can choose and sort and decide on
what you're going to use. The faceted browsing, as I said, was great
icing on the cake. It allowed us to introduce browsing with the LC
Classification scheme \-- I am going to do a demo, so don't worry, I'll
get to that in a minute \-- spell checking, "did you mean?" and
automatic word stemmings; some of those features that I mentioned that
we were looking for in an online system. The other thing that I should
probably mention is that the, because other Endeca customers have copied
our interface, is that the other thing that we wanted and that we like
very much at NC State is flexibility in interface design. The interface
that we put up there for our Endeca catalog is our interface; Endeca did
not create that. People have thought that because they've seen other
ones that have copied ours, they think that this is an out-of-the-box
Endeca implementation, but it was actually a design that we came up with
at NC State.

So the relevance ranking \-- just to give you a little bit of a
snapshot, it's based on, like I said, a suite of algorithms that they
include. And we were able to play with this. One of the nice things
about the speed of the indexing is \-- how many of you have sat in
integrated library system migration meetings where you know that the
decisions you're making could be the decisions for the next 10 years,
and so, you know, which field do you index, and things like that? We
found as we were playing with the Endeca software, if we didn't like it
we just reindexed that night and saw what it looked like in the morning.
If we didn't like that relevance algorithm we just changed it and
reindexed, or we could work with small groups of records.

One of the first changes that we made was putting up at the top the
query as entered. I like to talk about this as an ironing out, because
how many people are familiar with David Weinberger and his book
*Everything is Miscellaneous*? Okay, this is a book that librarians have
been writing for 25 years, except we called it "Everything is Poorly
Organized."

\[laughter\]

One of the first reactions that we had from a non-NC State user was from
David Weinberger, who was complaining that when he typed in "Cluetrain
Manifesto," another book that he is an author of, it was at the bottom
of the list because the Endeca system actually wanted to correct his
incorrect spelling of "cluetrain" into "clue, space, train." And so one
of the things that we changed immediately is giving the user the benefit
of the doubt that "as entered" is probably the most relevant, and we put
that at the top. Now you can do corrections and add those to the search
results, but that was what we did there.

We do phrase matching, field matching, the number of fields that are
matched, and field ranking so you can say that, you know, things like
"once in the title" and "once in the subject" is more relevant than
"twice in the notes field;" things like that. Weighted frequency, \--
tf, idf \-- term frequency, inverse document frequency, which is very
big in the full text world, but a little bit different and harder to use
in the bibliographic world. And then static ordering \-- things like
publication date and circulation statistics that we can use for
tiebreaking and the like.

We had to make decisions about which facets we were going to use, and
there was much debate about what to do about the Library of Congress
Subject Headings (LCSH). We decided to actually split them up, and the
list that I present here is kind of our way of splitting the facets that
we grouped into two categories. One was the "aboutness" category, the
Library of Congress headings split into their four component parts:
topic, genre, region and era and the LC Classification. And the other
thing is it describes the physical limitations of the book \-- is it
online, which library is it in, what format is it; things that help
people make decisions about where they're going to go when their paper
is due, et cetera, et cetera. And then we also have newness, which we've
actually just enhanced recently to include a period of newness as a
refinement, so that people can refine search results by whether it's new
in the last week, last month or the last three months.

So now I'm going to attempt something that few vendors would ever do \--
a live demo. A somewhat scripted live demo. So I'm going to go to our
catalog page, and you can see here -- I hope that the mic is picking me
up. Can everybody hear okay? You can see here that we have two search
boxes, another area of great debate. The search box on the bottom
actually searches our old Web2 system; it's called "search begins with,
title, Library of Congress subject heading, et cetera." There are some
of us that want to change its name to "classic search," but there it is.

\[laughter\]

One of the first things that I love to show, primarily because of its
speed, is to do no search at all, but just hit the submit button. And
you see that immediately I have 1.7 million bibliographic records at my
disposal, with all of their facets exposed. So for example, these narrow
results that you see down the left-hand side \-- again, there was very
little agreement as to what order \-- there was no consensus on what
order to put these refinements in, and I'll get to that in our
statistics in a minute, but you can see how they are presented here. We
present the first five.

So for example, if I wanted to see fiction works in the language of Urdu
\-- we actually have a fairly nice Urdu collection at North Carolina
State \-- then I can immediately have Urdu fiction. How many online
library catalogs can you sit down and say, "Show me Urdu fiction"? If
you type Urdu fiction into a search box, I think you're going to be
relatively disappointed with the results. But this is an example of kind
of coming right in and being able to browse the collection. I can then
go here and say, "Sort them by the most popular," which is an aggregate
of physical circulation, which is a problem in and of itself when you're
dealing with online resources. But you can see that you can go through
and do that kind of thing.

I'm going to do one of our favorite searches again, which is
"deforestation." Okay, so here's another example where I think breaking
up the subject headings \-- although it can be sometimes misleading \--
can also be helpful, because I might not have known that the "Amazon
river region" was the way to search for deforestation issues about the
Amazon. So I can choose "Amazon river region." And the paper is due in
two hours,

\[laughter\]

so I'm going to choose online, right away limit by online. Oh great,
class is cancelled and I got an extension,

\[laughter\]

I'm going to remove online and go back to a full list of results. And
then, as I mentioned, the full record display is actually used in our
existing online catalog, so when I click on one of these records you can
tell, because it's slower; it's using our existing online catalog so I
can pull up that record.

So this is a traditional catalog display. I can do my requesting through
the inter-library loan system, I can do a Library of Congress Subject
Heading search \-- here they are, LCSH \-- but a couple of kind of
prototypical things that we've been experimenting with is I can also do
something like "more titles like this." And we haven't really gauged the
usage of these yet, but I think they're kind of interesting, mostly
because I put them in the catalog.

\[laughter\]

But I can do "more titles like this" and actually do something like take
just the subfield A, so the Library of Congress Subject Headings, and
send them back as an "and"-ed keyword search back into the collection so
that I can see nine matching results that take that kind of general
topic. Or I can do something like an "or" keyword search. I can see that
this doesn't have a main author, but it's got added authors Charles Wood
and Roberto Porro, and I can say "more by these authors" and I can send
both of those authors "or"-ed together, back to the online catalog and
get all eight titles that those particular authors produced.

One of the things that we found was that even though we had the searches
"begin with" what we were preserving \-- for what the catalog does well,
which is known item searching \-- we found that it actually works pretty
well for known item searching too. So if I do "civilizations
\[misspelled\] of the ancient \[misspelled\]' \-- I was waiting for
somebody to groan while I typed ancient \-- and I'm automatically
corrected to include 'civilizations of the ancient," spelled correctly,
Near East. You can see that you get a very nice relevance up at the top:
"prehistory, civilization in the ancient Near East, ancient Near East,"
et cetera. That's an example of the spell correction.

One of the other things that you can do is you can set separate
thresholds for spell correction versus "did you mean?" So let's say I
type in "Hemmingway" with two M's. Well, there's actually an author who
spells their name "Hemmingway" with two M's, and actually we have six
titles by Hemmingway with two M's, and so we can list those six results,
but "did you mean Hemingway?" And so if you click on that you would
actually get that result, versus if I typed "Ernest Hemmingway" with two
M's, I'm not going to get that threshold and it's automatically going to
correct me to "Ernest Hemingway" and show me those results.

Another general keyword search to kind of show you how the facets can
work \-- if I do a search for "Ajax," which means different things to
different people, you can see right away in the facet classification on
topic that "Ajax" \-- and you can see the bias of the NC State
collection here, too; means Web site development, but it also might mean
something like history and criticism, Sophocles, okay. So this is
probably mostly coming from our early English books online collection,
which you can see the two different contexts that are there. I could
also, if I really wanted to \-- since maybe some of you have been doing
this search \-- but I can do a search for "Peloponnesian War," I can
even spell it wrong, and it will automatically correct it for me. I can
go to those topics and see that "finance of the Peloponnesian War," not
a very strong topic at North Carolina State University.

We leave that to our sister school down the road in Chapel Hill, but you
can see that you can do all kinds of things like that. We do have an
advanced search\-- I call this the political tab, where people want a
little more power. We had that very strong request to be able to do
searches that didn't include government documents; okay, so people knew
how to search for government documents, they wanted a search that didn't
include them. We found that relevance ranking is actually taking care of
this problem in and of itself \-- all by itself, so you can do that
there. We also have at the bottom what I like to call "librarian
search," but there it is. You can also enter a Boolean search however
you want.

You can actually browse the collection without typing any search. I
showed you an empty search browse of the collection, but here's another
example where you can actually go in by the Library of Congress
Classification \-- we've added some words next to the letters to make a
little bit more sense of them; that's had some usability discussions
around it \-- but I can choose political science, I can see that JC 11 -
JC 607 is political theory, I can scroll down and say \-- so right away
I've narrowed my J's to about 5,000 titles. I might want to go down and
say "Show me just the new titles in the last three months," and "Show me
the most popular titles," and you see the most popular titles are all
checked out because they're popular. Okay, that's all for online demo.
If we have time at the end we can go back and do some of your favorite
searches.

How did we do it? We had a small and nimble implementation team. It was
IT heavy because we are getting into some uncharted territory on the
technology. I was the project manager, technical lead was Emily Lynema,
whose name you might have heard; she was a recent "Library Journal"
mover and shaker, our integrated library system librarian, whose primary
responsibility, sad for him, was actually getting the data out of the
old system \-- which is not as daunting as it sounds, but it's kind of a
pain. And we also had another developer, program manager who was on it
because we weren't sure at the time what kinds of programming skills we
were going to need. We had a reference librarian, we had a cataloging
librarian without whom I think none of this would have been possible.
When it comes to discussions about what's indexed where and how and what
kind of fields you have, you cannot have pure IT people sitting around
having these conversations.

I'm very proud of the three semesters of cataloging I took in library
school, but it's not an area in which I can show any great expertise in
a research library. And then a member of our Digital Library Initiatives
Department, who was a librarian, who before he became a librarian
actually worked at Amazon and had a lot of experience with interface
design and was responsible for a lot of the mockups and wire-frame of
the design. As I said, the timeline was very quick. We started in the
spring, we acquired it in the summer; October to January was our design
and development. And the thing that we kept telling ourselves all along
is that it doesn't have to be perfect, okay? Perfect is the enemy of the
good. And one of the other nice things was that when it comes to online
library catalogs the bar is so low that it's very easy to get over.

\[laughter\]

So you know, the secret to success is to aim low, I guess is the lesson
there.

Identifying those appropriate facets \-- this was a real challenge in
the system design; whether we kept Library of Congress Subject Headings
together, whether we broke them apart. We also had to integrate two
independent data systems, so we still had our old integrated library
system and we had the Endeca interface and we had the Web2 SirsiDynix
public interface, so it was really three systems that we were putting
together. And we found that unique identifiers become very important in
a system design like this. We didn't have very good unique identifiers
in the system that we had, and so we had to do some extra legwork on
that. Since we created the Web application that the users interact with
we had the opportunity and the responsibility of designing the interface
from scratch. There was no need to work within a templating system; that
was provided by Endeca, but we didn't have to work within that.
Basically we just got the APIs for the system that we had. We used
wire-framing and prototyping to help make decisions about how the
interface \-- and when asked how did we do it from October to January,
there were many discussions about the interface and what it looked like
leading up to that actual use of the software.

This was our old catalog, much maligned even by the members of the team
who knew that I created it. We had done some usability testing on it, we
knew that it had too many boxes, there was too much shading, there were
all these attempts to kind of break up records and things like this, and
it had not progressed very far. And so we wanted to do some changes.
This is an example of the first version of our wire-frame that ignored
things like the complexity of including serials and the hit list.
Continuing resources, we're looking at the influences like
RedLightGreen, Amazon, Google, other kinds of interfaces that we could
copy from. And this was the eighth and final wire-frame. We introduced a
desire to have a brief view and a full view, which included actual
holdings of the item, giving that choice to the user to choose which one
they wanted. Also, we tackled the, the final wire-frame tackled the
complexity of continuing resources and the ambiguity of online resources
by creating unique displays for each type of record. So we're doing, we
were able to do a lot of work in the interface itself rather than on the
data end.

The holdings are organized by library, which makes sense to the user and
prevents duplicate display of library's, library holdings. And as you
saw in the demo, this is very similar to how our production interface
wound up looking. Some more challenges, the data-ish data. Let me
convince you that nobody outside the world of libraries knows what in
the world MARC-8 is. And I mean nobody.

\[laughter\]

I've been looking. I've actually, I've actually, lately I've been
seeking a programmer who's graduated in this century who can understand
a MARC record and make any kind of sense of it, which is another
challenge that I like to do. So we had this problem of taking our MARC
data \-- we were not in Unicode at NC State, but taking that MARC-8
encoding and putting it into something that a system that's not used to
libraries can deal with.

I know you all are much more familiar with this display of a MARC record
than probably most people are. But this is a raw MARC record in all of
its glory, and so when we showed this to Endeca I think they had seen
other pieces of data like this, but usually they saw transformed data.
And we had lots of discussions about MARC XML and things like that. The
folks at Endeca said, "Well, you know, you can use MARC XML if you want
to, but it just adds a lot of tagging and it'll just take longer to
index the files if you add all that tagging in there," so we could just
take some flat files. And so what we wound up with looked like this,
which looks an awful lot like a MARC record, but it was just a much more
simplistic MARC file, which includes \-- at the bottom you can see 999
fields, the actual item data for the things that are associated with the
bibliographic records.

One of the things that we said all along is that sunlight is the best
disinfectant. And so as we put up a faceted view of all of our data we
saw lots of problems with our data that we had never seen before. Not
lots of problems, because our cataloging is impeccable, as I'm sure
yours is as well.

\[laughter\]

But we did have some data issues that were revealed. And we were able to
go in and kind of do some global searching and replacing. They were
subfield Y, subfield V, subfield Z issues that had never been dealt
with, but actually being able to see it. When I was doing the demos for
this system for staff \-- so as we were going live we were demo-ing all
along, and we were in the middle of a renovation project. And all of our
tech services staff were offsite in one building, and they moved back;
they were welcomed back into the library.

\[laughter\]

So I went out there to give the presentation to them and you know,
everybody was kind of skeptical, a little bit nervous, kind of excited
about the online interface, but when I showed it to the catalogers and
acquisitions folks they were elated. They were by far the most excited
group of people of anybody that I showed it to, and that was, that was a
really kind of a nice moment with the system. And then one of the other
challenges, as I think I alluded to earlier, is that there is a
challenge for relevance ranking for bibliographic data. I challenge you
to search the literature for anything on relevance ranking and
bibliographic citation data. There's lots of literature on full text,
article, book length, but very little as regards bibliographic data. And
so this has been kind of an uncharted territory experiment for us.

And then you have the challenge of the actual data. The historic catalog
has fewer, often more general, less specific subject headings; this was
a problem that we encountered. We had inconsistent application of
subject headings, which I'm sure was a very small problem. Subject tools
like LCSH are not hierarchical, and so how do you deal with that when
presenting them in a faceted view? Some MARC data 5xx note fields are
not granular enough sometimes. A lot of useful bibliographic data is not
in a very good, controlled form. This was something I learned in working
with electronic resource management systems; would that we had authority
to control for publishers other than the LC name authority file, but a
nice kind of easy-to-use publisher field. And then the real bottom line,
which is other humans can read our data. We can do lots to make our data
human readable, but it's very difficult for other machines to read our
data.

So, some outcomes. One of the first things that we wanted to figure out,
like I mentioned, that was important to us: is it better? Is the Endeca
catalog more relevant? And so we did a somewhat objective, somewhat
subjective experiment where we decided what constituted relevance of a
result. And this was based on presence of the word in things like title
and subject, and looked at our top 100 topical searches where we took
\-- actually it was random \-- 100 topical searches from one month in
Fall 2005, our busiest month in the semester, and looked at the results
and found that \-- you see this jump from 40 percent highly relevant in
our old OPAC and 31 no hits \-- this is what happens when you don't have
spell correction and automatic stemming \-- versus 68 percent highly
relevant in our Endeca catalog. So, a 70 percent increase in the
relevance.

And again, this was somewhat arbitrary, but we thought telling. One
example of this, I'll just give you one quick example, that when we did
a search for "marsupial" in our old catalog we got 78 results. They were
the top five \-- I'm sorry, in the Endeca catalog \-- of the 78 results,
the top five all had marsupial or marsupials or Marsupialia in the title
or subject headings; that was some of the stuff coming from automatic
stemming. In our old catalog none of the top five had marsupial in the
title or subject heading. And in fact the top two results were tributes
to Malcolm C. McKenna and poisonous plants and related toxins.

\[laughter\]

So those were our top two results for that search. So that's just one
example of what we encountered.

We expected keyword searching to kind of take off. In our old system
title was the default search, not only because we suspected that a lot
of people were using the catalog for known item searching, but also
because we didn't want to present keyword as a default search because it
worked so poorly, and so we were expecting this kind of thing. The
multifield, you can see how it kind of trails off; that's from the
advanced search. Multifield searching accounts for about 1 percent of
the searching, and I think 99 percent of that one percent is author,
title, so people trying to do "Twain, Finn," things like that.

This is the "search and navigation." So we see that two-thirds of our
patrons \-- this is between last summer and the beginning of this year
\-- two-thirds of patrons coming in to use the catalog are still just
doing pure search. They come in, they type some words into the catalog,
they walk away. They might browse forward and backward, but they're not
doing any refinement, they're not choosing any of the facets, any of the
LC Classification; they're just doing a search. Twenty-five percent are
actually doing a search and then choosing one or more of the
facet-guided navigation features in the system. And 8 percent are doing
pure navigation; so that's that browse screen that I showed you. The
vast majority of that is new book browsing. Not a whole lot of it is the
LC Classification browsing, but actually just browsing by the fact that
the book is new.

I'm going to come back to that screen in a second because I think
there's something important here. As you break down the navigation by
dimensions you see the things that people are choosing. The important
caveat here is that you see the subject topic and LC Classification are
the vast majority of the refinements that are chosen. Now, those are
weighted because they have many more values, okay? So if I do a search
I've got many more values in a subject heading than I do in the number
of libraries that I can choose from. Or in the case of newness, you can
see the new accounts for 10 percent. Well, there's only two values. It's
either new or it's not. Okay, so that's why the subject topic and LC
Classification look somewhat weighted.

But I think they're worth mentioning, because if you take them and then
juxtapose them against that original screen that I had that showed that
33 percent, a third of our searches include navigation \-- that's the
search and navigation or pure navigation \-- then 20 percent of our
overall searching, nearly half of our navigation is subject-based
navigation, okay? Twenty percent overall, nearly half of all the guided
navigation that's going on is subject-based. Did that sink in? It's not
a big support for Library of Congress Subject Headings as much as it is
support for subject-based browsing of collections. Here's a different
view, a bar chart instead of a pie chart of the refinements that people
are using, but this is a little bit more interesting when you re-sort it
by the way that we're presenting the facets, okay? This is the order
that they're presented in the user interface. We would love somebody to
tell us what to do with this information \[laughs\].

As I mentioned, there was no consensus about what order we put these
things in. And so one of the things that we've talked about is we were
very surprised by author, which there was some general consensus to put
at the bottom of refinements because they're very particular. But it
gets used a lot. The anecdotal data that we have on this is that
refinement by author is really great for corporate authorship, which is
another thing that is very hard to guess, but when you type "chemistry"
and then you see the American Chemical Society as an author and want to
choose that refinement, it gets a little bit of usage there. So we've
had discussions about whether we should re-sort these, whether we should
move them, whether we should do something else with them; lots of
discussion about the LC Classification being up at the top of the
screen.

Some usability testing which we reported on \-- my colleagues Kristin
Antelman, Emily Lynema and I wrote a article for" Information Technology
and Libraries." We're not really touting the usability test very much
anymore; it was done rapidly and only with 10 users; five using the old
system, five using the new system. We desperately need to do some
retesting of the system. But based on some of the testing that we did
and some of the feedback that we got in that testing and actual data,
one of the things that we determined was that relevance ranking really
is key. Looking at March 2006 we found that 13 percent of users
continued to page two. Okay, so this was an indication to us that having
highly relevant results at the top of the list was very important. We
found that the faceted navigation is intuitive even for students who
didn't use it \-- this was some of the usability feedback \-- even if
they didn't want to use it, they understood what it did because it's out
there.

If you go out to Circuit City and you do a search for a laser printer
under \$500, you know, from this particular product manufacturer, you
can see that people are used to this. We found that we had to beware of
library jargon. We thought "LCSH" or "subject heading" was jargon; we
found that the word "keyword" is jargon. They didn't know what it meant.
Because of the mistrust that they had for the old system, a lot of them
wanted to use "subject keyword." They really wanted that to work; they
thought that it would be more precise if they did "subject keyword." But
since that's limited to LCSH, we put "LCSH subject" in there to kind of,
you know, steer them away from it unless they actually knew what they
were doing. And then we found that the user experience is influenced --
user behavior is influenced by their previous experience. So there was a
built up distrust for the old system that we didn't even realize until
they started using the new one. I think we've gotten over some of that.
The jargon is the big one; I'm still waiting for somebody from Library
of Congress to tell me what "command keyword" is \[laughs\]. So I
thought "keyword" was jargon; when I saw "command keyword" I thought,
"Oh my gosh, what do I do with that?"

This is some of our user reaction. The last one to me is just my
absolute favorite: "I found myself searching the catalog just for fun."
A non-librarian saying that is just remarkable, and that an
undergraduate in statistics would say it was even more remarkable. So
this is just some of our fun, anecdotal reaction to the system. So the
future that I try to posit as challenges and opportunities, because
we're not saying the system is perfect \-- I said at the onset that one
of our *raisons d'etre* was that it didn't have to be perfect as we were
building it.

So we have lots of challenges, all the fun of that uncharted territory
that I mentioned. These systems are new. You know, we've been using ours
for 18 months and I don't know what other kinds of aggregate data people
are looking at for how they are used. Another challenge is that we're
expanding the scope of the implementation to include the rest of the
Triangle Research Libraries Network (TRLN). In fact, I was in an all-day
meeting yesterday about this implementation with Duke, North Carolina
Central and UNC Chapel Hill, so we're taking our 1.7 million records and
making it 10 million records almost overnight, and we're going to see
how that goes. Basically we'll have one large dataset with five
different interfaces on top of it; one for the collective and one for
each individual school is a possibility.

We want to experiment with expanding the use of the digital collections
and potentially nonmetadata collections. One of the things that the
Endeca software does is relationship discovery and term discovery. So
you can actually index things that don't have any metadata, but try to
extract things like places and names and subject terms to get kind of a
clustered view of the things that are there. We need to figure out how
to more adequately expose our authorized headings. And another way of
putting this is what we're calling \-- I'm not going to go through all
of these challenges, but the one I want to focus on is the one we've
come to call the Revolutionary War problem; subject access and natural
language entry points, which is the next challenge in the next
generation of online catalogs that I don't think has been solved yet.

So I'm going to give you an example of what I mean by this. If I go into
the Endeca system and I do a keyword search for Revolutionary War, I can
see that I get 856 matching items. That's great; I can refine it, I can
narrow it down by LC. But the thing that I know, it's not the whole,
it's not the whole problem. If I pull up one of those records, one of
those individual records, I see the highly intuitive Library of Congress
Subject Heading "United States - History - Revolution - 1775-1783."

\[laughter\]

I challenge anybody to have been able to name those dates if I hadn't
put them up on the screen. And if I click on that highly intuitive
subject heading, even if I knew it, I get a browse list, okay? So this
is what our system currently does. Now, the thing that I will tell you
that I think users do intuit wrongly is that that first entry that is
the full heading without any subdivisions has 268 entries.

If I were the average undergraduate I would assume that all the ones
under it are included in that list, and they're not. The real problem,
I've got 10 more pages of subdivisions for this particular subject
heading. So, do you remember the 856 hits that I had in that initial
search for Revolutionary War? If I had submitted that search as a
phrased subject keyword search I would have gotten the 3,084 matching
items that were represented in those 10 pages of subdivisions. And so
this is one of the discussions that we're having about how we take the
best of the controlled vocabulary and the best of the guided navigation
and put them into one interface. This doesn't even scratch the surface
of "see:" references.

Future opportunities \-- I listed it as a challenge, so I thought I
would list it as an opportunity, is the expanding of the scope of the
implementation to the TRLN libraries, the 10 million records. We want to
enrich the catalog through external Web services; we do not want to
become a big aggregate pile for all of this data. We want to take all of
the advantages of the semantic Web and get some of this data in. One of
the big problems that we have over this particular topic is both the
ownership of this data and the fact that very little of it is available
in an online format that machines could use and read and integrate with
other systems. We want to build a use case based cross application,
shopping cart, which is kind of a low-lying thing that everybody wants.
But again, like the Revolutionary War problem, the thing that I really
want to concentrate on here is to "free the data." This is a phrase that
Emily Lynema coined, and one of the reasons that we built this interface
to our catalog was to free the data that we had in the system.

So, Lorcan Dempsey calls this the spinach problem, the "eat your
spinach" message that we have about our library; "come to the library,
it is good for you." This is also against the backdrop of an old search
model, which takes a local starting point of most libraries and then
expands that search to something like the Triangle Research Libraries
Network, then might expand it even further to OCLC WorldCat, and then
into the world of information that's out there. I first heard this
reversal of fortune articulated by Marshall Breeding. I thought it was a
really good one because the new search model, which starts from the
world and is trying to get to the library, has this huge brick wall
between them. And so what we're basically trying to do is break down
that wall and have an interchange that goes back and forth. If people
can land at our resources, they deserve something tastier than spinach
once they get there. So the impetus to "free our data" came from two
requests.

One was, can we have an RSS feed for the catalog \-- really simple
syndication, something that we could put it into a different reader,
give me a different view of new books or a search that might have
something added to it along the line, and can we integrate this into
kind of a quick search application, which I'll show you in a minute. We
have a library Web site search tool that searches across the collection,
and we wanted to put search results from the catalog in that site
search. The reason we couldn't do it with our old system is because if
we just showed the first five it was very likely that they'd be
irrelevant. And so we wanted something that could have something a
little bit better. And then the initial plan was to just build a simple
little search box. You see these all the time now on our browsers;
something where we can offer to users a very easy way to download a
plug-in that would go into their browser.

And then in addition to Technorati and Google and Internet Movie
Database or whatever it might be, they can actually search the local
catalog. Well, it wound up to be much more than that, and my colleagues
Emily Lynema and Tito Sierra get all of the credit for this. It started
as part of the competition, the software competition from OCLC \-- which
was stolen from them; they came in third place \-- but they created this
great tool called the Catalog WS or Catalog Web Service, an API
dynamically querying the catalog and presenting it in different ways.
And so the real strategy behind this was open access to that data; an
XML schema that could be reused and modified via style sheets so that we
weren't writing new interfaces for different devices of different views.
It also enabled other developers in the libraries to build applications
using the catalog data, and avoiding the kind of bottleneck problem that
you have with developers.

And so Emily didn't have to work on every single Endeca improvement that
was coming down, and we could reduce some of that bottleneck. I'm going
to give you an example of some of these bottlenecks, because even though
she was involved in many of these she didn't do all of them. So the
first one was that RSS feed. So every search, every query that you do in
the system has an RSS button, and if you choose it \-- we have an
explanation of this \-- but if you choose it, basically what it's going
to do is give you an RSS, sort it by newness so that if you put this
feed into your system, anything new that comes in related to that query
will be added to your RSS feed. So this is the purpose of an RSS feed.
I've seen lots of search systems where they have an RSS button, you
download it, you get this static view of 10 search results in an RSS
reader. Well, what's the point of that? I want to see new things that
are added based on those parameters. So that's just one.

This is our quick search, our library site search. So if you typed a
search in the upper right-hand box that says site index, you now get in
it the top three hits, the top titles from the search results; the top
three. You can click through to see all of the results, you can actually
refine \-- we just give a glimmer of some of the refinements there that
you can choose. This is a bit of a digression, but I think it's worth
mentioning because this application, which is all built on open source
software, does a lot of other things. So you can see at the very top,
I've just typed a search for the word "digital," and so it has a best
bet: are you looking for the digital media lab? So we have a finite list
of things. We would see lots of searches for Lexus, L-E-X-U-S, Nexus,
N-E-X-U-S, so we knew certain things like if somebody typed "reserves,"
they were looking for course reserves, if they type "reserve book room,"
they were looking for reserves, so we could do these best bets.

We also have an FAQ database. So, that little database which is kind of
like an electronic rolodex, right? Are you looking for just quick
information on things? You can do an article search with subject
recommendations. So we've taken the search term that somebody typed and
we're recommending subjects based on that search term if they want to go
search databases in that subject area. So you can see the specialty
research databases that are related to the word "digital." Just this
past week we turned on our journal search, which is powered by our
homegrown electronic resource management system. This is a database of
all print and electronic journals that we own in our collection, so you
can see the first few titles there. And then, after all of that, you get
the general Web site search for the rest of the collection.

So, cool stuff. Mobile device searching. If you go to our homepage and
just type a little "m" after the URL for mobile, you'll see this thing
\-- which as rendered in a browser looks like this very mini, text-only
kind of interface. You can actually choose catalog search from that.
These interfaces were purely done with style sheets; they're still using
the Endeca application and the API calls. You can do a search and get
this nice little display, and then this is what it would look like on a
cell phone. Now, you might not ever picture yourself doing this, but we
have some very strong feelings that this is something \-- given the
popularity of the iPhone and the fact that it's crashing wireless
networks all over the country \-- that this is going to be something
that a lot more people are going to be doing with their systems.

This is a new book, from our new books wall. So we have a very nice
50-inch plasma screen in our newly renovated learning commons, and so we
can take that new book search, which you saw is just a general browsable
query, and actually go out and get the book jacket covers for all those
titles and present them as this kind of nice new book search, which
looks much nicer on a 50-inch plasma than it does when I render it on my
little screen here and then blow it up for you. This is a resource
builder list. So I was able to create a dynamic list on new books with
the subject "September 11 terrorist attacks," and so basically all I had
to do \-- this is a staff tool \-- I can go in and then put the URL from
my catalog \-- which is not session-based, it will always work \-- and I
can take that URL and say, "Show me everything with this query added in
the last three months, and give me kind of a preview of what that will
look like." So that we can actually have people embed things in other
Web pages that would be refreshed with new content, and they wouldn't
have to go back and type, you know, put in another reference for the
search, find another book jacket; they could just do it dynamically.

So I'm wrapping up here, and I'm going to go ahead and return to my
library systems puzzle. The fact is that our legacy systems and
technology have precluded us from fitting these pieces together. At
least, that's my opinion. To have any hope of doing so, we must rethink
what the pieces are and use newer technology to reintegrate them.
Whether this integration is physical, through newer monolithic systems,
or virtual, using Web services, data feeds or interoperability, doesn't
really matter to me, but relying solely on legacy systems and hacked
versions of interoperability is neither tenable nor desirable. And you
can see that I've purposefully left some jagged edges on this puzzle
because I've got to get at least 20 more years out of this library gig.

\[laughter\]

So I want to work on some other things, and I think there's lots of
other things to work on in this one little section of the puzzle.

And finally, if you haven't seen it already \-- I wish I could have
given this entire presentation, unfortunately I missed it at ALA \-- but
two of my colleagues, my boss Kristin Antelman and our principle
metadata cataloger Charley Pennell, who some of you might have heard of,
gave a great presentation that talked about a lot of these issues from a
data and cataloger perspective, and what we're going to do about some of
these problems, some of the hurdles that we face as a culture and as an
organization to get past them. So I just wanted to give that a quick
plug. Hopefully it will be up on the ALCTS Web site pretty soon. And
with that I'll thank you and point you to our project site. If you go to
our main Web page and go to Endeca you'll see this presentation
ultimately, or links to it; links to the recording, information about
the Endeca project and everything that went into it, and a little bit
more about the implementation that we did. Thank you very much.

\--end of webcast \--