Journal of the Text Encoding Initiative
Issue 5 | June 2013

TEI Infrastructures

The Evolution of the Text Encoding Initiative:
From Research Project to Research Infrastructure
Lou Burnard

Electronic version
URL: http://journals.openedition.org/jtei/811
DOI: 10.4000/jtei.811
ISSN: 2162-5603
Publisher
TEI Consortium
Electronic reference
Lou Burnard, « The Evolution of the Text Encoding Initiative: From Research Project to Research
Infrastructure », Journal of the Text Encoding Initiative [Online], Issue 5 | June 2013, Online since 21
June 2013, connection on 08 May 2019. URL : http://journals.openedition.org/jtei/811 ; DOI :10.4000/jtei.811

This text was automatically generated on 8 May 2019.
TEI Consortium 2013 (Creative Commons Attribution-NoDerivs 3.0 Unported License)

The Evolution of the Text Encoding Initiative: From Research Project to Resea...

The Evolution of the Text Encoding
Initiative: From Research Project to
Research Infrastructure
Lou Burnard

1. In the Beginning
1

The Text Encoding Initiative was born into quite a different world from that of today. In
1987, there was no such thing as the World Wide Web, and construction of the tunnel
beneath the English Channel had only just begun. A major political power called theUnion of Soviet Socialist Republics still existed, while in the UK, Margaret Thatcher's
government had just been reelected for a third time, and in the US the Senate rejected forthe first (and so far only) time a presidential nomination to the Supreme Court. In
academic life, it was still (just about) possible to finance an undergraduate degree on thebasis of government grants. A typical "home computer" cost about 1,500 pounds in theUK, had an Intel 80286 processor and up to 640 Kb of memory, with maybe up to 50 Mb ofstorage on its internal hard disk, and probably ran some version of Microsoft's ubiquitous
MS-DOS, unless of course it was a Macintosh. New machines were beginning to appear on
the market, some of them with nearly enough memory and processing power to run
Microsoft's new Windows operating system, or IBM's optimistically named "OS/2," also
launched in this year. And meanwhile in another part of the forest Steve Jobs was busy
imagining the Next computer, which would run something like Unix, but with aWindowing interface. However, any serious computing would still be done on yourdepartmental minicomputer (perhaps a VAX or a PDP) or your institutional "mainframe,"as the massive energy-hungry arrays of transistors and magnetic storage systems sold bysuch companies as IBM, Univac, Burroughs, ICL, or Control Data were known.

2

At the same time, much of the work done on those massive machines looks quite familiar
today. The process of digitization of the office environment had already begun in somescientific disciplines with software such as TeX, Scribe, or tRoff becoming dominant in

Journal of the Text Encoding Initiative, Issue 5 | June 2013

1

The Evolution of the Text Encoding Initiative: From Research Project to Resea...

the production and dissemination of research articles and documentation. The tide of
personal computers able (allegedly) to close the gap between the writer and the publisher
would soon engulf us as surely as Microsoft Word would replace the seemingly
unstoppable Word Perfect 4.2 (released in 1986). Such neologisms as "desktop
publishing," "expert systems," and "digital resources" began to appear in serious
academic journals, as well as dominating the discourse of fledgling online communities
such as the Humanist discussion list launched in 1987. The Internet already existed, as did
many theories about how it might be used "hypertextually," though the World Wide Web
was still barely an idea. Both in the research community and, increasingly, beyond it, thegoals of corpus linguistics and artificial intelligence alike had established a need to workon large-scale digitized textual resources just as the technologies to support such work
were beginning to appear. Launching a major revision of the Oxford English Dictionary,
Oxford University Press for the first time proposed to do so using computational
methods. Text-based disciplines of all kinds were beginning to imagine the possibilities
offered by computationally tractable corpora of source texts created by such projects as
the Thesaurus Linguae Graecae, the Trésor de la Langue Française, or the Dictionary of
Old English. And, given the rapidity with which one storage technology was alreadyreplacing another, new discourses concerning the best methods of guaranteeing longterm access to newly created digital resources, and about the necessities for open accessand platform-independent formats alike, were beginning to be heard.
3

Symptomatic of that discourse were two key meetings held in 1987: one, in April,organized by a group of practicing historians in Europe, debated the possibility of
establishing some kind of consensual standardization for the encoding of primaryhistorical source data in computer-readable form. A paper prepared for that workshop by
Manfred Thaller, and subsequently published in the influential journal Historical Social
Science, together with a collected proceedings volume edited by the French historianJean-Philippe Genet (see, among others, Thaller 1986; Genet 1988), may havestrengthened the case put forward to the US National Endowment for the Humanities forthe funding of a larger international workshop on the issue; or perhaps they just spurredon the organizers of that workshop. But in either case, the majority of the European
institutions represented at one workshop reappeared at the other, along with many other
scholars from North America and elsewhere in the world. The TEI's foundational event1
was held at Vassar College, Poughkeepsie, New York, immediately after the joint annual
conference in Toronto of the Association for Literary and Linguistic Computing and the
Association for Computers and the Humanities, twin venerable precursors of today's
Alliance of Digital Humanities Organizations.

4

However, the purpose of this paper is not simply to chronicle the history or foundational
myth of the TEI, but rather to try to answer a more interesting question. If the TEI is sovery old—preceding as it does the Web, the DVD, and widespread use of technologies suchas the portable phone, cable television, and Microsoft Word—and given that computer
related technologies are hardly renowned for their longevity, how is it that the TEI is stillwith us, and still occupying a significant position in the world of research, after nearly
thirty years?

Journal of the Text Encoding Initiative, Issue 5 | June 2013

2

The Evolution of the Text Encoding Initiative: From Research Project to Resea...

2. Transformations of The Text Encoding Initiative
5

As first conceived, the Text Encoding Initiative was an international research project
intended to define a kind of digital demotic, as indicated by its alternative expansion Text
Encoding for Interchange. In a world dominated by mutually incompatible formats, eachcomputer manufacturer could impose its own conventions for the structuring andrepresentation of textual data. This was a world in which some computers worked in
EBCDIC and others in ASCII, where even the number of bits in a byte could vary between
6, 8, and 16, and where, as a consequence, there were serious technical obstacles even to
the simple transfer of data files from one machine to another, to say nothing of the
difficulties posed by mutually incompatible and proprietary file formats. Nevertheless the
TEI announced that it would facilitate the creation, exchange, and integration of textual
data in machine-readable form, for all kinds of texts, in every human language, fromevery historical or social context. In the world before the Web, these were ambitious
goals. But this quixotic ambition was further compounded by the TEI's declared goal of
making its proposals accessible both to the novice, looking for guidance on wellestablished best practice, and to the expert, seeking to establish new practice in responseto new research goals. It was this latter objective which, in retrospect, gives the TEI its
distinctive nature, and which largely distinguishes it from other standardization efforts,
both those now forgotten, and those currently entrenched or in formation.

6

In its earliest stages, the TEI was a creature of a time of transition, when the notion of"humanities computing" was just beginning to invent itself as a form of interdiscipline, aspace within which information specialists, computer scientists, and traditional
humanists might meet, if only to trade secret handshakes and suspicious glances; it was aperiod in which the notion of the disciplinarity (or otherwise) of that interchange was not
yet more important than its simple existence. The founding parents of the TEI were, fromthe standpoint of traditional academic life, a very mixed bunch of people, who had for themost part foregone the safety of traditional career paths for the excitement oftransgressing disciplinary boundaries. Many of those who met that snowy weekend in
Poughkeepsie came from research teams or institutions on the fringes of traditional
scholarship, owing allegiances to computer science, linguistics, philology, lexicography,
or literary studies in many languages, but not centrally placed within any of thosedisciplines. What united them was an expertise in the creation and management of digitaltext, a vision of its future importance to the traditional disciplines, and a concern thatlack of standardization and commercial pressure might prevent the realization of thatvision.

7

The outcome of the conference was a set of eminently practical recommendations as tohow an extensible set of guidelines consistent with the goal of a universal text-encoding
scheme might be achieved. These "Poughkeepsie Principles," with commentary focusing
on implementation, have long been available from the TEI's website2 and we do notdiscuss them further here, though they repay reading for anyone curious about thetheory underlying the shape and content of the TEI scheme. Instead we propose to focus
more on the organizational and managerial issues which have determined its evolution.

8

It is instructive to compare the organizational structures of the TEI during its first decadeof existence with that which has come into being during its second decade. Initiallyconceived as a research project answerable to a self-selecting group of experts, the TEI of

Journal of the Text Encoding Initiative, Issue 5 | June 2013

3

The Evolution of the Text Encoding Initiative: From Research Project to Resea...

the 1990s manifested a typically centralized structure, in which all the work is done by asmall number of people under the control of a small executive authority (see fig. 1). The
TEI of the present decade has a more distributed structure in which the task ofmaintaining and developing the Guidelines is the responsibility of many different people
drawn from a loosely defined community (see fig. 2).

Figure 1. TEI organizational structure, 1991

Journal of the Text Encoding Initiative, Issue 5 | June 2013

4

The Evolution of the Text Encoding Initiative: From Research Project to Resea...

Figure 2. TEI organizational structure, 2012
9That is not, of course, to imply that the original TEI editors worked in isolation from themany scholarly communities that their work was intended to benefit. On the contrary,
the organization of work required the editors to spend much of their time extracting
proposals and requirements from a very wide range of experts, which could then be
subsequently organized into a coherent whole for appraisal and ratification by further
groups representing those communities. In all, nearly 200 experts from both sides of theAtlantic gave their time and energy to codifying their own practice in such a way as tofacilitate its integration with that of others.

10

During the first funding cycle (1992–93), the TEI's work was carried out by four appointed
committees. The Text Documentation Committee, populated by documentalists andlibrarians, produced recommendations that eventually became the TEI Header. The
Metalanguage and Syntax Committee, populated by computer scientists, maderecommendations on the formal metalanguage in which the TEI outcomes should beformulated. The other two committees divided the universe of possible objects for text
markup between them, along rather loosely defined conceptual lines: one committee,
Text Representation, was supposed to identify the "significant particularities" of written
texts, specifically those which needed to be made explicit in a marked-up document; theother, Text Analysis and Interpretation, was supposed to focus on the encoding of "added
value" brought to a text by an analytic tool, such as a parser or (indeed) a careful reader.
This opposition between analysis and representation typifies much contemporary
debate about the proper function of markup, and indeed the process of reading itself.

11

The Text Documentation committee may be credited with the idea, still useful today, that
the TEI Header is intended to act as a primary source of information for a digital
resource, in the rather specialist sense that the librarian community uses that phrase.
The title page of a printed book may not correspond exactly with a catalogue entry for

Journal of the Text Encoding Initiative, Issue 5 | June 2013

5

The Evolution of the Text Encoding Initiative: From Research Project to Resea...

that book, but cannot be ignored during the production of one. Given that most digital
resources were going to be produced by non-cataloguers, the committee argued that the
TEI Header should make it possible for the non-expert to record whatever information
they deemed useful in such as way as to simplify, but not to replace, the task of thesubsequent cataloguer. This idea perhaps underlies much that professional cataloguers
find frustrating in the TEI header today.
12

The choice of SGML3 as a vehicle for expression of the TEI's recommendations was a fairly
obvious one for the Metalanguage Committee. This committee also established theprinciple that the use of this ISO standard was conditional, and that the TEI itself should
as far as possible be independent of any particular metalanguage syntax; the wisdom of
this decision and the consequent system architecture became apparent when, several
years later, the task of re-expressing the TEI in a new metalanguage called XML was takenon.

13

The Text Representation committee began by consolidating current encoding practiceacross a number of related disciplines to provide the essentials of the TEI scheme: thebasic components of textual objects, viewed as hierarchically organized containers, inwhich identifiable, possibly internally structured, components such as bibliographic
references, highlighted phrases, or proper names float in a kind of "soup" of plain text.
To these, the committee added recommendations for markup of textual variance andhypertextual links, amongst other topics reflecting its diverse membership. A recurrent
theme in its discussions was the desire to encode an idealized version of the text itself,
independently of its realization in a particular source, while at the same time wanting topreserve what was significant in that source. This debate found expression in a number ofpublished scientific articles4; it also probably underlies the TEI's notorious tendency toprovide both cake and the eating of it (or, as they say in France, both the butter and themoney for the butter). Nevertheless, as critics were quick to point out, whenever it
became hard or impossible to give equal time to both the presentational or visual and theanalytic or structural properties of a text, this committee, and therefore the TEI,
consistently came down on the side of the latter. Many of those at Poughkeepsie felt that
contemporary digital publishing systems, with their emphasis on "camera-ready" copy,
engendered a distraction from the true business of scholarship, as had been cogently
argued in a highly influential article published in the Communications of the ACM shortly
before the Poughkeepsie Conference.5

14

The Analysis and Interpretation Committee began with the rather ambitious goal ofproviding ways of encoding in a normative way the full range of linguistic analysis. In an
early working paper, Langendoen (1990) notes that amongst many other topics, it would
need to deal with:• Underspecification, uncertainty, multiple hierarchies• Phonology and prosody• Morphology and word-level tagging• Higher-level syntactic analysis• Structural ambiguity• Anaphora and Deixis• Idioms• Figures of speech (not for this cycle)• etc. etc.

Journal of the Text Encoding Initiative, Issue 5 | June 2013

6

The Evolution of the Text Encoding Initiative: From Research Project to Resea...

15

If this seems a touch ambitious, it should not be forgotten that this was the period during
which Europe's Language Engineering industries were born, and that part of the motive
behind the EU's funding of the TEI was precisely to formulate recommendations about
specific linguistic categories for the use of that industry.

16

The linguists represented on the A&I Committee did not, however, belong to a school inwhich such concepts as "noun" or "verb" were unproblematic: instead they formulated amore abstract system, a kind of metamodel of linguistic annotation based on feature
structure theory,6 which could be used to represent any kind of linguistic (or indeed nonlinguistic) annotation. This model eventually achieved wide acceptance in the language
engineering field and is, so far, the only part of the TEI to have been formally adopted asan ISO standard,7 but its power and generality of application were not immediately
appreciated by the computational linguist who just wanted to build a simple parser. To
meet such needs, the committee also proposed a range of generic segmentation andsynchronization mechanisms of various kinds.

17

Inevitably, the encoding needs and mechanisms identified by these two committees
frequently coincided in all but name. Despite the vigorous application of Ockham's razor,
the TEI often found itself proposing a range of different ways of encoding linguisticallymotivated segmentation and of recording analytic judgments, with more or less internal
structure, reflecting the diversity of the user community. This process continued further
with the appointment, during the second funding cycle, of a number of specialist working
groups, charged with testing the TEI proposals within specific research areas andidentifying any gaps. As with the larger committees of the first phase, the members ofthese specialist working groups were nominated by experts in the field, and their
meetings funded on the understanding that they would produce detailed proposals for
integration into the Guidelines, the first version of which—a modest 275-page volume
known as P1—had been distributed in print form in 1990. In the event, some of theseworking groups were content to assess the applicability of that draft to their own area ofexpertise, or to make general comments about their usability, but in many cases thiswork resulted in significant expansion of what was already proposed, and in a few cases
substantial entirely new material was developed, extending the scope and coverage of the
Guidelines considerably. One working group, for example, produced proposals for theencoding of transcribed speech (which had not featured at all in P1), while anotherproduced a complex tagset for the markup of existing print dictionaries. Meanwhile, theTEI editors revised chapter by chapter on the basis of the feedback received. Rather thandelay publication until the whole work was complete, the new version (P2) was
distributed in fascicle form during 1992, as each new chapter took shape.

18

The chairs of all the working groups and other significant contributors were invited tosign off on the editorial process at a four-day Technical Review Meeting, held at Eynsham
Hall near Oxford in May of 1993. After further revisions consequent on that review, thefirst complete version of the TEI Guidelines, known as P3, appeared at the international
SGML conference held in Montreux in May 1994. TEI P3 took the form of two substantial
green volumes, making up a 1300-page reference manual documenting and defining some
600 SGML elements which could be combined and modified in a variety of ways to create
specific SGML document type definitions (DTDs) for particular purposes. In 1994 the firstedition of TEI Lite also appeared. This was a simplified subset of the whole scheme, which
was originally written to accompany the first of many training workshops held in Chicago
in December of the same year. This Chicago "metaworkshop" was conceived as a training

Journal of the Text Encoding Initiative, Issue 5 | June 2013

7

The Evolution of the Text Encoding Initiative: From Research Project to Resea...

session for trainers, and did much to establish how the TEI is generally introduced to a
novice audience.
19

In the five years that followed, the proposals of the TEI established themselves, without
benefit of grant funding or centralized management, as a necessary part of theintellectual infrastructure. In the US, several influential university libraries began todeliver online versions of set texts and to host the textual databases needed for research.
Training in using the TEI for basic encoding appeared as a module on specialist librarycourses; research projects in their grant submissions would routinely add the phrase "wewill follow the TEI recommendations" (often followed by "but . . .") ; a new generation ofresearch assistants found it necessary to understand the difference between <div1> and<div>, or why there was a <pb> element but no <page> element. A detailed history of
the way in which this happened is hard to write, so rapidly did the TEI become a part of
the humanities computing ecosystem at this period. To some extent we may speculatethat there was no serious alternative for people who found inadequate the purely
presentational focus and lack of formality in the HTML of the time, or that in a world
where the technical infrastructure was rapidly changing, something so clearly groundedin traditional scholarship, and so clearly underpinned by theoretical rigor, offered atleast the promise of a long-term return on the serious costs of investing in the digital
future.

20

In his preface to a collection of working papers about the TEI published in 1995, Charles
Goldfarb, inventor of the SGML standard, remarked with typical prescience, "The vaunted'information superhighway' would hardly be worth travelling if the landscape were
dominated by industrial parks, office buildings, and shopping malls. Thanks to the Text
Encoding Initiative, there will be museums, libraries, theaters, and universities as well." 8
The TEI Recommendations were endorsed by the US National Endowment for the
Humanities, the UK's Arts and Humanities Research Board, the Modern Language
Association, the European Union's Expert Advisory Group for Language Engineering
Standards, and many other agencies that funded or promoted digital library andelectronic text projects. They had become part of the intellectual infrastructure of the
day.

21

For the purposes of this article, however, the key point to note is that by the end of thetwentieth century, although everywhere cited as if it had some kind of formal existence,
the TEI was no longer under the active care or management of anyone. Many of thosemost responsible for its original development had moved on to other challenges, mostnotably the principal editor Michael Sperberg-McQueen, who in 1996 had been appointed
co-editor of the World Wide Web's emerging standard XML. At a conference organized tocelebrate the TEI's tenth anniversary at Brown University in 1997, there was cake and ana cappella choir, but no consensus as to the organizational way forward. Did the TEI need
any kind of formal management? Was anyone interested in maintaining it into the future,or was it a great idea to be fixed in stone, until consensus emerged that its time had now
passed?

22

Ideas about an appropriate structure for continued maintenance and development of the
TEI had been discussed within the original Steering Committee of the project over two or
more years without practical result; the need for some kind of scientific council able to
make decisions about the future technical development of the Guidelines had also been
foreseen, and indeed a body not unlike the TEI's current Technical Council had held two
meetings. As a result of their work, a new and final version of P3 appeared online, with a

Journal of the Text Encoding Initiative, Issue 5 | June 2013

8

The Evolution of the Text Encoding Initiative: From Research Project to Resea...

small number of egregious errors corrected, and the addition of one new element. It was
already clear, however, that a very much more extensive program of work was needed tobring P3 up to date. The difficulty was in setting up a financial and organizational
infrastructure within which that extensive program of work might be undertaken.
23

The task was eventually accomplished after what might be caricatured as a kind of"management buyout." At a meeting held at King's College London in January 1999,
representatives of the three learned societies in whose name the TEI Guidelines hadoriginally been published, and representatives of four key academic institutions at which
the TEI was widely used and promoted, met and agreed to transfer ownership andmanagement of the Guidelines to a new entity: a non-profit international membershiporganization called the TEI Consortium, the constitution and structure of which were laidout in a detailed proposal submitted to the meeting by representatives from the
University of Virginia and the University of Bergen (see TEI 1999).

24

The goal of the TEI Consortium would be to establish a permanent home for the TEI as ademocratically constituted, academically and economically independent, self-sustaining,
nonprofit organization. It was clear that the TEI Guidelines had a major role to play in theapplication of new XML-based standards that were driving the development of textprocessing software, search engines, web browsers, and indeed the Web in general. To reexpress the TEI's SGML schema fragments as XML was not particularly difficult, but the
TEI also needed to adapt to a technical environment completely transformed by the riseof the Web. The first act of the Consortium was to publish a new version of the Guidelines,
known as TEI P4. This was a largely unmodified version of TEI P3, except for its reexpression using XML syntax rather than SGML. At the same time, the new Consortium
declared its intention of embarking on a major revision of the entire Guidelines which
would bring them up to date, extending and improving their coverage, and also radicallytransforming the way in which they would be maintained and developed in the future.

25

In a presentation given at the first annual meeting of the members of the new TEI
Consortium, held in Pisa in November 2001, Michael Sperberg-McQueen (2001) itemized
the things that in his view the "old" TEI had done right. These included some interesting
technical aspects: for example, the TEI interchange format, a subset of SGML that
corresponded very closely with what subsequently became XML; the TEI extended pointersyntax, which approximated to what became XPointer and XPath; the fact that DTDs are
not written, but generated using an additional layer of abstraction which integrates theformal and informal or documentary aspects of the architecture—the ODD system. But
Sperberg-McQueen also underlined the importance of the work invested "to earncommunity buy-in," in his phrase, thus re-asserting the essentially community-driven
aspects of the project.

26

At the same meeting, Lou Burnard and Syd Bauman listed a dozen or more possible areasin which the Guidelines needed enhancement. These included topics evidently in need of
update because the world had moved on, such as character encoding following thedefinition of Unicode, and also areas where other research communities had already been
active, independently of (but perhaps inspired by) the TEI, such as the detailed proposals
for the encoding of manuscript descriptions arising out of the European MASTER project.
But for the most part they presented an ambitious shopping list of items in which theyfelt work was needed, with no promise that the work would actually be done without
support from the user community.

Journal of the Text Encoding Initiative, Issue 5 | June 2013

9

The Evolution of the Text Encoding Initiative: From Research Project to Resea...

27

Beyond simply adding yet more elements and more recommendations, it was clear that
the TEI's internal architecture needed substantial revision. The move to XML meant thatmany new technologies were now available, and many assumptions about document
processing which were entirely valid in 1994 needed to be rethought for the new digitalworld, of which the TEI wished to become a good citizen. In his report to the members in
2005, Christian Wittern, as chair of the Council, likened the process to the restoration of avenerable piece of architecture: "When I grew up in the small southwest German town of
Tübingen, conservation and re-creation of the medieval town centre was a major project
there. This resulted in many a half-timbered house being completely redone from within
without actually taking down the structure, but resulting in a totally new interior . . . .
Something similar is happening at the moment with P5 . . ."928

During the TEI's first two development phases, working groups had communicatedextensively by e-mail, circulating and discussing drafts at a leisurely pace punctuated by
expensive if productive face-to-face meetings funded by the TEI. The working
environment for this new phase was rather different: for most working groups, and mostnotably the TEI Council itself, telephone conferencing became the norm, with only very
occasional face- to-face meetings funded by the TEI. Nevertheless, when the first version
of TEI P5 was finally published in 2007, much of its new intellectual content was the resultof working groups10 independent of the TEI Consortium. The new chapters on manuscript
description, on the description of named entities, on the documentation of charactersand glyphs outside Unicode, and on the evolution of the ODD system were all created in
the same way: a group of enthusiastic and largely self-selected experts worked up andtested proposals, which were then consolidated within the new Guidelines for ratificationby the TEI Council. For the most part, the work of such groups was funded by specific
grants, by institutional benevolence, or by individuals working on their own time. As of
TEI P5, the TEI consciously transformed itself into a classic open source project, not only
making all of its inner workings openly accessible to the public at large (or at least thoseparts of the public unintimidated by the Sourceforge user interface) but also becoming
reliant on community input both to define and to execute all of its future development.

29

For example, at release 1.2.0 of the TEI P5 Guidelines, a substantial set of proposals wasintroduced to support encoders wishing to represent the visual appearance or physicalmakeup of source documents, to align portions of a digitized page image with exacttranscriptions of the text on it, to assign parts of a transcription to documented stages inthe evolution of a text, and a number of other facilities typifying the discipline of
documentary editing, or édition génétique. These extensions did not come about because
the TEI Consortium identified a need for them, applied for a grant, chartered a working
group, and managed the process. On the contrary, they came into being because anumber of existing TEI users, dissatisfied with the current state of the TEI, identified aneed, obtained funding for a wide-ranging consultative exercise, formulated proposals,
and then worked with the TEI Council to ensure their eventual inclusion into the
Guidelines. This is not an isolated example, but typifies the way in which the TEI has now
become better able to evolve in response to the changing priorities of the researchcommunity.

30

That plasticity has also been facilitated by a number of technical developments, inparticular in the evolution of the current TEI processing architecture. Even in its original
SGML manifestation, the TEI was conceived as a modular system, which could becustomized to suit the needs of particular research communities, enabling them to define,

Journal of the Text Encoding Initiative, Issue 5 | June 2013

10

The Evolution of the Text Encoding Initiative: From Research Project to Resea...

for example, a schema containing all and only the elements of importance to them, to add
new elements, or to modify existing ones. However, to carry out such modifications
successfully required a degree of technical knowledge significantly beyond that of theaverage digital humanist, even with the availability of a web interface (the so-called Pizza
Chef) designed to simplify the task. With the shift to XML, the creation of such tools
became both simpler and more essential. With the advent of TEI P5 in particular, users of
the TEI are increasingly expected to desire to engage with the full sophistication of the
TEI architecture rather than to rely on precompiled one-size-fits-all subsets such as TEI
Lite. This in turn has led to the development of more sophisticated schema generation
tools such as Roma, which greatly reduce the complexity of developing a customization.
31

The TEI Guidelines are, of course, written and maintained as a large TEI-conformant
document, from which schemas in various formal languages (RELAXNG, DTD, and W3C
Schema) and documentation in various formats, ranging currently from PDF to EPUB, are
all automatically generated using a suite of XSLT stylesheets developed and maintainedby Sebastian Rahtz. Although developed primarily for the TEI's own internal needs, thissame suite of XSLT stylesheets is sufficiently generically designed that it has come to be
regarded as an integral part of the TEI—even though the TEI was originally designed as an
abstraction independent of any particular processing model. The free availability of thissuite of stylesheets, and its necessary maintenance in tandem with the Guidelines
themselves, has greatly facilitated the development of TEI-aware systems by a newgeneration of web developers and programmers. To take two particular examples:oXygen, a popular commercial XML development tool, uses them to offer on-the-fly
visualization of any TEI document; and the Oxgarage web application packages them toprovide basic conversions (in both directions) between many different formats, including
several popular word-processing formats and TEI XML.

32

Each new version of TEI P5 (since 2007, there has been a new release of P5 twice a year) is
the consequence of an ongoing revision process, in which outstanding errors or feature
requests submitted by the general public are considered and acted upon by the Council.
As with many other open-source projects, the number of different people active at any
one time is relatively small, but the essential point is that the TEI itself is not the only
agency managing the review or proposing extensions. A TEI orthodoxy undoubtedly
exists, but it does not control the evolution of the Guidelines, and is constantly undercritical review by the user community, via mechanisms such as the TEI discussion list, theSourceforge Trackers, and debate on the TEI Council list, all of which are publicly
accessible. A quick glance at the Sourceforge site shows that this activity is increasing:
between 2005 and 2010, visits to the site averaged a million page impressions per year,
while between 2010 and 2013, the figure is nearer four million a year. Looking at feature
requests and bug reports alone, in 2011 the Council reviewed a total of 170 tickets; in 2012
the number increased to 245.11 The annual meeting of the TEI membership, initially alegal obligation for the TEI Consortium, has of recent years been transformed into an
open TEI Conference, showcasing current debate about the use of the TEI within thedigital humanities research community, much of which eventually finds its way to alarger audience via journals such as the Journal of the Text Encoding Initiative.

Journal of the Text Encoding Initiative, Issue 5 | June 2013

11

The Evolution of the Text Encoding Initiative: From Research Project to Resea...

3. TEI Today
33

There are, it has been noted, two kinds of unsuccessful standardization effort. One kind
fails because it is based on a theory which is not yet sufficiently mature for widespreaduse. The other kind fails because it addresses a community which has not yet achievedconsensus and within which any proposal therefore seems alien to at least some of its
intended beneficiaries. Today's TEI is engineered to avoid both kinds of problem. An
immature theory—for example a new element definition—can be incorporated into the
TEI model without perturbing the rest: that is one of many advantages inherent in the TEI
architecture. Hence, it is easy for ideas developed for the benefit of one project to bemade visible to a wider community, tested, evaluated, and, if appropriate, eventually
included in the standard. The notion of TEI conformance now forming part of the TEI
incorporates precisely this model: conformance is defined as expressing clearly whichparts of a model are to be understood in the TEI sense, and which parts are not. This does
not preclude the addition of new features or the modification of old ones (mutations in
the evolutionary sense); it requires only that they be clearly identified as such.

34

As to the problems of diversity within the target audience, the TEI has always sought tobe a truly international enterprise. In its first phase, this desire was manifested by the
careful geographical balance of working groups membership; more recently it has beenshown in the support for multilinguality built into the Guidelines themselves. Moresignificantly perhaps, the Guidelines today are hospitable to other standards. Where an
existing XML vocabulary such as SVG or MathML exists, the TEI schema generally prefers
to incorporate it (within its own namespace) rather than to reinvent it. And, at theconceptual level, there is scope for reexpressing the semantics of any TEI element set
using other conceptual formalisms (for example, the CIDOC Conceptual Reference Model
has been used to provide a mapping for almost all TEI elements).

35

For longevity, however, a standards initiative must also address the need for financial andadministrative support. The TEI has increasingly adopted an open-source–style businessmodel, in which its users are the key agents providing that support. The frontiers of the
TEI user community are, however, both ill-defined and large. Within the community, todate, a sufficiently large number of institutions have proved willing to provide small
amounts of regular funding to keep the infrastructure itself in existence. Opinions differ
notably on either side of the Atlantic as to whether that regular funding should be
provided by individuals or by collectivities, and consequently how its continuation should
best be ensured or its functions promoted; ensuring that continuity is the function of the
TEI's Board of Directors, who are elected by the members of the Consortium, withresponsibility to them and also the (currently non-voting) subscribers.

36

Fortunately, at least as far as concerns the maintenance and development of its coretechnical deliverables, the TEI organizational model is increasingly independent offunding. Final editorial decisions as to what does or does not change in the Guidelines aretaken by an elected body of experts, the TEI Council, but this is carried out in response toreports of bugs and suggestions for change which may come from individuals, from
special interest groups, or from working groups specially chartered by the TEI or others,
and which may have members even beyond the TEI user community. Although there areindividuals who sit on both the TEI Board and the TEI Council to ensure goodcommunication between the two, it is clear that the TEI needs both administrative andJournal of the Text Encoding Initiative, Issue 5 | June 2013

12

The Evolution of the Text Encoding Initiative: From Research Project to Resea...

technical expertise, and that there is an advantage in providing distinct forums for thetwo.
37

Arguably, with this change the TEI ceased being an academic research project and became
something different. To talk of it as a "community-based research project" (to use theofficially sanctioned phrase) conveys something of that difference but also obscures apart of it. Research projects come to an end: they have a goal, which may or may not be
achieved, and a fixed term. The TEI, however, has transformed itself into an open-ended
activity, which will continue to be usable for as long as it is felt to be useful. Its strengths
are enhanced by its community of users; its weaknesses can be corrected by them. In astrictly Darwinian sense, the TEI has evolved by fostering and profiting from mutations
that are considered beneficial to the user community, while ignoring those which are not.
Because it is felt to be useful to so many, and because its products underpin so much ofcurrent research activity, it seems not unreasonable to regard it as constituting in itself a
research infrastructure. If nothing else, its organizational model seems highly
appropriate for all such infrastructural institutions or initiatives.

BIBLIOGRAPHY
Burnard, Lou D. 1988. "Report on Workshop on Text Encoding Guidelines." Literary and Linguistic
Computing 3(2): 131–33. doi: 10.1093/llc/3.2.131.
Coombs, James H., Allen H. Renear, and Steven J. DeRose. 1987. "Markup Systems and The Futureof Scholarly Text Processing." Communications of the ACM 30(11): 933–47. doi:10.1145/32206.32209.
DeRose, Steven J., David G. Durand, Elli Mylonas, and Allen H. Renear. 1990. "What is Text,
Really?" Journal of Computing in Higher Education 1(2): 3–26. doi:10.1007/BF02941632.
Genet, Jean-Philippe, ed. 1988. Standardisation et échange des bases de données historiques: actes de latroisième table ronde internationale, Paris, 15–16 mai 1987. Paris: Editions du CNRS.
Ide, Nancy, and Jean Véronis, eds. 1995. Text Encoding Initiative: Background and Context. Dordrecht:
Kluwer.
Langendoen, Terry. 1990. Plan of Action for Remaining Work of A and I Committee. http://
www.tei-c.org/Vault/AI/air03.gml.
Langendoen, D. Terence, and Gary F. Simons. 1995. "Rationale for the TEI Recommendations for
Feature-Structure Markup." In Ide and Véronis 1995, 191–210.
Sperberg-McQueen, C. M. 2001. "The TEI is Dead: Long Live the TEI." Paper presented at the TEIMembers Meeting, Pisa, Italy, November 2011. http://www.tei-c.org/Membership/
Meetings/2001/tei1116alt.html.
TEI (Text Encoding Initiative). 1988. Design Principles for Text Encoding Guidelines. TEI ED P1.
Last revised January 9, 1990. http://www.tei-c.org/Vault/ED/edp01.htm.
———. 1999. An Agreement to Establish a Consortium for the Maintenance of the Text Encoding
Initiative. http://www.tei-c.org/About/consortium.html.

Journal of the Text Encoding Initiative, Issue 5 | June 201313

The Evolution of the Text Encoding Initiative: From Research Project to Resea...

———. 2005. TCR03: Report of the TEI Council to the Members Meeting. http://www.tei-c.org/
Activities/Council/Reports/tcr03.xml.
Thaller, Manfred. 1986. "A Draft Proposal for a Standard for the Coding of Machine Readable
Sources." Historical Social Research/Historische Sozialforschung 40: 3–46, repr. in Modelling Historical
Data, ed. Daniel Greenstein, Halbgraue Reihe zur historischen Fachinformatik, A11 (St.
Katharinen: Max-Planck-Institut für Geschichte in Kommission bei Scripta Mercaturae Verlag,
1991), 19–64
Wittern, Christian, Arianna Ciula, and Conal Tuohy. 2009. "The making of TEI P5." Literary and
Linguistic Computing 24(3): 281–96. doi:10.1093/llc/fqp017.

NOTES
1. See Burnard 1988 for a contemporary report on the event.
2. See TEI 1988.
3. Many resources are available providing information about SGML (Standard Generalized
Markup Language) and its successor language XML (Extensible Markup Language); Wikipedia is
as good a starting point as any.
4. For example DeRose et al. 1990.
5. See Coombs, Renear, and DeRose 1987.
6. A good introduction to the TEI's use of this formalism is provided by Langendoen and Simons(1995).
7. The two chapters of P5 which define feature structure representation and feature systemdefinition are copublished by ISO as ISO 24610-1:2006 and ISO 24610-2:2011 respectively.
8. Ide and Veronis 1995.
9. See TEI 2005.
10. See Wittern, Ciula, and Tuohy 2009 for a summary of the work undertaken.
11. These and other statistics on Sourceforge usage are available at https://sourceforge.net/
projects/tei/stats/.

ABSTRACTS
It is twenty-five years since the Text Encoding Initiative was first launched as a research project
following an international conference funded by the US National Endowment for the Humanities.
This article describes some key stages in its subsequent evolution from research project into
research infrastructure. The TEI's changing nature, we suggest, is partly a consequence of its
close and highly responsive relation with an active user community, which may also explain both
its longevity and its effectiveness as a part of the digital humanities research infrastructure.

INDEX
Keywords: humanities computing, history, infrastructures, text encoding

Journal of the Text Encoding Initiative, Issue 5 | June 2013

14

The Evolution of the Text Encoding Initiative: From Research Project to Resea...

AUTHORLOU BURNARD
Lou Burnard has been closely associated with the TEI since its inception, initially as European editor, andmore recently as an elected member of the TEI Board. He also plays an active part on the TEI Council.
Since retirement from Oxford University Computing Services, where he was responsible for thedevelopment of the Oxford Text Archive, the British National Corpus, and several other key projects inthe Digital Humanities, he has been working as a private consultant, most recently with the French
infrastructural organization TGE Adonis.

Journal of the Text Encoding Initiative, Issue 5 | June 201315