Actual implementation of original RoadMap 2000

See Notes below for interpretation of KSJ's comments

                                                               KSJ's comments

                                 Yr 1   Yr 2   Yr 3   Yr 4

Corpus development               ****   ****   ****   ****     LIMITED

Evaluation software
  development                    ****   ****   ****   ****     YES

Extractive summary,
 intrinsic evaluation
- single document
  - simple genre                 ****   ****   ****            YES
  - sophisticated genre                        ****   ****     NO

- multiple document
  - single genre, language       ****   ****   ****            YES
  - evolving                            (**)   ****            LIMITED 
(novelty)
  - multiple genres, languages                        ****     NO

Abstract summaries,                                            <<<<< see 
below, 1
 intrinsic evaluation
- single document
  - simple genre                 ****   ****   ****            YES
  - sophisticated genre                        ****   ****     NO

- multiple document
  - single genre, language       ****   ****   ****            YES
  - evolving                            (**)   ****            LIMITED 
(novelty)
  - multiple genres, languages                        ****     NO

Abstract, multidoc summaries,
 extrinsic evaluation
- TDT-specific evaluation                      ****   ****     NO

 question answering
- single question                              (**)   ****     YES
-multiple questions, single lang                      (**)     NO
-multiple questions, languages

Notes
-----

1 The RoadMap distinguished extracts from non-extractive abstracts. DUC so far
has only done abstracts in a very limited sense, basically at the level of
selected sentence manipulation: this is perhaps best called quasi-extractive
summarising.

2. In addition to the RoadMap specification, DUC has included some very
limited (simulated) extrinsic evaluation, in Yr 3 and 4, by usefulness
(single document) and responsiveness (multidocument).

3. YES only means that something has been done under this heading: it is
not an indicator of the extent or value of that something.