### 6.4.3 The "anndata" API class diagram

After opening/loading a file, its content is stored in a `sppasTranscription` object. A `sppasTranscription` has a name, and a list of `sppasTier` objects. Tiers can't share the same name, the list of tiers can be empty, and a hierarchy between tiers can be defined. Actually, subdivision relations can be established between tiers. For example, a tier with phonemes is a subdivision reference for syllables, or for tokens; and tokens are a subdivision reference for the orthographic transcription in IPUs. Such subdivisions can be of two categories: alignment or association.

A `sppasTier` object has a name, and a list of `sppasAnnotation` objects. It can also be associated to a controlled vocabulary, or a media.

All these objects contain a set of meta-data.

An annotation is made of 2 objects:

* a `sppasLocation` object,

* a list of `sppasLabel` objects.

A `sppasLabel` object is representing the “content” of the annotation. It is a list of `sppasTag` each one associated to a score.

A `sppasLocation` is representing where this annotation occurs in the media. Then, a `sppasLocation` is made of a list of localization each one associated with a score. A localization is one of:

* a `sppasPoint` object; or

* a `sppasInterval` object, which is made of 2 `sppasPoint` objects; or

* an un-used `sppasDisjoint` object which is a list of `sppasInterval`.

API class diagram

#### Label representation

Each annotation holds a serie of 0..N labels. A label is also an object made of a list of `sppasTag`, each one with a score. A `sppasTag` is mainly represented in the form of a string, freely written by the annotator but it can also be a boolean (True/False), an integer, a floating number, a point with (x, y) coordinates with an optional radius or a rectangle with (x, y, w, h) coordinates with an optional radius value.

#### Location representation

In the `anndata` API, a `sppasPoint` is considered as an *imprecise value*. It is possible to characterize a point in a space immediately allowing its vagueness by using:

* a midpoint value (center) of the point;

* a radius value.

Representation of a `sppasPoint`
**Example**

The screenshot below shows an example of multimodal annotated data, imported from 3 different annotation tools. Each `sppasPoint` is represented by a vertical dark-blue line with a gradient color to refer to the radius value.

In the screenshot the following radius values were assigned:

* 0ms for prosody,

* 5ms for phonetics, discourse and syntax

* 40ms for gestures.

Example of multimodal data

## 6.5 Creating scripts with andata

### 6.5.1 Preparing the data

To practice, you have first to create a new folder in your computer - on your Desktop for example; with name "sppasscripts" for example, and to execute the python IDLE.

Open a File Explorer window and go to the SPPAS folder location. Then, copy the `sppas` directory into the newly created "sppasscripts" folder. Then, go to the solution directory and copy/paste the files `skeleton-sppas.py` and `F_F_B003-P9-merge.TextGrid` into your "sppasscripts" folder. Then, open the skeleton script with the python IDLE and execute it. It will do... nothing! But now, you are ready to do something with the API of SPPAS!

When using the API, if something forbidden is attempted, the object will raise an Exception which means the program will stop.

### 6.5.2 Read/Write annotated files

We are being to Open/Read an annotated file of any format (XRA, TextGrid, Elan, ...) and store it into a `sppasTranscription` object instance. Then, the object will be saved into another file.

```python
# Create a parser object then parse the input file.
parser = sppasRW(input_filename)
trs = parser.read()
```

```python
# Save the sppasTranscription object into a file.
parser.set_filename(output_filename)
parser.write(trs)
```

Only these few lines of code are required to convert a file from a format to another one! The appropriate parsing system is extracted from the extension of file name.

To get the list of accepted extensions that the API can read, just use `parser.extensions_in()`. The list of accepted extensions that the API can write is given by `parser.extensions_out()`.

**Practice:** Write a script to convert a TextGrid file into CSV (solution: `ex10_read_write.py`)
### 6.5.3 Manipulating a sppasTranscription object

The most useful functions to manage the tiers of a sppasTranscription object are:

* `create_tier()` to create an empty tier and to append it,

* `append(tier)` to add a tier into the sppasTranscription,

* `pop(index)` to remove a tier of the sppasTranscription,

* `find(name, case sensitive=True)` to find a tier from its name.

Below is a piece of code to browse through the list of tiers:

```python
for tier in trs:
    # below, do something with the tier:
    print(tier.get_name())
    # Search for a specific tier,
    # None is returned if not found.
    phons_tier = trs.find("PhonAlign")
```

**Practice:** Write a script to select a set of tiers of a file and save them into a new file (solution: `ex11_transcription.py`).

### 6.5.4 Manipulating a sppasTier object

A tier is made of a name, a list of annotations, and optionally a controlled vocabulary and a media. To get the name of a tier, or to fix a new name, the easier way is to use `tier.get_name()`. The following block of code allows to get a tier and change its name.

```python
# Get the first tier, with index=0
tier = trs[0]
print(tier.get_name())
tier.set_name("NewName")
print(tier.get_name())
```

The most useful functions to manage annotations of a sppasTier object are:

* `create annotation(location, labels)` to create and add a new annotation

* `append(annotation)` to add a new annotation at the end of the list

* `add(annotation)` to add a new annotation

* `pop(index)` to delete the annotation of a given index

* `remove( begin, end )` to remove annotations of a given localization range

* `is_disjoint()`, `is_interval()`, `is_point()` to know the type of location

* `is_string()`, `is_int()`, `is_float()`, `is_bool()`, `is_fuzzypoint()`, `is_fuzzyrect()` to know the type of labels

* `find( begin, end )` to get annotations in a given localization range

* `get_first_point()`, `get_last_point()` to get respectively the point with the lowest or highest localization

* `setradius (radius)` to fix the same vagueness value to each localization point

**Practice:** Write a script to open an annotated file and print information about tiers (solution: `ex12_tiers_info.py`)
### 6.5.5 Manipulating a sppasAnnotation object

An annotation is a container for a location and optionally a list of labels. It can be used to manage the labels and tags with the following methods:

* `is_labelled()` returns True if at least a `sppasTag` exists and is not `None`

* `append_label label)` to add a label at the end of the list of labels

* `get labels best tag()` returns a list with the best tag of each label

* `add_tag tag, score, label_index)` to add a tag into a label

* `remove_tag tag, label_index)` to remove a tag of a label

An annotation object can also be copied with the method `copy()`. The location, the labels and the metadata are all copied; and the 'id' of the returned annotation is then the same. It is expected that each annotation of a tier as its own 'id', but the API doesn't check this.

**Practice:** Write a script to print information about annotations of a tier (solution: `ex13_tiers_info.py`)

### 6.5.6 Search in annotations: Filters

#### Overview

This section focuses on the problem of *searching and retrieving* data from annotated corpora.

The filter implementation can only be used together with the `sppasTier()` class. The idea is that each `sppasTier()` can contain a set of filters, that each reduce the full list of annotations to a subset.

SPPAS filtering system proposes 2 main axis to filter such data:

* with a boolean function based either on the content, the duration or on the time of annotations,

* with a relation function between annotation locations of 2 tiers.

A set of filters can be created and combined to get the expected result. To be able to apply filters to a tier, some data must be loaded first. First, a new `sppasTranscription()` has to be created when loading a file. Then, the tier(s) to apply filters on must be fixed. Finally, if the input file was NOT an XRA, it is widely recommended to fix a radius value before using a relation filter.

```python
f = sppasFilter(tier)
```

When a filter is applied, it returns an instance of `sppasAnnSet` which is the set of annotations matching with the request. It also contains a 'value' which is the list of functions that are truly matching for each annotation. Finally, `sppasAnnSet` objects can be combined with the operators ']' and '&', and expected to a `sppasTier` instance.
Filter on the tag content

The following matching names are proposed to select annotations:

* 'exact': means that a tag is valid if it strictly corresponds to the expected pattern;

* 'contains' means that a tag is valid if it contains the expected pattern;

* 'startwith' means that a tag is valid if it starts with the expected pattern;

* 'endwith' means that a tag is valid if it ends with the expected pattern.

* 'regexp' to define regular expressions.

All these matches can be reversed, to represent does not exactly match, does not contain, does not start with or does not end with. Moreover, they can be case-insensitive by adding 'i' at the beginning like 'iexact', etc. The full list of tag matching functions is obtained by invoking `sppasTagCompare().get_function_names()`.

The next examples illustrate how to work with such pattern matching filter. In this example, `f1` is a filter used to get all phonemes with the exact label 'a'. On the other side, `f2` is a filter that ignores all phonemes matching with 'a' (mentioned by the symbol '~') with a case insensitive comparison (iexact means insensitive-exact).

```python
tier = trs.find("PhonAlign")
f = sppasFilter(tier)
ann_set_a = f.tag(exact='a')
ann_set_aA = f.tag(iexact='a')
```

The next example illustrates how to write a complex request. Notice that `r1` is equal to `r2`, but getting `r1` is faster:

```python
tier = trs.find("TokensAlign")
f = sppasFilter(tier)
r1 = f.tag(startwith="pa", not_endswith='a', logic_bool="and")
r2 = f.tag(startwith="pa") & f.tag(not_endswith='a')
```

With this notation in hands, it is easy to formulate queries like for example: Extract words starting by "ch" or "sh":

```python
result = f.tag(startwith="ch") | f.tag(startwith="sh")
```

Practice:: Write a script to extract phonemes /a/ then phonemes /a/, /e/, /A/ and /E/. (solution: `ex15 annotation_label_filter.py`).

Filter on the duration

The following matching names are proposed to select annotations:

* 'It' means that the duration of the annotation is lower than the given one;

* 'le' means that the duration of the annotation is lower or equal than the given one;

* 'gt' means that the duration of the annotation is greater than the given one;
* 'ge' means that the duration of the annotation is greater or equal than the given one;

* 'eq' means that the duration of the annotation is equal to the given one;

* 'ne' means that the duration of the annotation is not equal to the given one.

The full list of duration matching functions is obtained by invoking `sppasDurationCompare()`. `get_function_names`

Next example shows how to get phonemes during between 30 ms and 70 ms. Notice that r1 and r2 are equals!

```python
tier = trs.find("PhonAlign")
f = sppasFilter(tier)
r1 = f.dur(ge=0.03) & f.dur(le=0.07)
r2 = f.dur(ge=0.03, le=0.07, logic_bool="and")
```

**Practice:** Extract phonemes 'a' or 'e' during more than 100ms (solution: `ex16 annotation_dur_filter.py`).

## Filter on position in time

The following matching names are proposed to select annotations:

* `rangefrom` allows to fix the begin time value,

* `rangeto` allows to fix the end time value.

Next example allows to extract phonemes 'a' of the 5 first seconds:

```python
tier = trs.find("PhonAlign")
f = sppasFilter(tier)
result = f.tag(exact='a') & f.loc(rangefrom=0., rangeto=5., logic_bool="and")
```

## Creating a relation function

Relations between annotations is crucial if we want to extract multimodal data. The aim here is to select intervals of a tier depending on what is represented in another tier.

James Allen, in 1983, proposed an algebraic framework named Interval Algebra (IA), for qualitative reasoning with time intervals where the binary relationship between a pair of intervals is represented by a subset of 13 atomic relation, that are:

* distinct because no pair of definite intervals can be related by more than one of the relationships;

* exhaustive because any pair of definite intervals are described by one of the relations;

* qualitative (rather than quantitative) because no numeric time spans are considered.

These relations and the operations on them form "Allen's Interval Algebra".

Pujari, Kumari and Sattar proposed INDU in 1999: an Interval & Duration network. They extended the IA to model qualitative information about intervals and durations in a single binary constraint network. Duration relations are: greater, lower and equal. INDU comprises of 25 basic relations between a pair of two intervals.
`anndata` implements the 13 Allen interval relations: before, after, meets, met by, overlaps, overlapped by, starts, started by, finishes, finished by, contains, during and equals; and it also contains the relations proposed in the INDU model. The full list of matching functions is obtained by invoking `sppasIntervalCompare()`. `get_function_names()`.

Moreover, in the implementation of `anndata`, some functions accept options:

* `before` and `after` accept a `max_delay` value,

* `overlaps` and `overlappedby` accept an `overlap_min` value and a boolean `percent` which defines whether the value is absolute or is a percentage.

The next example returns monosyllabic tokens and tokens that are overlapping a syllable (only if the overlap is during more than 40 ms):

```python
tier = trs.find("TokensAlign")
other_tier = trs.find("Syllables")
f = sppasFilter(tier)
f.rel(other_tier, "equals", "overlaps", "overlappedby", min_overlap=0.04)
```

Below is another example of implementing a request. Which syllables stretch across 2 words?

```python
# Get tiers from a sppasTranscription object
tier_syll = trs.find("Syllables")
tier_toks = trs.find("TokensAlign")
f = sppasFilter(tier_syll)

# Apply the filter with the relation function
ann_set = f.rel(tier_toks, "overlaps", "overlappedby")

# To convert filtered data into a tier:
tier = ann_set.to_tier("SyllStretch")
```

**Practice 1:** Create a script to get tokens followed by a silence. (solution: `ex17_annotations_relation_filter1.py`).

**Practice 2:** Create a script to get tokens preceded by OR followed by a silence. (solution: `ex17_annotations_relation_filter2.py`).

**Practice 3:** Create a script to get tokens preceded by AND followed by a silence. (solution: `ex17_annotations_relation_filter3.py`).

## 6.6 More with SPPAS...

In addition to `anndata`, SPPAS contains several other API. They are all free and open source Python libraries, with a documentation and a set of tests.

Among others:
* `audiodata` to manage digital audio data: load, get information, extract channels, re-sample, search for silences, mix channels, etc.

* `calculus` to perform some math on data, including descriptive statistics.

* `resources` to access and manage linguistic resources like lexicons, dictionaries, etc.
# References

## 7.1 References

### 7.1.1 How to cite SPPAS?

For a general citation of SPPAS, use the reference below. A PDF version of this publication is available in the folder *documentation* of the package.

*Brigitte Bigi (2015). SPPAS - Multi-lingual Approaches to the Automatic Annotation of Speech*. In "the Phonetician" - International Society of Phonetic Sciences, ISSN 0741-6164, Number 111-112 / 2015-I-II, pages 54-69.

For a specific feature of SPPAS, like the automatic syllabification, the forced-alignment, or for the request system, please refer to the related specific publication. The full list of references and the corresponding PDF files are available on the author website, at the following URL: http://www.lpl-aix.fr/~bigi/publications.html

### 7.1.2 SPPAS software description

*Brigitte Bigi (2015). SPPAS - Multi-lingual Approaches to the Automatic Annotation of Speech*. In "the Phonetician" - International Society of Phonetic Sciences, ISSN 0741-6164, Number 111-112 / 2015-I-II, pages 54-69.

*Brigitte Bigi, Daniel Hirst (2013). What's new in SPPAS 1.5?, Tools and Resources for the Analysis of Speech Prosody, Aix-en-Provence, France, pp. 62-65.*

*Brigitte Bigi (2012). SPPAS: a tool for the phonetic segmentations of Speech*. The eight international conference on Language Resources and Evaluation, Istanbul (Turkey), pages 1748-1755, ISBN 978-2-9517408-7-7.

*Brigitte Bigi, Daniel Hirst (2012) SSpeech Phonetization Alignment and Syllabification (SP-PAS): a tool for the automatic analysis of speech prosody*. Speech Prosody, Tongji University Press, ISBN 978-7-5608-4869-3, pages 19-22, Shanghai (China).
### 7.1.3 About linguistic resources

Brigitte Bigi, Abiola S. Oyelere, Bernard Caron (2020). **Resources for Automated Speech Segmentation of the African Language Naija (Nigerian Pidgin)**. Human Language Technologies Challenges for Computer Science and Linguistics, LNAI 12598, pp. 164-173, Springer, Heidelberg.

Mélanie Lancien, Marie-Hélène Côté, Brigitte Bigi (2020). **Developing Resources for Automated Speech Processing of Quebec French**. In Proceedings of The 12th Language Resources and Evaluation Conference, pp. 5323–5328, Marseille, France.

Brigitte Bigi, Bernard Caron, Abiola S. Oyelere (2017). **Developing Resources for Automated Speech Processing of the African Language Naija (Nigerian Pidgin)**. In 8th Language and Technology Conference: Human Language Technologies as a Challenge for Computer Science and Linguistics, pp. 441-445, Poznań, Poland.

### 7.1.4 About Search for IPUs

Brigitte Bigi and Béatrice Priego-Valverde (accepted). **The automatic search for sounding segments of SPPAS: application to Cheese! corpus** Human Language Technologies Challenges for Computer Science and Linguistics, LNAI, Springer, Heidelberg.

Brigitte Bigi, Béatrice Priego-Valverde (2019). **Search for Inter-Pausal Units: application to Cheese! corpus**. In 9th Language & Technology Conference: Human Language Technologies as a Challenge for Computer Science and Linguistics, pp. 289-293, Poznań, Poland.

### 7.1.5 About Text Normalization

Roxana Fung, Brigitte Bigi* (2015). **Automatic Word Segmentation for Spoken Cantonese**, The Oriental Chapter of COCOSDA (International Committee for the Co-ordination and Standardization of Speech Databases and Assessment Techniques).

Brigitte Bigi (2014). **A Multilingual Text Normalization Approach**, Human Language Technologies Challenges for Computer Science and Linguistics LNAI 8387, Springer, Heidelberg. ISBN: 978-3-319-14120-6. Pages 515-526.

Brigitte Bigi (2011). **A Multilingual Text Normalization Approach**, 2nd Less-Resourced Languages workshop, 5th Language & Technology Conference, Poznan (Poland).

### 7.1.6 About Phonetization

Brigitte Bigi (2016). **A phonetization approach for the forced-alignment task in SPPAS**, Human Language Technologies Challenges for Computer Science and Linguistics, LNAI 9561, pp. 515–526, Springer, Heidelberg. ISBN: 978-3-319-14120-6.

Brigitte Bigi (2013). **A phonetization approach for the forced-alignment task**, 3rd Less-Resourced Languages workshop, 6th Language & Technology Conference, Poznan (Poland).
Brigitte Bigi, Pauline Péri, Roxane Bertrand (2012). **Orthographic Transcription: Which Enrichment is required for Phonetization?**, Language Resources and Evaluation Conference, Istanbul (Turkey), pages 1756-1763, ISBN 978-2-9517408-7-7.

### 7.1.7 About Forced-Alignment

Brigitte Bigi, Christine Meunier (2018). **Automatic speech segmentation of spontaneous speech**. Revista de Estudos da Linguagem. International Thematic Issue: Speech Segmentation. Editors: Tommaso Raso, Heliana Mello, Plinio Barbosa, Volume 26, number 4, pages 1489-1530, e-ISSN 2237-2083.

Brigitte Bigi, Christine Meunier (2018). **euh, rire et bruits en parole spontanée : application à l'alignement forcé**. Actes des 32èmes Journées d'études sur la Parole, Aix-en-Provence, France.

Brigitte Bigi (2014). **The SPPAS participation to Evalita 2014**, Proceedings of the First Italian Conference on Computational Linguistics CLiC-it 2014 and the Fourth International Workshop EVALITA 2014, Pisa (Italy). Editors R. Basili, A. Lenci, B. Magnini. ISBN 978-886741-472-7. Volume 2. Pages 127-130.

Brigitte Bigi (2014). **Automatic Speech Segmentation of French: Corpus Adaptation**. 2nd Asian Pacific Corpus Linguistics Conference, p. 32, Hong Kong.

Brigitte Bigi (2012). **The SPPAS participation to Evalita 2011**, Working Notes of EVALITA 2011, Rome (Italy), ISSN: 2240-5186.

### 7.1.8 About Syllabification

Brigitte Bigi, Katarzyna Klessa (2015). **Automatic Syllabification of Polish**, 7th Language and Technology Conference: Human Language Technologies as a Challenge for Computer Science and Linguistics, pp. 262–266, Poznan, Poland.

Brigitte Bigi, Caterina Petrone, Leonardo Lancia (2014). **Automatic Syllabification of Italian: adaptation from French**. Laboratory Approaches to Romance Phonology VII, Aix-en-Provence (France).

Brigitte Bigi, Caterina Petrone (2014). **A generic tool for the automatic syllabification of Italian**, Proceedings of the First Italian Conference on Computational Linguistics CLiC-it 2014 and the Fourth International Workshop EVALITA 2014, Pisa (Italy). Editors R. Basili, A. Lenci, B. Magnini. ISBN 978-886741-472-7. Volume 1. Pages 73-77.

Brigitte Bigi, Christine Meunier, Irina Nesterenko, Roxane Bertrand (2010). **Automatic detection of syllable boundaries in spontaneous speech**, Language Resource and Evaluation Conference, pages 3285-3292, La Valetta, Malte.
### 7.1.9 About Repetitions

Brigitte Bigi, Roxane Bertrand, Mathilde Guardiola (2014). **Automatic detection of other-repetition occurrences: application to French conversational speech**, 9th International conference on Language Resources and Evaluation (LREC), Reykjavik (Iceland), pp. 2648-2652. ISBN: 978-2-9517408-8-4.

### 7.1.10 About analyses tools

Brigitte Bigi (2019). **Filtering multi-levels annotated data.** In 9th Language & Technology Conference: Human Language Technologies as a Challenge for Computer Science and Linguistics, pp. 13-14, Poznań, Poland.

Brigitte Bigi, Jorane Saubesty (2015). **Searching and retrieving multi-levels annotated data**, Proceedings of Gesture and Speech in Interaction, Nantes (France).

### 7.1.11 About the API

Brigitte Bigi (2018). **Annotation representation and file conversion tool.** Contributi del Centro Linceo Interdisciplinare 'Beniamino Segre' ISSN 0394-0705), 137, pp. 99-116.

Brigitte Bigi, Tatsuya Watanabe, Laurent Prévot (2014). **Representing Multimodal Linguistics Annotated Data**, 9th International conference on Language Resources and Evaluation (LREC), Reykjavik (Iceland), pages 3386-3392. ISBN: 978-2-9517408-8-4.

### 7.1.12 Related references

#### Some results of analyses

Christine Meunier, Brigitte Bigi (to be published). **Propriétés phonétiques et lexicales dans les variations temporelles des phonèmes en parole conversationnelle**, Studii de lingvistică, SL11: « Variation phonétique en parole continue », coord.: Yaru Wu, Martine Adda-Decker, Lori Lamel, Ioana Vasilescu

Christine Meunier, Morgane Peirolo, Brigitte Bigi (2020). **La mobilisation du tractus vocal est-elle variable selon les langues en parole spontanée ?**, In Journées d'Etudes sur la Parole, 33e édition, pp. 433-441, Nancy, France.

Béatrice Priego-Valverde, Brigitte Bigi, Salvatore Attardo, Lucy Pickering, Elisa Gironzetti (2018). **Is smiling during humor so obvious? A cross-cultural comparison of smiling behavior in humorous sequences in American English and French interactions.** Intercultural Pragmatics. 10/2018; 15(4):563-591.

Brigitte Bigi, Roxane Bertrand (2016). **Laughter in French Spontaneous Conversational Dialogs**, Tenth International Conference on Language Resources and Evaluation (LREC 2016), pp. 2168-2174, Portoroz, Slovenia.
Laurent Prevot, Jan Gorisch, Roxane Bertrand, Emilien Gorene and Brigitte Bigi (2015). A SIP of CoFee: A Sample of Interesting Productions of Conversational Feedback, 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Prague (Czech Republic), pp. 149–153.

Brigitte Bigi, Katarzyna Klessa, Laurianne Georgetown, Christine Meunier (2015). A syllable-based analysis of speech temporal organization: a comparison between speaking styles in dysarthric and healthy populations. Interspeech2015, Dresden (Germany), pages 2977-2981.

Marion Tellier, Gale Stam, Brigitte Bigi (2013). Gesturing While Pausing In Conversation: Self-oriented Or Partner-oriented?, TIGER, Tillburg.

Laurent Prévot, Brigitte Bigi, Roxane Bertrand (2013). A quantitative view of feedback lexical markers in conversational French, 14th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Metz, France.

Marion Tellier, Gale Stam, Brigitte Bigi (2012). Same speech, different gestures?, 5th International Society for Gesture Studies (ISGS), Lund, Sweden.

## Some corpora

Birgit Rauchbauer, Youssef Hmamouche, Brigitte Bigi, Laurent Prévot, Magalie Ochs, Thierry Chaminade (2020). Multimodal Corpus of Bidirectional Conversation of Human-human and Human-robot Interaction during fMRI Scanning. In Proceedings of The 12th Language Resources and Evaluation Conference, pp. 668–675, Marseille, France.

Béatrice Priego-Valverde, Brigitte Bigi, Mary Amoyal (2020). “Cheese!”: a Corpus of Face-to-face French Interactions. A Case Study for Analyzing Smiling and Conversational Humor. In Proceedings of The 12th Language Resources and Evaluation Conference, pp. 467–475, Marseille, France.

Christine Meunier, Cécile Fougeron, Corinne Fredouille, Brigitte Bigi, Lise Crevier-Buchman, Elisabeth Delais-Roussarie, Laurianne Georgetown, Alain Ghio, Imed Laaridh, Thierry Legou, Claire Pillot-Loiseau, Gilles Pouchoulin (2016). The TYPALOC Corpus: A Collection of Various Dysarthric Speech Recordings in Read and Spontaneous Styles, Tenth International Conference on Language Resources and Evaluation (LREC 2016), pp. 4658-4665, Portoroz, Slovenia.

Jan Gorish, Corine Astésano, Ellen Gurman Bard, Brigitte Bigi, Laurent Prévot (2014). Aix Map Task corpus: The French multimodal corpus of task-oriented dialogue, 9th International conference on Language Resources and Evaluation (LREC), Reykjavik (Iceland), pages 2648-2652. ISBN: 978-2-9517408-8-4.

Daniel Hirst, Brigitte Bigi, Hyongsil Cho, Hongwei Ding, Sophie Herment, Ting Wang (2013). Building OMProDat: an open multilingual prosodic database, Proceedins of Tools ans Resources for the Analysis of Speech Prosody, Aix-en-Provence, France, Eds B. Bigi and D. Hirst, ISBN: 978-2-7466-6443-2, pp. 11-14.
Ellen Gurman Bard, Corine Astésano, Alice Turk, Mariapaola D'imperio, Noel Nguyen, Laurent Prévot, Brigitte Bigi (2013). **Aix MapTask: A (rather) new French resource for prosodic and discourse studies**, Proceedings of Tools ans Resources for the Analysis of Speech Prosody, Aix-en-Provence, France, Eds B. Bigi and D. Hirst, ISBN: 978-2-7466-6443-2, pp. 15-19.

Sophie Herment, Anastasia Loukina, Anne Tortel, Daniel Hirst, Brigitte Bigi (2012). **AixOx, a multi-layered learners corpus: automatic annotation**, Proceedings of international conference on corpus linguistics, Jaèn (Spain).

**Other tools re-implemented into SPPAS**

Maciej Karpinski, Katarzyna Klessa (2018). \*\* Methods, Tools and Techniques for Multimodal Analysis of Accommodation in Intercultural Communication\*\*, CMST 24(1) 29–41 (2018), DOI:10.12921/cmst.2018.0000006

Dafydd Gibbon (2013). **TGA: a web tool for Time Group Analysis**, Tools ans Resources for the Analysis of Speech Prosody, Aix-en-Provence, France, pp. 66-69.

D.-J. Hirst (2011). **The analysis by synthesis of speech melody: from data to models**, Journal of Speech Sciences, vol. 1(1), pages 55-83.

Daniel Hirst and Robert Espesser (1993). **Automatic modelling of fundamental frequency using a quadratic spline function.** Travaux de l'Institut de Phonétique d'Aix. vol. 15, pages 71-85.
# SPPAS Release notes

## 8.1 The early versions

Versions 1.0 to 1.3 was made only of `tcsh` and `gawk` scripts. It was developed under Linux system and was efficiently tested under Windows with Cygwin.

### 8.1.1 Version 1.0

(2011, 9th March)

The only feature was that it was able to perform speech segmentation of English read speech.

### 8.1.2 Version 1.1

(2011, 7th June)

This was mainly the debug of the previous version and some code re-organization and cleaning.

### 8.1.3 Version 1.2

(2011, 23rd July)

The support of English, French and Italian was added: a lexicon, a pronunciation dictionary and an acoustic model of each language was created. Three annotations were implemented: Tokenization, Phonetization, Alignment.

The basis for a *multi-lingual methodology* was already there.

### 8.1.4 Version 1.3

(2011, 12th December)

This is a transitional version, from the scripts language to Python programming language. The main fixes and improvements were:
**Development:**

* MacOS support

* bugs corrected in many scripts

* check.csh -> check.py

* sppas.csh -> sppas.py

* GUI: zenity -> pyPytk

* sox limited to the use of re-sampling audio signal

* a python library to manage wav files is added.

* IPUs segmentation automatic annotation

**Resources:**

* Italian dictionary improved

* Italian acoustic model changed (triphones trained from map-task dialogues)

* French acoustic model changed

But the program still haven't a name and isn't distributed.

## 8.2 The birth of SPPAS

### 8.2.1 SPPAS 1.4.0

(2012, 14th June)

It's the official birth of SPPAS: the software has a name, a license, a GUI, a web-page and is freely distributed to the community.

The source code is only based on Python 2.7 language and the GUI is based on WxPython 2.8.x or 2.9.x (on MacOS, wxpython must be 32 bits). SPPAS requires also sox and julius software to be installed.

**Automatic annotations:**

* add Momel: Momel (MOdelling MELody) is an algorithm developed by Daniel Hirst and Robert Espesser for the analysis and synthesis of intonation patterns. Momel needs a .hz files with pitch values: ASCII file with one value each 10ms. This Momel implementation can be used directly, with a large set of options: > python $SPPAS/scripts/lib/momel.py -h

or in SPPAS, using the GUI, or in text-mode (only with default options): > $SPPAS/sppas.py -i INPUT -momel

* add the first version of INTSINT, proposed by Daniel Hirst. INTSINT is an acronym for INternational Transcription System for INTonation. INTSINT codes the intonation of an utterance by means of an alphabet of 8 discrete symbols constituting a surface phonological representation of the intonation.

**Packaging:**

* merge AUTHORS, VERSION and README files in the README file
* create packages in the lib directory (2012-03-20)

* ipu-segmentation in python (2012-02-16)

* improve ipu segmentation algorithm (2012-05-24)

* phonetization in python (2012-02-27), with an unknown word phonetization.

* alignment entirely in python (2012-03-29)

* syllabification implemented with python, and tool/syllabify.py

* add a simple terminal controller (not needed for Unix, just for "DOS")

* Momel can read PitchTier (from Praat) files

* Momel can deal with long sounds (not only one IPU)

* manage source files, comments, exceptions...

* SPPAS works as a single instance by mean of a lock file.

* SPPAS can be installed in a directory containing spaces and can deal with file names with spaces (bug fixed: 2012-06-11)

## GUI:

* GUI with the wxpython library (2012-03-27)

* Manage a list of selected files

* Add a "File information"

* Add a "Wav player"

* Fix options to each annotation step with the menu

* Open the log file after each process

## Resources:

* French dictionary is using UTF-8 (instead of iso8859-1 in previous versions)

* French dictionary is based on X-SAMPA phone set

* Chinese: work with chinese characters instead of pinyin.

## Known Bugs:

* The wav player can not play wav files if the filename contains '#'.

* The first line of the Italian dictionary must be changed to "# [#] #".

## 8.2.2 SPPAS 1.4.1

(2012, 13th July)

### Resources:

* Updated English acoustic model (from voxforge, 2012-06-25)

* English acoustic models converted to X-SAMPA

### Automatic annotations:

* IPUs Segmentation annotation performs a simple silence detection if no transcription is available (the volume is automatically adjusted)
* A specific language can be selected for each annotation depending on available resources

* Updated transcription conventions:

- truncated words: a '-' at the end of the token string (an ex- example)

- liaisons: the 'letter' between '=' (an =n= example)

- noises: * (only for FR and IT)

- short pauses: + (a + example)

- silences: # (a # example)

**GUI:**

* Create (systematically) a merged annotations TextGrid file.

**Development:**

* Package's management

### 8.2.3 SPPAS 1.4.2

(2012, 2nd August)

**GUI:**

* add a panel with a Transcription editor

* IPU segmentation can split a wav into tracks

* IPU segmentation can fix a shift value to boundaries

* IPU segmentation: min-volume option is removed (because the min-volume value is automatically adjusted)

* "File Information" button adds some tier manipulation tools: cut/copy/paste/rename/duplicate/move/preview/filter

**Known bug:**

* The filter frame is not working under Windows XP.

### 8.2.4 SPPAS 1.4.3

(2012, 10th October)

This is primarily a bug-fix release. The author is addressing many thanks to all users who send their comments!

**GUI:**

* Frames and Dialog design is more uniform.

* Users preferences changed. Themes and colors introduced.

* Help is available.
**Automatic annotations:**

* Bug fixed for phonetization/alignement if the input transcription contains series of silence intervals or series of speech intervals. Previous versions was supposing a **strict IPUs** input transcription.

* Tokenization is done.

**Development:**

* Code cleaning in the package wxGUI.

* Debug...

## 8.2.5 SPPAS 1.4.4

(2012, 6th December)

**GUI:**

* add information/options when "Request" a file

* debug Request/Filter (for Windows systems)

**Automatic annotations:**

* New Italian acoustic model

* New Chinese acoustic model, and some minor changes in the dictionary

**Development:**

* ".trs" files support (transcriptions from Transcriber)

* debug (alignment, tokenization)

* add ".lab" files export (HTK format)

**Known bugs:**

* Alignment: it fails under Windows, if *julius* is not installed properly.

* Syllabification: the last syllable of each file is "broken".

* Alignment error if unknown words during phonetization.

## 8.2.6 SPPAS 1.4.5

(2013, 15th January)

**Development:**

* Correct a few bugs of the previous version (phonetization, alignment, syllabification)

* ".eaf" files support (transcriptions from Elan software)

* add script tierfilter.py

* add script tiercombine.py
**Automatic annotations:**

* Experimental version of Vietnamese

* add Tokenization as a specific annotation level

* add Phonetization of PinYin

**GUI:**

* Request/Filter a tier: multiple patterns filtering

* Request: add a "New File" button

## 8.2.7 SPPAS 1.4.6

(2013, 12th February)

**GUI:**

* Improved Request/Filter a tier:

- add new modes: not contains, not starts with, not ends with

- add time constraints: minimum duration, maximum duration, meets

- multiple modes selection (replace radio buttons by check buttons)

* Add Requests/Stats to obtain basic statistics

* Requests Copy/Cut/Paste/Duplicate/Save debugged

**automatic annotations:**

* IPU segmentation: can take a "Name" tier to fix names of tracks (if the "Split into tracks" option is checked)

## 8.2.8 SPPAS 1.4.7

(2013, 25th March)

**Development:**

* re-organization of lib, except for the wxGUI library.

* Cancel the sppas.lock file creation when SPPAS is running.

* Requests/Filter: "Starts search at..." and "Ends search at..."

* Requests/Filter: remove negative search (because of a bug...). Replaced by a "reverse" option.

**Resources:**

* add experimental version of Taiwanese automatic segmentation (from romanized transcriptions)
**Annotations:**

* New version of INTSINT, based on the algorithm proposed in (Hirst 2011) and implemented in the last version of Momel/INTSINT Praat plugin!

**8.2.9 SPPAS 1.4.8**

(2013, 30th May)

**Development:**

* reorganization of the GUI code. SPPAS is made of:

- automatic annotations;

- 3 components (wavplayer, transcriber, requests).

* sppas.py removed. sppas.bat created (for Windows).

* Input/Output library entirely changed

**GUI:**

* components (wavplayer, transcriber, requests) in separate frames

* new design

* Help and documentation changed, expanded and improved

**8.2.10 SPPAS 1.4.9**

(2013, 3rd July)

SPPAS has a new and more colored logo!

**Development:**

* bug correction:

- Bug fixed in IPU segmentation with option "Split into tracks"

- Bug fixed in Momel with some rare files

- Bug fixed to create a merged file

- Bug fixed in align and IPU seg.

- Bug fixed in Transcriber

* library organization revised

**GUI:**

* Add an export button to the File List Panel

* Migrate wav infos from the Requests component to the Wav player component

* Volume control removed in the Wav Player component

* Save As improved in the Requests component
## 8.2.11 SPPAS 1.5.0

(2013, 2nd August)

**Development:**

* bug correction

**Annotation:**

* Tokenization: TOE support finalized

**GUI:**

* Transcribe: debug of IPU player, for MacOS

**Resources:**

* New French acoustic model. Attention: phoneset changed.

* New French dictionary: use the new phoneset!

* New Taiwanese acoustic model. Attention: phoneset changed: accept some chinese...

* New Taiwanese dictionary: use the new phoneset + some chinese syllables added.

* Vietnamese removed: due to the lack of data, the model can't be improved.

## 8.2.12 SPPAS 1.5.1

(2013, 29th August)

**Development:**

* bug correction in annotations

* help system debugged

**Annotation:**

* Phonetization of unknown words improved

**Resources:**

* French dict modified

## 8.2.13 SPPAS 1.5.2

(2013, 27th September)

All resources are moved into the "resources" directory.
**Development:**

* bug correction in annotations and GUI

**Resources:**

* French dict modified

* New resources for the tokenization

**Components:**

* "Statistics": an advanced component to estimate (and save!) descriptive statistics on multiple annotated files

**8.2.14 SPPAS 1.5.3**

(2013, 25th October)

**Resources:**

* Add Spanish support

**Components:**

* improved Statistics component

* Add a "Filter" component, first version with a basic GUI

**GUI:**

* Help updated.

**8.2.15 SPPAS 1.5.4**

(2013, 3rd December)

**Components:**

* Wav Player changed.

* Information and Requests removed. Replaced by Data Roamer.

* Transcriber removed. Replaced by IPU Transcriber.

* Statistics updated.

* Filter updated.
**Annotations:**

* Add Repetitions (detection of sources only)

Notice that this is the first stable release.

**8.2.16 SPPAS 1.5.5**

(2013, 23rd December)

**Development:**

* improved annotationdata (add methods: Find, Index; add uncertain label; debug radius).

**Components:**

* debug

**GUI:**

* Tips at start-up

* New theme: Christmast

**8.2.17 SPPAS 1.5.6**

(2014, 28th January)

**Development:**

* improved annotationdata (add methods: Near, Search).

* Package cleaning!

**Components:**

* debug

**Resources:**

* New Italian dictionary (including gemination)

**8.2.18 SPPAS 1.5.7**

(2014, 18th February)
**Development:**

* Plugins manager added.

**Resources:**

* Japanese support, thanks to the resources available on the Julius website.

**8.2.19 SPPAS 1.5.8**

(2014, 18th March)

**Development:**

* annotationdata: more flexibility while adding annotations, add sub-divisions, export csv modified, doc-strings, import/export in a native format (xra, version 1.0).

**Documentation:**

* add a PDF file of slides: "SPPAS for dummies"

**8.2.20 SPPAS 1.5.9**

(2014, 15th April)

**Components:**

* new: DataViewer, experimental version

**Annotations:**

* syllabification rules: accept 2 types of vowels (V and W)

* syllabification: faster!

**Development:**

* Export Elan

* Import/Export xra. XRA is the native format of SPPAS annotation files. See etc/xra for details.

**Resources:**

* New French acoustic model
## 8.2.21 SPPAS 1.6.0

(2014, 22th May)

**Package:**

* Rename folder "Doc" to "documentation"

* A documentation is created

* SPPAS-for-dummies updated

**Development:**

* helpsystem removed

* GUI: package re-organization, re-implementation.

* Alignment: choice of the aligner (julius, hvite or basic)

**Resources:**

* new acoustic model: FR-Read and FR

* new acoustic model: ZH

* new acoustic model: TW

* new acoustic model: SP

## 8.2.22 SPPAS 1.6.1

(2014, 26th September)

**Development:**

* bug correction (export, syllabification, alignment)

* DataViewer, major bugs corrected.

**Resources:**

* Italian: new pronunciation dictionary and new acoustic model

* add resources for Catalan

**GUI:**

* add a new Export button

## 8.2.23 SPPAS 1.6.2

(2014, 21th October)
**Resources:**

* language names changed. They are now corresponding to the international standard ISO639-3. See http://www-01.sil.org/iso639-3/

* Mandarin Chinese dictionary changed.

* Catalan dictionary changed.

## 8.2.24 SPPAS 1.6.3

(2014, 2nd November)

**Resources:**

* Add resources for Cantonese

**Documentation:**

* It's now (more or less) finished!

**Development:**

* Support of Elan improved (add Controlled vocabulary)

**GUI:**

* Change the organization of the main frame: annotations above components

This version is known to be a stable release.

# 8.3 The beginnings of SPPAS

## 8.3.1 SPPAS 1.6.4

(2014, 5th December)

From this version, SPPAS requires wxpython to be updated to version 3.0, particularly for MacOS users, and they need to install the 64 bits version. It is recommended to Windows users to install Python 2.7 and wxpython in 32bits.

**Development:**

* Package re-organized!

* Phonetization of unknown words improved

* Support of upper/lower of the extension of speech files (wav, WAV)

* Tokenization of languages with dictionaries in upper case (eng, ita): bug fixed.
* Creates systematically a dump file of resources for a faster load at the next use

* Read TextGrid files exported by Elan: bug fixed.

* sppas.command checks the system and run either in 32 or 64 bits (MacOS)

**Components:**

* IPUscribe replaces IPUTranscriber mainly for the support of large files: tested with a file of 1 hour speech (143 Go) and 800 IPUs.

* SndRoamer replaces WavPlayer

* Dataroamer has also a new version

## 8.3.2 SPPAS 1.6.5

(2014, 17th December)

This is primarily a bug-fix release.

**Development:**

* all programs in "bin" and "scripts" sub-directories were revised and tested, or removed.

**Annotation:**

* Tokenization: code cleaning and re-organisation.

**GUI:**

* Procedure outcome report: print a warning message in the log file if no file is matching the expected extension

## 8.3.3 SPPAS 1.6.6

(2015, 19th January)

Web site host has changed: http://sldr.org/sldr00800/preview/

Documentation completed and updated. Now, only the documentation of all the components is missing.

**Annotations:**

* log messages more explicit and status printed with intuitive colors.

* management of input/output file format re-done: now easier for the user.
**Development:**

* package architecture revised: mainly “sppasgui” and “components” merged in “wxgui”, and many other changes.

* thread removed in automatic annotation process.

* debug of alignment: if too short units.

* radius value properly fixed in most of the automatic annotations.

**GUI:**

* GUI is more homogeneous and pretty (hope!)

* Show the date in the status bar

* New Settings frame:

- 4 icon themes available

- Choice of foreground and background colours

- Choice of the font

- Choice of the input/output file format of annotations

* New in Help menu:

- access to the project homepage

- access to the online documentation

* New Feedback window

* New Help browser

* Add Keyboard shortcuts:

- ALT+F4 to exit,

- CTRL+A to add files in FLP

- SHIFT+CTRL+A to add a directory in FLP

- Del to remove selected files of the FLP

- SHIFT+Del to erase selected files of the FLP

- CTRL+C to copy files

- CTRL+E to export files

- F5 to refresh the FLP

- F1 to open the help browser

- F2 to open the “About” frame

**Components:**

* GUI design unified for DataRoamer, SndPlayer, IPUscribe and SppasEdit

* New Tier Preview frame (still under development)

* SndPlayer print information about a sound file with colors:

- Green: the information corresponds to the SPPAS requirements
- Orange: the information does not exactly corresponds to the requirements however SPPAS is able to deal with (after conversion)

- Red: SPPAS does not support. It must be converted before using it!

### 8.3.4 SPPAS-1.6.7

(2015, 16th February)

**Automatic Annotations:**

* By default, tokenization produces only one tier. Check an option to get TokensStd and TokensFaked, in case of EOT.

* radius value properly fixed in most of the automatic annotations.

**GUI:**

* Tested and debugged on MacOS (version 10.9.5, with wxpython 3.0.2)

**Development:**

* Tier hierarchy partly implemented: TimeAlignement and TimeAssociation are two "links" that can be fixed between tiers.

**Annotations:**

* Add Polish support

### 8.3.5 SPPAS-1.6.8

(2015, 9th April)

**Resources:**

* new French acoustic model

* new English acoustic model (VoxForge nightly build of March, 15th, 2015)

* add phoneset mapping tables

**Development:**

* Add a phoneme mapping in models, to allow both the dictionary to include real X-SAMPA symbols and the acoustic model to be compatible with Hvite requirements (only ASCII).

* annotationdata bug correction with min and max values

* IPUs Segmentation:
- bug correction when split into tracks with a tier "Name"

- add the ipu number in speech segments if silence/speech segmentation

* Self-repetitions debug in finding the repetition interval

**GUI:**

* DataRoamer: "New" button debugged

* DataRoamer: Add a button "Radius" to adjust manually the vagueness of each boundary of a tier

**8.4 The development phase**

8.4.1 SPPAS-1.6.9

(2015, 14th May)

The installation of dependencies is simplified: `sox` is unnecessary. Python 2.7.x, WxPython and Julius are
the only remaining dependencies.

**Development:**

* package annotationdata.filter updated to support last changes in annotationdata: multiple labels and numerical labels.

• package annotationdata.io:

- praat.py newly created. Support of TextGrid, PitchTier and IntensityTier files completely re-written

- htk.py newly created to support .lab and .mlf files

- sclite.py newly created to support .stm and .ctm files

- signaix.py newly created to support .hz files

- SPPAS native format XRA upgraded to version 1.1 to support improvements of the library.

• package annotationdata:

- updated object Transcription to support more/different input data

- updated hierarchy

- updated meta-data

* package signal: partially re-written. As a consequence, sox won't be neither used, but the file conversion (if required) is slower.

**GUI:**

* DataFilter component partially re-written to facilitate its use

- Preview frame modified
## 8.4.2 SPPAS-1.7.0

(2015, 3th July)

**Development:**

* package annotationdata.io:

- add support of subtitles: sub, srt

- elan.py created to replace eaf.py: allows a full support of Elan annotations

- transcriber.py created to replace trs.py for a better support of Transcriber files

- anvil.py allows to import anvil files

* package annotationdata:

- updated hierarchy (simplified)

- updated meta-data

* package signal:

- support of audio files re-written: can open/save wav, aiff and au files

- add a lot of possibilities to manage one channel of an audio file

**GUI:**

* DataFilter component finalized: can deal with alternative labels, and typed labels (string, number, boolean)

* Statistics component fully re-written to facilitate its use

* SndRoamer displays more properties of audio files (min, mean and max added)

**Annotations:**

* IPUs Segmentation produces 2 tiers if a transcription is given: the IPUs segmentation itself, and the Transcription time-aligned at the IPUs level.

## 8.4.3 SPPAS-1.7.1

(2015, 5th August)

**Development:**

* package re-organization:

- package signal is now standalone

- package resources is now standalone

- package presenters created

* updated statistics estimations

* package annotationdata:
- add Media object

- add CtrlVocab object

- add inheritance of MetaObject for Annotation

• package annotationdata.io:

- full debug of all file formats

- add comments and documentation

- add also some tests

- add Media/CtrlVocab in some file formats

- add Annotation Pro support of antx files (in API)

**Components:**

• add Time Group Analyser (TGA) in Statistics

• add Kappa estimation on labels of 2 tiers with the same number of intervals

**Annotations:**

• New version of XRA: 1.2

• Make XRA the default input/output file format

## 8.4.4 SPPAS-1.7.2

(2015, 3th September)

**Development:**

• updated XRA reader/writer to solve problems with upper/lower cases of elements

• updated Elan reader to be compatible with format 2.8

• updated Praat reader/writer for single/double quotes

• support of AnnotationPro antx files in the GUI

• add the julius score in PhonAlign annotation (can be seen only if XRA)

• remove tk dependency

## 8.4.5 SPPAS-1.7.3

(2015, 9th October)

**Resources:**

• New word-based vocab for Cantonese

• New word-based dict for Cantonese

• New phoneme-based acoustic model for Cantonese
**Development:**

* Descriptive statistics debugged for detailed panels.

**8.4.6 SPPAS-1.7.4**

(2015, 6th November)

**Resources:**

* Add a vocab for Portuguese

* Add a dict for Portuguese

* Add an acoustic model for Portuguese. It has to be noticed that it was constructed from French/Spanish/Italian models, not trained from data.

**Samples:**

* Add Portuguese

* Change some samples in various languages

**Development:**

* Debug of the Tokenizer.

**8.4.7 SPPAS-1.7.5**

(2015, 11th December)

**Development:**

* Add vagueness to the annotation duration estimation

* Bug correction while saving the procedure outcome report in the GUI

**GUI:**

* Changed all pictures to remove the problem with colorset of some of them

* Add a christmastheme for the pictures of tips

* IPUscribe improvements:

- add keyboard shortcuts to play/pause/stop the sound

- add a new button to auto-play the sound
### 8.4.8 SPPAS-1.7.6

(2016, 28th January)

Web site host has changed: http://www.sppas.org/

**Development:**

* IPU segmentation:

- bug correction with option “split into tracks”

- new option to add or not add the IPU index of each IPU into the transcription tier

* Tokenization:

- bug correction in input tier selection

- bug correction for replacements

**Resources:**

* add a vocabulary of Korean

* add an acoustic model for Korean (made from the English and the Taiwanese ones).

**GUI:**

* DataRoamer: bug correction of “Duplicate”

**Others:**

* Add a sample in Korean

* Updated references to cite SPPAS in publications

### 8.4.9 SPPAS-1.7.7

(2016, 30th March)

**Resources:**

* Correction of errors in most of the acoustics models, for /@@/ and /dummy/ models

* New vocabulary and pronunciation dictionary for Mandarin Chinese.

**GUI:**

* Dialogs are customized

* DataRoamer debug and changes: save buttons moved in the main toolbar.

* HelpSystem is (welcome) back!
**Development:**

* Add the API and a script to train acoustic models with HTK.

* Add Kullback-Leibler distance estimator.

* Add a script to compare segmentation of 2 tiers.

* Classes of the package resources are debugged, improved and extended.

* Alignment: bug correction if input starts by an empty interval.

* sppas.command modified for MacOS-X (use python if python2 not available)

* GUIs in bin directory are updated (test of python, wxpython, and so on).

## 8.4.10 SPPAS-1.7.8

(2016, 5th May)

**Resources:**

* Add a pronunciation mapping table eng-fra.map: to be used for the speech segmentation for French speakers reading an English text.

**Development:**

* Phonetization is extended: it can take into account a mapping table of phones to generate new pronunciation variants.

* Phonetization: use of minus instead of dots to separate phones as recommended in X-SAMPA standard.

* Alignment is restructured and extended: it can take into account two acoustics models and mix them.

* Filter is extended: Relations can be based on a delay between the intervals

* annotationdata: add Xtrans reader (extension: .tdf)

* Code cleaning in several packages.

**GUI:**

* Automatic annotations: more explicit log messages.

## 8.4.11 SPPAS-1.7.9

(2016, 3rd June)

**GUI:**

* Some debug in SppasEdit

* SndPlayer renamed AudioRoamer because new functionalities were added:

- See detailed information about each channel of an audio file

- Can extract/save a channel or a fragment of channel

- Can modify the frame rate and the sample width of a channel
- Can add a bias on amplitude values of a channel

- Can multiply amplitude values of a channel

- Can remove the offset of amplitude values of a channel

**Automatic annotations:**

* IPUs Segmentation fully re-implemented.

- Silence/speech segmentation improved for both quality and fastness

- Package code cleaning and re-organization.

**8.5 The stabilization phase**

**8.5.1 SPPAS-1.8.0**

(2016, 30th August)

**GUI:**

* Design fully revisited and tested under Linux Mint, Windows 10 and MacOS 10.9

**Development:**

* SLM package created: can estimate a statistical language model (without smooth method) on a small corpus

**Automatic annotations:**

* Add a diagnosis of files

* Tokenize extended: applied also on alternative labels

* Phonetize extended: applied also on alternative labels

* Alignment code cleaning and partly re-implemented

* Add “Chunk alignment”

* Use of a .ini file to configure each annotation instead of a sppas.conf file

**8.5.2 SPPAS-1.8.1**

(2016, 28th November)

A few tutorials are available on the web site.

**Automatic annotations:**

* Align: an ActivityDuration tier can be optionally added.

* Support of 3-columns tab-delimited files with .txt extension. It allows the compatibility with Audacity Label track files.

* Acoustic models training validated.
**Resources:**

* Catalan: new pronunciation dictionary and new acoustic model.

**8.5.3 SPPAS-1.8.2**

(2017, 18th January)

**Analysis:**

* debug of DataFilter

**Resources:**

* French vocabulary and dictionary updated

**Development:**

* new plugins package with a new plugin manager

* GUI integration of this new plugins system

* some unittest appended and all existing ones updated

* annotationdata.io renamed annotationdata.aio

* docstrings of some packages converted from epytext to reST syntax

**GUI:**

* DataStats, DataFilter and DataRoamer toolbars don't scroll anymore

* Themes management changed.

* Main font is managed by the Themes.

**8.5.4 SPPAS-1.8.3**

(2017, 10th March)

**Development:**

* Elan reader highly improved (faster reader).

* updated plugins

**8.5.5 SPPAS 1.8.4**

(2017, 10th April)

**Development:**

* Elan writer modified: create a time slot for each localization.
### 8.5.6 SPPAS 1.8.5

(2017, 20th April)

**Development:**

* Vizualizer renamed Visualizer

* AudioRoamer: bug with an icon corrected

* New Phonedit mrk format support.

* Updated AnnotationPro Antx reader/writer

### 8.5.7 SPPAS 1.8.6

(2017, 19th June)

**Resources:**

* Polish dictionary and acoustic model updated.

### 8.5.8 SPPAS 1.9.0

(2017, 28th July)

**Programming:**

* Relative imports used in the standard way for Python

* PEP 8 code style (except for wxgui/annotationdata)

* PEP 257 reST code documentation style (except for wxqui/annotationdata)

* Unittests:

- existing tests verified, improved, extended

- new tests added

- tests migrated into the packages

* Compatibility for both Python 2.7 and Python > 3.2:

- `makeunicode.py` contains functions and classes to deal with strings

- `utils`, `term`, `structs`, `resources`, `presenters`, `plugins`, `calculus` packages: migration is done

* Exceptions are separately managed

* Introduction of a system for the internationalization of the messages. Done in English and French for the packages: `audiodata`, `calculus`, `plugins`, `resources`, `structs`, `term`, `utils`

* Package re-organization:
- new package "models", with acm and slm

- utils

- resources

- ...

* meta.py is replacing sp_glob.py

* new scripts: tieraligntophon.py, dictmerge.py

* new bin: pluginbuild.py

* Robustness to read malformed HTK-ASCII pronunciation dictionaries.

* Bug corrected in the management of pronunciation variants

* re-structured package TextNormalization

* re-structured package Repetitions

* new version of the plugins: updated and debugged.

**Resources:**

* Add support of Naija language (pcm)

* English-French mapping table updated

**Annotations:**

* Tokenizer renamed into Text Normalization:

- a lot of debug mainly for English language, and punctuation managements

- new option: can output a customized tier.

* Repetitions:

- some debug

**Communication:**

* Add a description document of the orthographic transcription convention in the package.

* Web page updated, new tutorials available

* Documentation updated

* Better information about the licenses

### 8.5.9 SPPAS 1.9.1

(2017, 1st September)
**Programming:**

* Bug correction in the ELAN reader.

* Bug correction with quotation marks of the Praat writer.

**Resources:**

* Add an acoustic model for English, including laughter and noises.

8.5.10 SPPAS 1.9.2

(2017, 6th October)

**Programming:**

* Bug correction in the diagnosis of audio files.

* Package "anndata" continued, in the scope of replacing "annotationdata".

* Acoustic model training procedure debugged and improved, code cleaned, scripts updated, etc.

8.5.11 SPPAS 1.9.3

(2017, 18th October)

**Programming:**

* Critical bug correction in Alignment: error when loading the tiedlist.

* Correction of TextNormalization of broken words: "-" is now not removed.

**Resources:**

* Italian pronunciation dictionary updated.

* French vocabulary updated: list of compound words revised.

**Known bugs:**

* Spanish alignment does not work: corrupted acoustic model

8.5.12 SPPAS 1.9.4

(2018, 15th January)

**Programming:**

* Script to train acoustic models updated

* Script to evaluate alignments updated

* Bug corrections of the search tier system
**Resources:**

* Spanish:

- new pronunciation dictionary

- new acoustic model

**8.5.13 SPPAS 1.9.5**

(2018, 26th April)

**Programming:**

* I/O development of the package "anndata".

* Reading and writing of annotated files is now based on the new 'anndata' package instead of the old 'annotationdata' package. This was a big issue and it allows a better support of annotated files (less bugs and the amount of lost information is drastically reduced).

**8.5.14 SPPAS 1.9.6**

(2018, 25th May)

**Resources:**

* New: support of German language for Text Normalization (except num2letter), Phonetization and Alignment.

* French: new acoustic model with an updated phoneset and a better accuracy.

* Laugh and noise are uniformly represented in all languages.

* English: context-independent model is the default, instead of context-dependent model, for Alignment.

**Development**

* debug of the Transcriber file reader.

* debug in the management of the hierarchy in file readers/writers.

* Resources package: add compatibility with PLS (W3C xml) pronunciation dictionaries

**Known bugs:**

* Syllabification of French: the configuration file is not correct.

**8.5.15 SPPAS 1.9.7**

(2018, 23rd July)
## Development

* new filter system in anndata

* IPUs segmentation: debug in fixing automatically the threshold for the search of silences (can't be negative!)

* Annotations: improved messages for the procedure outcome report

* Annotations: Text normalization is based on anndata API instead of annotationdata

- the result is now a sequence of individual tokens (each token is a sppsLabel()), instead of a single string separating tokens with space.

* Annotations: Phonetization is based on anndata API instead of annotationdata

- the result is now a sequence of sppsLabel() with alternative tags: each tag corresponds to a pronunciation variant

* Annotations: Syllabification re-programmed and now based on anndata API

- only sequences of phonemes are syllabified (i.e. no silence, no laugh nor noise in the result)

- O class name changed into P class name

- do not generate the tier “structures” tier anymore

- generating the “classes” tier is optional

* Annotations: INTSINT is based on anndata API instead of annotationdata

* IPUScriber is based on anndata API instead of annotationdata

## Resources:

* French syllabification rules updated to be compliant with the new phoneset (used since version 1.9.6)

## Documentation

* updated chapter 6: scripting with Python and SPPAS. It is now based on anndata API instead of annotationdata.

## 8.5.16 SPPAS 1.9.8

(2018, 06th September)

e-mail contact is changed to:

* contact@sppas.org for general questions of users

* develop@sppas.org for developer questions or for a bug alert

## Development

* bug correction for GMT with a negative value, like GMT-4.

* most of the scripts are based on anndata API instead of annotationdata

* dependency to markdown removed

* new solution to work with global variables with the new package config and the new classes like sppasGlobalSettings, sppasPathSettings, ...
* translation management moved in package config

* better way to work with imports

* increased pep8 and pep257 compatibility (should be continued)

* `anndata` is used in `DataRoamer`, `Visualizer`, `DataFilter`

* new script `trsshift.py` to shift the transcription of a delay in time

## Documentation

The API documentation is based on Sphinx http://www.sphinx-doc.org.

## 8.5.17 SPPAS 1.9.9

(2018, 23th October)

## Development

* package for the 'alignment' is fully re-structured (re-implemented partially), new unithreads are added, and compatibility with python 3, pep8 and pep257.

* new GUI based on `python3 + phoenix`: it prints an information message in case SPPAS is launched with these versions.

* package 'prensenters' updated: `unithreads`, `python 3`, `pep8` and `pep257`.

## Plugins

* `sampa2ipa` included to the SPPAS package

* `audiosegmenter` included to the SPPAS package

## Annotations

* IPUs Segmentation removed. It is replaced by:

1. Search for IPUs: to find silences/tracks from an audio file

2. Fill in IPUs: to find silences/tracks from an audio file and its transcription

3. `AudioSegmenter` plugin: to segment audio files into several tracks

* `annotation.py`: bug correction with the default output extension

* bug correction when the phonetization was unknown

## Resources

* add a file 'monophones' into each model, for the compatibility with HVite.

* errors of the acoustic models of "spa" and "nan" corrected.
## 8.5.18 SPPAS 2.0

(2019, 4th January)

The main change of this release is that the package annotationdata has been removed, so that all packages are using anndata instead. One of the most important consequence of this change is that all packages, except the wxGUI, are compatible with both Python 2.7 and Python 3.4+.

### Annotations

* Alignment: bug correction if the last unit is empty.

* Self-Repetitions and Other-Repetitions split into two different packages

* Un-used package 'Chunks' removed

* Better management of annotations: sppasBaseAnnot, sppasParam and sppasManager revised.

### UI

* All programs in bin folder updated. A 'manual mode' and an 'auto mode' are available in the annotations to fix input/output, and a -log option is proposed to save a report.

* Plugins: marsatagplugin added and debugged.

## 8.5.19 SPPAS 2.1

(2019, 28th February)

### Development

* Praat TextGrid: it's now allowed to use the symbol '=' in annotation labels.

* Use of 'json' files instead of 'ini' files to configure SPPAS, the annotations and the plugins.

* New package 'analysis' with the filter and statistic systems.

### Annotations

* Search for IPUs: the program has been improved and evaluated.

* Alignment: priority is given to standard tokens (if EOT).

* Alignment: problem of whitespace in filenames solved.

### UI

* Plugins: new plugin to remove un-translated IPUs and to re-index the IPUs

* extended internationalization of the messages (for English and French)
## 8.5.20 SPPAS 2.2

(2019, 09th May)

sppas.bat and sppas.command, the two main ways to launch the GUI, were fully re-written in order to search for 'pythonw' command first. It results in the following advantages: - it increases the compatibility with MacOS systems; - it allows to not display the dark frame of 'python' under Windows.

### Development

* New package 'files' for a further use.

* Increased compatibility of file formats: can read alternative tags with `{!}` system, whitespace vs CR, etc.

### Plugins

* Debug of the plugin to clean IPUs.

### Annotations

* Search for IPUs: a special threshold value is set when the audio recording has a very low median value (ie very low volume values)

* Normalization/Phonetization/Alignment/Syllabification can work without audio

* Search for IPUs and Fill in IPUs: problem with upper/lower extension solved

### GUI

* "Delete" button of the "File Explorer": files are no longer definitively deleted. Instead, they are moved into a hidden folder with name '.trash' in the package of SPPAS.

* wxFrame to display log messages.

* Annotations in GUI: better compatibility to annotate written texts

### Resources

* `fra.dict`: corrected 6 errors with a non-existing 'eu' sound

## 8.5.21 SPPAS 2.3

(2019, 25th June)

### Development

* JSON configuration files of sppas modified.

* package "files" is used: the annotations manager is using a workspace instead of a list of files.
* the GUI based on wx4 is managing properly the workspaces and the annotations: a page with "Files" is displayed and allows to "add/remove/delete" files in the list, "import from/export to/pin&save/rename" workspaces, to "create/edit/delete" references and to "check with filters/check all/associate/dissociate" files and references.

* Workspaces are saved in "JSON" format in the "workspaces" folder.

## Annotations

* New annotation "Activity". This annotation was previously available as an option of "Alignment".

* Text Normalization: the Num2Letter module has been fully re-programmed.

* Text Normalization: new option "occurrence & duration" to estimate the number of tokens of each IPU and the duration of this latter.

* Annotations are categorized as: STANDALONE, SPEAKER or INTERACTION. The GUI based on wx3 only shows the STANDALONE ones, but the GUI based on wx4 can deal with all of them.

## Resources

* New folder "num" with dictionaries of numbers for cmn, eng, fra, ita, jpn, khm, pol, por, spa, vie.

## 8.5.22 SPPAS 2.4

(2019, 26th June)

### Development

* the GUI based on wx4 is managing the plugins

* bug correction in 'files' package

## Annotations

* New annotation "Re-Occurrences", of type INTERACTION. Can only be used with the CLI or the GUI based on wx4.

* Either one or two new options in all the annotations:

- inputpattern: to choose the pattern of the input file (like "-token" for Phonetization)

- outputpattern: to choose the pattern of the output file (like "-token" for Tokenization)

* CLI: the script annotation.py is fully re-implemented

## Resources

* New pronunciation dictionary for Iranian Persian (pes), for Phonetization

* New acoustic model for Iranian Persian (pes), for Alignment

## 8.5.23 SPPAS 2.5

(2019, 30th July)
**Development**

* the GUI based on wx4 can convert files.

**Annotations**

* New annotation "RMS" to estimate the Root-mean square of an audio file in given intervals

**8.5.24 SPPAS 2.6**

(2019, 1st October)

**Various**

* New script `dictsampa.py` to convert a pronunciation dictionary in IPA into SAMPA encoding

* New script `clippingrate.py` to estimate the clipping rate at several factors in sub-parts of the audio file

* New plugin `StatGroups` to estimate distributional statistics on sequences of numbers in annotations.

**Development**

* the GUI based on wx4 can convert files (debugged and extended)

* the GUI based on wx4 can display the content of a file (ListView) in the page "Analyze"

* GUI: "Save all" button debugged in DataFilter (remaining bug in DataRoamer)

**8.5.25 SPPAS 2.7**

(2019, 2nd December)

**Various**

* Add the possibility to remove un-labelled annotations in tiers (in DataFilter)

* Plugin "StatGroups" updated: it can optionally create a tier with the not-selected intervals

**Resources**

* Modified French lexicon: include words of Quebec French

* New Quebec French pronunciation dictionary

* New Quebec French acoustic model

* Modified French syllabification file: added vowels of Quebec French

* Tiedlist of Italien model re-introduced... Alignment of "ita" is once again completely functional

**Development**

* the GUI based on wx4: in "Files", the list of files uses collapsile panels instead of a table
## 8.5.26 SPPAS 2.8

(2020, 17th January)

Two new scripts (`sppas-py3.bat` and `sppas-py3.command`) added to run directly the new GUI based on `py3+wx4`.

### Development

* the GUI based on `wx4`: some debug, improved compatibility with the 3 OS and `DataFilter` and `DataS-tats` implemented in the page "Analyze"

### Annotations

* New annotation: Stop Tags: estimate if tags of annotations are relevant or not. Relevance is estimated like for Other-Repetitions detections.

## 8.5.27 SPPAS 2.9

(2020, 03th April)

This is the last version for which both Python 2.7 and Python 3.5+ are supported. Running SPPAS with Python 2.7 will not be maintained past 2020, May when SPPAS 3.0 will be released.

### Development

* the GUI based in `wx4` is extended and debugged: audio roamer and ipuscriber integrated in the page 'Analyze', a multi-player is added - it can play several media at a time.

### Resources

* updated dictionary and acoustic model of Polish language

### Annotations

* New annotation LexMetric to estimate occurrences and rank

### Known bugs:

In the page 'Analyze', the view "Multi-Player" does not properly displays the annotation boundaries and the scrollbar is not updated when needed.
## 8.6 Migrate to Python 3

**Python 2.7 reached the end of its life.** This version expects Python to 3.x as Python 2.7 is no longer maintained. No new bug reports, fixes, or changes will be made to SPPAS when used with Python 2.

A new setup program allows to install external programs to enable some features of SPPAS. It includes 'wxpython' to enable the Graphical User Interface.

### 8.6.1 SPPAS-3.0

(2020, 20th May)

#### Known bugs

* Does not support Julius 4.5+

#### GUI:

* The version based on wxPython 3 is deprecated. Instead, the one based on wxPython 4 is the default but some features are still not implemented.

* A large amount of debug of the GUI. Remaining bug: In the page 'Analyze', the view "Multi-Player" does not properly update the scrollbar when needed.

* Page annotate: can display the content of any report of the list, can delete existing reports.

#### Development:

* new package "preinstall" and new script preinstall.py to install external programs

* Package "files" renamed to "wkps". Updated classes and methods for a better support of reading and writing wjson files; a version 2.0 is then introduced. Add of a "MISSING" state allowing to properly manage missing entries of a workspace, relative paths added, etc.

* config package, classes and methods fully verified/modified including a new sppasAppConfig() class

* Modified main to enable to launch SPPAS GUI with the command: "python3 sppas"

* new bin "juliusdownload.py" to automatize the installation of julius under Windows.

#### Resources:

* Updated lexicon, dictionary and acoustic model for Naija Language.

#### Automatic annotations:

* Automatic alignment, when EOT, is rescuing a failed interval alignment with the "Tokens" tier instead of the default one.

### 8.6.2 SPPAS-3.1

(2020, 10th July)
**Annotations:**

* new annotation FaceDetection to detect faces in an image

* new annotation FaceLandmark to find coordinates of 68 points on an image representing a face, and only one

* new annotation SpkLexRep to find what are the tokens or phases repeated from one file to another one of the same speaker

* linguistic resources are downloaded and installed with the setup or directly from Ortolang web site

**Others:**

* renamed .deps~ into .app~

### 8.6.3 SPPAS-3.2

(2020, 4th September)

**Known bugs**

* Under Windows, when a ComboBox is clicked, the Log Window is focused.

**Annotations**

* better support of antx files

* new FaceTracking annotation to detect faces of a video, still under development, and not in the GUI.

* transfer the metadata of the input transcription to the output transcription. Reliable only for annotation formats that support metadata.

* face detection: sort detected faces by confidence score

**Development**

* bug correction and tests added

* debug of the features reading with UTF-8 encoding allowing the setup to work properly

* new videodata package to read and manipulate video files.

**GUI**

* The Home page includes links to the SPPAS web site

* some wx.ComboBox replaced by our sppasComboBox to look the same on all platforms

* Keyboard shortcuts added to message dialogs:

- ESC: Cancel

- Return: OK

- n: No

- y: Yes

* New keyboard shortcuts in the "Files" page of the main frame:
- alt + a: add files

- alt + e: export workspace

- alt + i: import workspace

- alt + s: pin&save workspace

- alt + n: rename the workspace

- alt + r: create a new reference

- alt + f: check some files

- alt + g: check all files

- alt + l: link checked files to checked references

### 8.6.4 SPPAS-3.3

(2020, 20th October)

#### Package

- ".logs" and ".trash" folders are no longer hidden. Now, they are named "logs" and "trash".

#### GUI

- The page Analyze is simplified: there's no longer a notebook to open files into tabs, and there's no longer different "views". Instead, 1/ to edit the text content of a file, the page "Files" has a new button "Edit checked" in the top toolbar; and 2/ to view files in a time-line style, a new page "Editor" was created. The latter is still under development.

#### Annotations

- Support of versions 4.5+ of the Julius CSR Engine, used by the "Alignment automatic annotation".

- new FaceTrack annotation to detect faces into a video. Currently not fully tested.

#### Development

- new package "videodata" to support video files with opencv.

- preinstall: a different way to search for the python command. It should solve problems when the Python3 executable is "python" command instead of "python3".

### 8.6.5 SPPAS-3.4

(2020, 4th December)

#### GUI

- web links to the reference publications are indicated for each automatic annotations, either in PDF or URL.

- the main app is no longer using the option "useBestVisual" - a user reported that the app crashes with anaconda/ubuntu if enabled.
* some debug and improvement of the page Analyze. Now, it looks better under Windows.

* the page Editor is continued. It allows to:

1. open audio, video and transcription files (no limit on the number of files);

2. play all audio and video files really synchronously: the max delay between media players is 10ms;

3. a slider allows to select 5 different periods to play: either the whole duration, or the visible part, or the selected part, or the visible part until the selected part or the visible part after the selected part.

4. scroll/zoom the files to change the visible part;

5. display annotations both in a spreadsheet and in a timeline;

6. select an annotation;

7. edit the annotation content: a text entry allows to edit the labels of an annotation either in a serialized string, in XML or in JSON.

8. add/delete/split annotations.

9. save.

## Development

* `setup.bat` was modified, expecting it can install `wxpython` even if `python` was not installed from the Windows store. not tested.

* `analyze`: added a new option in the filter system when using relations. It allows to crop annotations of X when they are overlapping Y (overlaps, overlapped by and contains).

## 8.6.6 SPPAS-3.5

(2021, 20th January)

## GUI

* Page Annotate: the installation of new languages and new annotations is enabled.

* Page Analyze:

1. view of the waveform of audio files

2. the files of the timeline can be sorted

3. in the timeline, right-click on a transcription file to check-uncheck tiers

## Annotations

* LexMetric: bug correction of the supported files. No remaining unjustified error message.

* FaceDetection: can detect faces on both an image or a video.

* FaceMark renamed FaceSights: can fix 68 sights of detected faces on both an image or a video.

## Development

* two new scripts: 1/ to convert a folder of images into a video and 2/ to export a video into a folder of images
### 8.6.7 SPPAS-3.6

(2021, 23rd February)

#### Plugins

* cleanipus updated: solved import error and version updated to 1.1

#### Annotations

* INFO1220 migrated from report to logging

* replaced by dummy for unknown phonetizations

* FaceDetection: more options, more models

* New annotation FaceIdentity to assign an identity on detected faces

#### Development

* bug correction in audioframes with an exception not properly raised

* anndata: sppasWEKA renamed to sppasTable and enhanced with export of TRA files – Table Rich Annotations

* installer: added numpy to video feature with version > 1.20 due to the Windows 2004 update and its consequences...

* workspaces: when reading wjson, the references are properly assigned to files

* sppasLPC renamed to sppasCuedSpeech

#### CLI

* the script trstotable.py is replacing trstoweka.py

* other-repetition script can take a workspace as input

### 8.6.8 SPPAS-3.7

(2021, 14th April)

#### Annotations

* creating a merged file is enabled by default

* Overlaps: new INTERACTION annotation to create a tier with overlapping activities.

* Alignment: a better support of non us-ascii characters

* Stopwords: no remaining error about file extension.

* FaceDetection: the default is to load both a DNN and a HAAR model instead of two DNNs. Enlarged portrait size (if option is enabled).

* FaceIdent: can export a smoothed portrait video.

* CuedSpeech: can export the video with the position of the 5 vowels and a number to represent the consonant during the audio key. The resulting tier with the key is made of two labels: the consonant number and the vowel number.
* OtherRepetitions: 3 more tiers about the OR source are created (word strain, nb of labels, source type). Bug correction in the position of the first found echo.

* SelfRepetitions: 3 more tiers about the SR source are created (word strain, nb of labels, source type)

* Linguistic resources and annotation resources are no longer downloaded from ortolang but from gandi instead. Manual download is still on ortolang (when it works!)

## Analysis

* filter system: New filter 'nlab' on the number of labels of annotations

## Development

* bug correction in video coordinates reader

* videodata: the FPS is not converted to integer anymore, now it can be a float. Support of .mov files with H264 codec.

* updated sppasui.json to re-rank annotations of faces

* images: extended support with transparency and overlay added

## GUI

* page files: bug correction when adding a folder, automatically adjust the column width of the filename when adding new files

* page analyze: new filter added to the SingleFilter, modified files can be saved

* page editor: annotation list view and labels edit re-organized, a new rise panel for the timeline view of file allows to see action buttons when collapsed instead of when expanded. No remaining infinite loop (macos, linux) when displaying tiers. A proportional frame when reading a video.

* Problem with foreground color of lists solved (macos, linux)

## 8.6.9 SPPAS-3.8

(2021, 26th May)

## Annotations

* New annotation IVA - Interval Values Analysis.

* Lexical Metric: Add the number of annotations and the number of labels in segments. Segments are fixed from a list of separators.

* Fill in IPUs: more options to fix shift start and shift end.

* CuedSpeech: bug correction in key generation at the end of an IPU with a consonant.

* CuedSpeech: return 2 different tiers with the key, either with a single combined label or with two different labels (one for C then one for V).

* Alignment: a better way to convert point to intervals in case of alignment without audio.

* Other Repetitions: optionally can add a tier with all echo candidates.

* Extended search for syllables tier name: it allows to apply TGA to any tier with 'Syll' in its name.
## GUI

* buttons have new custom events: it allows to customize their look when mouse is entering, leaving, ...

* page editor: annotation boundaries can be moved. The selected bound is in green if several annotations share the same time value, or in red if the bound corresponds to only one time value. Both the midpoint and the radius can be modified.

* page editor: can create an annotation in the timeline by dagging with the mouse.

* page editor: Some minor changes in the look of the page.

* page editor: bug correction with threads in video player under MACOS. Allows to play several videos at a time, like on the other systems.

* page editor: bug correction in the list of annotations when adding before.

* page files: bug correction when adding a non-valid file.

* page files: adding the parent folder ‘..’ is forbidden.

## Development

* preinstall: added linux+microsott in the list of supported distrib for those who installed Ubuntu from the Windows Store

* a global sppasLogSetup() instance is declared in the config.

* the script slmtrain.py can take sentences from stdin and then write the probas on stdout.

* andata: refactor of sppasRW into sppasTrsRW.

* andata: added a trs_type in all reader/writer classes and in sppasTrsRW file properties. Allows to classify annotated files into: ANNOT, MEASURE or TABLE.

* annotations: by default, automatic annotations can only create files of the type ANNOT (except if overwrite the method).

* config: add a log error message if no write access to sppas directory.

## Scripts

audioinfo: new options to get clipping values in details or no clipping values at all

## 8.6.10 SPPAS-3.9

(2021, 6th July)

## Annotations

* More logging messages for a better understanding of the automatic annotations procedures

* bugs corrections in RMS and SpkLexRep

* The “merge” can now be checked/unchecked in the page to annotate. If checked, it merges only checked files of each root.

## GUI

* Under MacOS, the GUI application is not crashing anymore on exit - previously it was randomly either not crashing, or bus error or segmentation fault

* Make the GUI work correctly with the MacOS Dock
* Page File: Edit files button is moved to the vertical toolbar and two buttons are added to remove missing files and to add all files sharing checked roots

* Page Editor: An elaborated search dialog is available in the editor in order to find label patterns in annotations of one or several tiers

## Development

* setup for Windows is simplified

* a new class autils.SppasFiles() allows a better control of the files to manage in the automatic annotations

* annotations: No longer required vs optional inputs. A more generic solution is used instead in all the annotations

# 8.7 Stable

Python 2.7 is no longer supported.

## 8.7.1 SPPAS-4.0

(2021, 5th October)

** One of Python 3.6 or 3.7 or 3.8 is required.**

## GUI

* sppas.bat: do not launch pythonw3 anymore but python3 in order to display error messages if any

* GUI for wx3+py2 is definitively removed

* A new “Colored” icons theme

* Translation enabled under MacOS

* Bug correction: the settings are not modified anymore when cancel is clicked

* Fade in and fade out of dialog windows can be fixed into the settings

* The splash delay can be fixed into the settings

* The Page Editor can play synchronously audio and video files

* Page Editor: Bug correction in displaying the waveform. It is now properly synchronized on the timeline.

* Page Editor: Bug correction in the annotation label editor which can now write special chars - very useful for an enriched transcription.

* Page Files: Remove missing files also remove remaining missing roots and paths.

## Development

* config: simplified use of locale. We now use language but not country (en_US => en ; fr_FR => fr)

* gui: the application has a new custom splash with a gauge to indicate the loading progress

* gui: the install app is updated – code cleaning, better macOS support, etc

* annotations: bug correction in Face Detection because in some videos an image of the buffer can be None

* annotations: bug correction of input pattern management in LPC, Syllabify, FaceIdent, Repet

* annotations: JSON config file modified for LPC and Syllabify
* tips moved to structs package (they are now unused)

* `ui`: only one `gettext` message definition of `_` in `init`. No longer a "msg" definition in each class...

* `ui players`: code re-organized, class diagram simplified, package moved into `ui.phoenix`

* `videodata`: a faster solution to seek at a `pos` when the `pos` is closely later

* `preinstall`: the installer uses `sys.executable` to get the appropriate python

## Annotations

* `CuedSpeech`: use a different code to represent vowels and consonants. The new configuration file is not compatible with previous versions of SPPAS.

## 8.7.2 SPPAS-4.1

(2021, 25th November)

### Known problems

* 'audioplay' feature is not available for python 3.9 or 3.10 because `simpleaudio` pypi package is not available. Another solution has to be found...

* ` opencv-contrib 4.5.4+ ` is not supported. The authors changed something in the library. Some tests have to be performed to find what exactly changed and how SPPAS can deal with. ` opencv-contrib ` is required by the 'video' feature.

### Package

* The XRA 1.5 scheme is described in a PDF documentation file (16 pages)

* The package includes the PDF of the documentation of resources

* The package includes a PDF with the workflow of 19 auto. annotations

### GUI

* `page edit`: the waveform is not drawn in an over-period anymore.

* `page edit`: new keyboard shortcuts in the timeline to:

- zoom in (`ctrl+i`),

- zoom out (`ctrl+o`),

- scroll before (`ctrl+left`)

- scroll later (`ctrl+right`)

- select the next annotation (`Alt+RIGHT`)

- select the previous annotation (`Alt+LEFT`)

## Annotations

* Search for IPUs: the min sil value is taking into account the shift values. In the new results, none of the silences will have a duration less than the given one.
* By default, the Face Detection, Face Landmark and Person Face Identity are creating an XRA file instead of a CSV one.

* Person Face Identity is creating a video and an XRA of the portrait of each identified person. It also has a better selection of identities, based on both coordinates overlap and image similarity.

* The “do merge” option is turned off by default when using either the GUI or the CLI

* An annotation development progress indicator is introduced. This is a percentage value defined into the annotation JSON files and displayed in the GUI if no reference is available.

* Person Face Identity: two different videos are created when 'tag' option is enabled, one without the tagged face and one with.

## Resources

* Support of Bengali language.

## Development

* an exception is raised at the **init** stage if python is deprecated.

* `anndata.annlocation`: bug correction in filtering system with overlaps intervals

* `anndata`: Two new types are introduced for a `sppasTag`: it can be a "fuzzy point" or a "rectangle"

* `anndata`: A `sppasLabel` has a new member 'key' (string).

* new XRA format. Version 1.5 can take into account the 2 new types of a `sppasTag` and the 'key' attribute of a label.

* bug correction in the workspaces: the application captures the `ValueError` that is raised when using `" relpath"` on Windows OS if the path is on a different drive.

* bug correction in the workspaces when adding brothers in roots

* Less restrictions in the definition of a pattern filename (now it can have a single char).

* `"sppas"` is not longer added in the python path by the **init**

* models package migrated into 'Align' to provide circular imports. The training of ACM models is updated.

* Bengali language added to the `num2letter` module

* `fileutils.py` migrated into `utils` package; `makeunicode.py` migrated into `config` package

* annotations: bug correction with exception `EmptyOutput` and an added message in `TextNorm` and `Phon` (name of the tier).

* `structs`: bug correction in `lang list`. It isn't ignoring the resource name anymore.

* `videodata`: bug correction in the `videowriter.set_size()` in order to write videos of a custom size.

* `sppas.json` renamed `sppasmeta.json` with a few meta added

* `install`: takes the version into account when installing a `pypi` package

* `setup.bat`: add a 5 seconds delay after `wx` is installed in order to let Windows OS realize that something changed...

## 8.7.3 SPPAS-4.2

(2022, 28th January)

### Known problems

* This version (and the previous ones) does not work with Python 3.10.x
* 'audioplay' feature is not available for python 3.9 or 3.10 because simpleaudio pypi package is not available. Another solution has to be found...

* The use of Python 3.6, 3.7 or 3.8 is then recommended.

## Package

**Important** The setup is now creating a Python Virtual Environment in the hidden folder `.sppaspyenv~`. Then, the setup has to be launched each time a new SPPAS package is installed. The spps commands are launching the GUI with Python of this virtual environment.

## Annotations

* **Syllabification**: a new option to output one more tier with syllable structures

* **CuedSpeech**: code cleaning and re-organization, two more tiers are created: HandShapes and HandPositions. They are predicting respectively the moments D1-D2 (change of shape) and M1-M2 (change of position). The prediction model is a set of rules based on the work described in (Attina, 2005), page 117 and page 136. It can export a video with badges moving and standing depending on this model, or with pictures of a hand when the debug logging level is set (in the `.app~` file).

* **FaceDetection**: can choose the model(s) to enable or disable in the options. MediaPipe Face Detection is added to one of the possible recognizers - it's very fast but results are worse than other models.

* **FaceSights**: MediaPipe Face Mesh is used to predict the 68 face sights (weight=6) interpolated with OpenCV LBF (weight=1). No longer use of the DAT model nor the empirical sights.

* **PersonFaceIdentity**: A new filter is added to remove un-relevant known identified persons, when number of stored images is less than 1/3 of the expected ones. Added options: mincoordinates, minrefcoordinates, mindistcoordinates, mindistimgs, nbfrimg. And now, the person is identified with the automatic face recognizer first, then with the coordinates instead of the contrary.

## GUI

* **Page Edit**: new buttons in the player control panel to play one frame of a video in both ways (next or previous frame)

* **Page Files**: a better view of the filename in the tree, particularly under Windows, and the pattern is now in a specific column.

## Development

* the "video" feature is now also requiring MediaPipe - https://mediapipe.dev/#!

* workspaces: extended max length of the key of an attribute. It was 12, it's 20.

* features.ini and object detection: can support all versions of opencv-contrib. They changed the output of the detection data structure at version 4.5.4.

* imgdata: the objects detection can load all models then enable or disable any of them.

* Sights and SightsWriter migrated into imgdata package.

* spps.command is 'cd' to $SPPAS in order to make spps.command working properly with the Finder on MacOS and is creating a 'symlinks' virtual environment on MacOS because it does not support the 'copies' option

* setup.bat: search for python.exe first.
### 8.7.4 SPPAS-4.3

(2022, 22th March)

#### Known problems

This version has an “import” bug: it does not start at all on some computers.

#### Resources

* Updated French acoustic model, added French pronunciation dictionary for standard French with name fre.dict, and updated documentation of French resources.

#### Annotations

* MediaPipe (https://mediapipe.dev/) can be used by Face Detection and is used by default in the Face Landmarks automatic detection. It is installed in order to enable the video feature.

* Face landmarks: smooth sights trajectory

* Search for IPUs: the default output pattern is -ipus

* RMS: can optionally add 2 tiers with min and max value of each interval

* CuedSpeech: a new tier is created with the coordinates of the vowels (if sights are given)

* If any, add properly the language in the metadata of the output transcription file, and add the linguistic resource filename in the metadata of the tier.

#### Development

* anndata: bug correction when reading and writing .lab files

* plugins: audioseg bug correction when searching for the annotated file and a new option is added.

* plugins: classphones bug correction in imports and new options.

* GUI-Edit: the player control can play frame-by-frame with a step value, not only 1.

* facesights & faceidentity: smooth points trajectory with another algorithm

* cuedspeech: what, when and where files separated in 3 packages

#### GUI

* updated icon: a more modern version with fewer colors

* updated splash image in order to add a representation of both a video and a hand

* plugins: solved a problem of page refresh under Windows when adding or deleting a plugin

#### Scripts

* formants.py creates a file with f1, f2, f3, f4 of all phonemes. It requires Praat and to install the pypi of ParselMouth.

* saplay.py plays an audio file or a part of it. It uses either pyaudio or simpleaudio - one of them must be installed with pypi.
* `img_toCartoon.py` is able to cartoonize an image or a video.

* `sppasseg.py` allows to perform the full automatic segmentation process. It's a three automatic steps process:

1. Prepare the data (if required):

- Nothing is done if the ortho. transcription is already time-aligned into the IPUs;

- Fill in IPUs if the ortho. transcription is already available but not time-aligned. Notice that it can't work if the recording protocol was not very very very strict: it works only if the number of sounding segments EXACTLY matches the given number of transcribed segments.

- Search for IPUs and auto. transcribe with the STT DeepSpeech. It requires installing DeepSpeech in the currently used Python and to give the path of the models. Get it here (DeepSpeech + English model): https://github.com/mozilla/DeepSpeech/releases/ or here for another language: https://discourse.mozilla.org/t/links-to-pretrained-models/62688

2. Perform Speech Segmentation:

- no option to configure the annotations. They are all fixed internally!

- can estimate an intensity coefficient;

- can predict silent pauses (the ones into the IPUs).

3. Format the output (if asked):

- can map the phoneme into another set (IPA, Praat, ...) if a mapping table is given. It's a file with two columns, both separated by a whitespace: 1st is SPPAS phoneme, 2nd is expected phoneme.

- can create a data structure with all the results and write into a JSON file.

### 8.7.5 SPPAS-4.4

(2022, 5th April)

#### Known problems

The use of Python 3.6 or 3.7 is possible but **Python 3.8 is recommended** because: - the feature “audioplay” can’t be enabled with a more recent version, - recent versions of MediaPipe have better results and the most recent one is available for python 3.8.

#### Annotations

* `Hand&Pose`: a new automatic annotation of an image or a video. SPPAS is a wrapper for MediaPipe Hand detection and Mediapipe Pose detection. It also proposes a custom solution in order to detect right-left hands of a person.

### 8.7.6 SPPAS-4.5

(2022, 15th April)
## Known problems

For a full experience of SPPAS **using of Python 3.8 or 3.9 is recommended**. The use of Python 3.6 or 3.7 is possible.

## Install

* Under Windows, pipwin is installed and used when pip failed to install a package. pipwin installs unofficial python package binaries for Windows provided by Christoph Gohlke here: http://www.lfd.uci.edu/~gohlke/pythonlibs/

* Both simpleaudio and PyAudio are installed - if available for the OS. The "audi" feature is enabled if one of them is successfully installed.

## Package

* a demo is added with all its annotated files. Only the orthographic transcription was performed manually. SPPAS did the rest.

## Resources

* updated French dictionary (fra.dict)

## GUI

* Page Edit: select the nearest annotation when clicking in a hole of a tier

* Page Analyze: when a tier is copied/paste, its media and controlled vocab are also copied/paste to the destination file.

## Annotations

* Alignment: add the aligner name in the metadata of the result, i.e. either julius, hvite or basic.

* FaceSights: bug correction when writing coordinates in a CSV file.

## 8.7.7 SPPAS-4.6

(2022, 22th August)

## Known problems

For a full experience of SPPAS **using of Python 3.8 is recommended** because some of the dependencies are no longer maintained and no new release is published. Other versions of Python are possible but without any warranty.
## Annotations

* CuedSpeech is continued. The Proof of Concept is turned into some stable and tested packages but it is not yet a version 1.

* FaceIdentity: new option to shift the portrait/selfie at left or right

* FaceIdentity and FaceSights: debug of the smoothing. The points are no longer shivering.

* HandPose: bug correction when writing hand sights for an image for XRA format

* Updated demo files with the results of the automatic annotations

## Plugins

* The plugins folder migrated from the base directory to the sppas directory; and each plugin is now a python package.

* audioseg: A version 3.0 is released, an option "video" was added in order to segment both the audio and the video into tracks. Track filenames start with an index instead of the interval number.

## Resources

* updated French dictionary (fre.dict) with new entries added and with "J" instead of "n j" in all words containing "gn"

## Development:

* anndata.tier: bug correction in index() method in order to search for the index of a point in a point-tier.

* preinstall: a new key "pip_opt" in the features.ini file. The installer is now trying pip with option '-pre' if the installation of the stable version failed. The pip_opt allows to set options to pip. It is used by wxPython to fix the pre-build host.

* imgdata cooridinates: added a xy ratio in parameters of portrait() in order to force a ratio between width and height. Default is 14:16.

* andata.label: can optionally add the same tag into a label, without adding their scores

* andata.aio.praat: an appropriate error message in case of chronological textgrid file.

## 8.7.8 SPPAS-4.7

(2022, 12th October)

## Annotations

* CuedSpeech is continued. The transition duration of the shape is reduced (d2 is before) and the hand picture is not blur on the video when the position is changing.

* Text Normalization: a new option is used by default to estimate the number of tokens in each IPU but only if the token is speech, events are ignored (i.e. dummy, noise, laugh, pause and silence).

## Development

* Python 3.10 compatibility.
* Bug correction with imports if video is not enabled.

* Bugs correction in workspaces: Can append a file with a whitespace before the pattern, like "toto - Copie.txt" and the relative path should work properly.

* Installation: bug correction with quotes when launching a command that solved the installation problem - setup.bat does not need to be launch twice anymore, once is enough!

* Installation: do not stop an installation if a pip package fails to be updated.

* Script mlf2labs.py: a new script to split an MLF file into several lab files.

* The custom event's management changed in the Phoenix UI.

## GUI

* There's no longer a LogWindow in the installation wizard.

* Bug correction in slider and progress dialog (mainly used by the Edit page).

* Revised toolbars.

* Page Files: 2 buttons moved from associate to files (remove missing, add bros) and the delete button migrated from files to associate.

* Page Editor: Search dialog debugged.

## Resources

* Updated list of phones for Bengali language, updated dictionary and acoustic model
# Appendix

## 9.1 List of error messages

SPPAS introduced a system for the internationalization of its messages. In the same time, a quality and a number was assigned progressively to each of them in the packages. This system is aimed at providing the users with appropriate error messages and a deeper understanding of the problems.

### 9.1.1 Global errors

**:ERROR 0001:**

This is an un-classified error, with the generic name "sppasError". A message after the error should indicate the problem.

**:ERROR 0513:**

This is the sppasPermissionError, with the message: access to {place} is denied. It occurs when SPPAS was not able to create a file or a directory in some place of the computer. The most frequent cause is that the user do not have permissions to write in the directory where the SPPAS package is installed, or the rights are configured incorrectly or not configured at all.

To work around this issue, use one of the following workarounds as appropriate for your situation:

1. install SPPAS package in a place you have appropriate rights;

2. ask your OS Administrator to modify the rights of the directory in which SPPAS package is installed;

3. launch SPPAS with Administrator rights (not recommended).

### 9.1.2 From "annotations" package

**:ERROR 1010:**

Unknown option with key {key}.
**:ERROR 1020:**

This error has the message: Empty input tier `{name}`. It occurs when an automatic annotation is expecting an annotated tier but the given one does not contain annotations.

**:ERROR 1030:**

Missing input tier. Please read the documentation.

**:ERROR 1040:**

Bad input tier type.

**:ERROR 1050:**

Inconsistency between the number of intervals of the input tiers. Got: `[:d]` and `[:d]`.

### 9.1.3 From "utils" package

:ERROR 1210: The directory `{dirname}` does not exist.

:ERROR 1220: The directory `{dirname}` does not contain relevant data.

### 9.1.4 From "audiodata" package

:ERROR 2000: No audio file is defined.

:ERROR 2005: Audio type error: not supported file format `{extension}`.

:ERROR 2010: Opening, reading or writing error.

:ERROR 2015: No data or corrupted data in the audio file `{filename}`.

:ERROR 2020: `{number}` is not a right index of channel.

:ERROR 2025: From `{value1}` to `{value2}` is not a proper interval.

:ERROR 2050: No channel defined.

:ERROR 2060: Channels have not the same sample width.

:ERROR 2061: Channels have not the same frame rate.

:ERROR 2062: Channels have not the same number of frames.

:ERROR 2070: Invalid sample width `{value}`.

:ERROR 2080: Invalid frame rate `{value}`.
### 9.1.5 From "calculus" package

:ERROR 3010: Both vectors p and q must have the same length and must contain probabilities.

:ERROR 3015: Value must range between 0 and 1. Got {:f}.

:ERROR 3016: Probabilities must sum to 1. Got {:f}.

:ERROR 3025: Error while estimating Euclidian distances of rows and columns.

:ERROR 3030: The given data must be defined or must not be empty.

:ERROR 3040: Value {value} is out of range: expected value in range [{min_value},{max_value}].

### 9.1.6 From "plugins" package

:ERROR 4010: Missing plugin configuration file.

:ERROR 4014: Missing section {section_name} in the configuration file.

:ERROR 4016: Missing option {:s} in section {:s} of the configuration file.

:ERROR 4020: Unsupported plugin file type.

:ERROR 4024: Unsupported plugin file type.

:ERROR 4030: A plugin with the same name is already existing in the plugins folder.

:ERROR 4040: No plugin with identifier {plugin_id} is available.

:ERROR 4050: No such plugin folder: {:s}.

:ERROR 4060: A plugin with the same key is already existing or plugin already loaded.

:ERROR 4070: {command_name} is not a valid command on your operating system.

:ERROR 4075: No command was defined for the system: {:s}. Supported systems of this plugin are: {:s}.”””

:ERROR 4080: No option with key {:s}.

### 9.1.7 From "resources" package

**:ERROR 5005:**

Encoding error while trying to read the file: {name}.

**:ERROR 5010:**

Error while trying to open and read the file: {name}.

**:ERROR 5015:**

Read file failed at line number {number}: {string}.

**:ERROR 5020:**

The n value of n-grams pattern matching must range [1;{maximum}]. Got {observed}.
**Others:**

:ERROR 5022: The gap value of pattern matching must range [0;{maximum}]. Got {observed}.

:ERROR 5024: The score value of unigrams pattern matching must range [0;1]. Got {observed}.

:ERROR 5030: The dump file can't have the same extension as the ASCII file ({extension}).

:ERROR 5040: The count value must be positive. Got ({count}).

### 9.1.8 From "structs" package

:ERROR 6010: {meta} is not a known meta information.

:ERROR 6020: Unknown resource type: expected file or directory. Got: {string}.

:ERROR 6024: The resource folder {dirname} does not exist.

:ERROR 6028: The language must be "und" or one of the language list. Unknown language {lang}.

### 9.1.9 From "models" package

:ERROR 7010: Expected a {data_name} of type {expected_type}. Got {data_type} instead.

:ERROR 7500: The file {!s:s} contains non UTF-8 characters: {s}.

:ERROR 7505: Fail formats: unrecognized file format {!s:s}.

:ERROR 7510: Fail formats: the folder {!s:s} does not contain a known model.

:ERROR 7515: No model found or empty model in {!s:s}.

## 9.2 GNU Free Documentation License

Version 1.3, 3 November 2008

Copyright (C) 2000, 2001, 2002, 2007, 2008 Free Software Foundation, Inc. http://fsf.org/

Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.

### 0. PREAMBLE

The purpose of this License is to make a manual, textbook, or other functional and useful document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.

This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.

We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software
does. But this License is not limited to software manuals; it can be used for any textual work, regardless of
subject matter or whether it is published as a printed book. We recommend this License principally for works
whose purpose is instruction or reference.

# 1. APPLICABILITY AND DEFINITIONS

This License applies to any manual or other work, in any medium, that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. Such a notice grants a world-wide, royalty-free license, unlimited in duration, to use that work under the conditions stated herein. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as "you". You accept the license if you copy, modify or distribute the work in a way requiring permission under copyright law.

A "Modified Version" of the Document means any work containing the Document or a portion of it, either
copied verbatim, or with modifications and/or translated into another language.

A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (Thus, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.

The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License. If a section does not fit the above definition of Secondary then it is not allowed to be designated as Invariant. The Document may contain zero Invariant Sections. If the Document does not identify any Invariant Sections then there are none.

The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License. A Front-Cover Text may be at most 5 words, and a Back-Cover Text may be at most 25 words.

A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, that is suitable for revising the document straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup, or absence of markup, has been arranged to thwart or discourage subsequent modification by readers is not Transparent. An image format is not Transparent if used for any substantial amount of text. A copy that is not "Transparent" is called "Opaque".

Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML, PostScript or PDF designed for human modification. Examples of transparent image formats include PNG, XCF and JPG. Opaque formats include proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML, PostScript or PDF produced by some word processors for output purposes only.

The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to
hold, legibly, the material this License requires to appear in the title page. For works in formats which do not
have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text.

The "publisher" means any person or entity that distributes copies of the Document to the public.

A section "Entitled XYZ" means a named subunit of the Document whose title either is precisely XYZ or contains XYZ in parentheses following text that translates XYZ in another language. (Here XYZ stands for a specific section name mentioned below, such as "Acknowledgements", "Dedications", "Endorsements", or "History".) To "Preserve the Title" of such a section when you modify the Document means that it remains a section "Entitled XYZ" according to this definition.

The Document may include Warranty Disclaimers next to the notice which states that this License applies to the Document. These Warranty Disclaimers are considered to be included by reference in this License, but only as regards disclaiming warranties: any other implication that these Warranty Disclaimers may have is void and has no effect on the meaning of this License.

## 2. VERBATIM COPYING

You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may publicly display copies.

## 3. COPYING IN QUANTITY

If you publish printed copies (or copies in media that commonly have printed covers) of the Document, numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.

If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.

If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a computer-network location from which the general network-using public has access to download using public-standard network protocols a complete Transparent copy of the Document, free of added material. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.

It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.
## 4. MODIFICATIONS

You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:

* A. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission.

* B. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has fewer than five), unless they release you from this requirement.

* C. State on the Title page the name of the publisher of the Modified Version, as the publisher.

* D. Preserve all the copyright notices of the Document.

* E. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.

* F. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below.

* G. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice.

* H. Include an unaltered copy of this License.

* I. Preserve the section Entitled "History", Preserve its Title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section Entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence.

* J. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission.

* K. For any section Entitled "Acknowledgements" or "Dedications", Preserve the Title of the section, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein.

* L. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles.

* M. Delete any section Entitled "Endorsements". Such a section may not be included in the Modified Version.

* N. Do not retitle any existing section to be Entitled "Endorsements" or to conflict in title with any Invariant Section.

* O. Preserve any Warranty Disclaimers.

If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles.
You may add a section Entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties—for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.

You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version.

## 5. COMBINING DOCUMENTS

You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice, and that you preserve all their Warranty Disclaimers.

The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections Entitled "History" in the various original documents, forming one section Entitled "History"; likewise combine any sections Entitled "Acknowledgements", and any sections Entitled "Dedications". You must delete all sections Entitled "Endorsements".

## 6. COLLECTIONS OF DOCUMENTS

You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.

You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.

## 7. AGGREGATION WITH INDEPENDENT WORKS

A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, is called an "aggregate" if the copyright resulting from the compilation is not used to limit the legal rights of the compilation's users beyond what the individual works permit. When the Document is included in an aggregate, this License does not apply to the other works in the aggregate which are not themselves derivative works of the Document.

If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one half of the entire aggregate, the Document's Cover Texts may be placed on covers that bracket
the Document within the aggregate, or the electronic equivalent of covers if the Document is in electronic form. Otherwise they must appear on printed covers that bracket the whole aggregate.

## 8. TRANSLATION

Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License, and all the license notices in the Document, and any Warranty Disclaimers, provided that you also include the original English version of this License and the original versions of those notices and disclaimers. In case of a disagreement between the translation and the original version of this License or a notice or disclaimer, the original version will prevail.

If a section in the Document is Entitled "Acknowledgements", "Deductions", or "History", the requirement (section 4) to Preserve its Title (section 1) will typically require changing the actual title.

## 9. TERMINATION

You may not copy, modify, sublicense, or distribute the Document except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense, or distribute it is void, and will automatically terminate your rights under this License.

However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation.

Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice.

Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, receipt of a copy of some or all of the same material does not give you any rights to use it.

## 10. FUTURE REVISIONS OF THIS LICENSE

The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See http://www.gnu.org/copyleft/.

Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation. If the Document specifies that a proxy can decide which future versions of this License can be used, that proxy's public statement of acceptance of a version permanently authorizes you to choose that version for the Document.
## 11. RELICENSING

"Massive Multiauthor Collaboration Site" (or "MMC Site") means any World Wide Web server that publishes copyrightable works and also provides prominent facilities for anybody to edit those works. A public wiki that anybody can edit is an example of such a server. A "Massive Multiauthor Collaboration" (or "MMC") contained in the site means any set of copyrightable works thus published on the MMC site.

"CC-BY-SA" means the Creative Commons Attribution-Share Alike 3.0 license published by Creative Commons Corporation, a not-for-profit corporation with a principal place of business in San Francisco, California, as well as future copyleft versions of that license published by that same organization.

"Incorporate" means to publish or republish a Document, in whole or in part, as part of another Document.

An MMC is "eligible for relicensing" if it is licensed under this License, and if all works that were first published under this License somewhere other than this MMC, and subsequently incorporated in whole or in part into the MMC, (1) had no cover texts or invariant sections, and (2) were thus incorporated prior to November 1, 2008.

The operator of an MMC Site may republish an MMC contained in the site under CC-BY-SA on the same site at any time before August 1, 2009, provided the MMC is eligible for relicensing.