# Transcribing sign languages with TYPANNOT: the typographic system that retains and displays layers of information

Claire Danet, Dominique Boutet, Patrick Doan, Claudia S. Bianchini, Adrien Contesse, Léa Chévrefils, Morgane Rébulard, Chloé Thomas, Jean-François F Dauphin

<figure><img src="image_1.png" /></figure>

## To cite this version:

Claire Danet, Dominique Boutet, Patrick Doan, Claudia S. Bianchini, Adrien Contesse, et al.. Transcribing sign languages with TYPANNOT: the typographic system that retains and displays layers of information. Grapholinguistics and its applications, 2021, Grapholinguistics in the 21st century 2020, 5 (2), pp.1009-1037. 10.36824/2020-graf-dane. hal-02973783

HAL Id: hal-02973783

https://hal.science/hal-02973783

Submitted on 21 Oct 2020

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
Transcribing sign languages with TYPANNOT: the
typographic system that retains and displays layers of
information

ABSTRACT

There are more than 140 sign languages (SLs) in the world and studying them is a relatively recent field of research (starting in the 1960s). Linguists have the need to represent the different levels of gestures that make up the signs in order to analyze the way SLs work. Such transcription requires the use of a dedicated graphic system (Slobin et al. 2001).

TYPANNOT, the transcription system presented in this article, is a typographic system that allows the description of all formal features of SLs. Our contribution to the field of grapholinguistics is a phonological model and a transcription system for SLs that are rooted in the articulatory possibilities of the signer's body. Compared to existing graphematic systems, our approach of SLs description is both phonological, allowing descriptions of the different articulatory structures (low level) involved in SLs, and logographical, allowing users to read the transcriptions from a unified perspective (high level).

We will detail the design principles that drive the development of such a typographic system, the graphemic model that derives from linguistic study, and the tools that allow researchers to use TYPANNOT to its fullest capacities.

This article also outlines the kinesiological approach (Boutet, 2018), which TYPANNOT uses, noting radical changes in the way researchers should look at meaning through gesture. This approach opens new perspectives in researching movement itself as a central source of meaning in human communication via gesture.

Keywords: Sign languages, Transcription, Typographic system, articulation, Input interface, TYPANNOT
# Introduction

SLs, of which there exist at least 144 worldwide (Ethnologue.com, 2020), are gestural languages with grammatical/linguistic structures based on body expression (Sandler and Lillo-Martin 2006). This visuogestural modality means that SLs work in very different ways compared to vocal languages (VL). First, SLs are 4-dimensional (3 space dimensions plus time) by nature and employ spatialization, meaning that grammatical elements manifest in space like on a scene, for example verbs appear to move from the sender to the receiver. Second, SLs can articulate several gestures in parallel, using the entire body to express multiple information simultaneously (Braffort, 1996). If VLs can be seen as monolinear, meaning only one piece of information is being communicated at a time, SLs can be seen as plurilinear (Cuxac, 2001). Third, semiotically speaking, SL signs are iconic: their appearance tends to resemble some aspect of the thing or action being denoted.

Since the 1960s, with the development of linguistic research and the recognition within hearing communities that SLs are full-fledged languages, most linguists have agreed on the phonological<fnref n="1" /> decomposition of SLs into two parameters: a) manual, such as the configuration of the hand, its orientation in space, its location and its movement; and b) non-manual, such as gaze, facial expression, and torso posture.

Unfortunately, this consensus didn't translate into the conception and the adoption of a unified transcription system for SLs. Until today, the multi spatial and multi parametric properties of SLs cannot be properly represented by neither a dedicated symbolic system nor a VL description system.

Nevertheless, various attempts for writing SLs do exist: e.g. in France, many historical instances can be cited, such as “Mimography” (Bébian, 1825) and “D'Sign” (Jouison, 1995), or more recently “Signography” (Haouam-Bourgeois, 2007), “Schematization” (Guiteney, 2007) and “SL-video” (Brugeille, 2007). These different graphic forms were created to compensate for the lack of traceability in situations such as teaching SL or sharing artistic expressions (poetry, sign-singing, theater pieces, etc.). Bianchini (2012) even considers that SL writing would be an additional route for hearing people to enter the Deaf world.

With the birth of SL linguistics, and in particular after the research of William Stokoe (1960), different notation systems have emerged. Some were created with the aim of detailed analysis and transcription (among them, Stokoe, 1960; Prillwitz et al., 1989); these are phonographic systems, in which each grapheme transcribes a phoneme. Other, more logographic systems (a grapheme representing an entire lemma, e.i. word), like SignWriting (Sutton, 1995) or Si5s (Augustus, 2010), represent a more functional and accessible approach for the Deaf community.

Whatever the motivation for finding ways to represent SLs is, their visuogestural natures, their spatial and temporal dimensions, as well as their plurilinearity transform the task into a particularly daunting challenge (Boutet et al., 2018).

Keeping in mind those specificities, the GestualScript team at ÉSAD-Amiens<fnref n="2" /> reviewed the existing linguistic models and graphematic systems in order to understand their underlying strengths and limitations (§1). This work fueled the design thinking behind the conception of TYPANNOT (§2), a typographic system that takes advantage of new technologies to tackle SLs representation problems while adopting a radical perspective in order to completely revisit the existing descriptive models (§3).

# 1 Existing SL representation for various scopes

Although pursuing different objectives, SignWriting (SW) and HamNoSys are two examples of notation systems that both rely on a parametric approach to organize the representation of SLs. While SW aims to offer writing within the framework of teaching with SL, HamNoSys is focused on

<footnote n="1">¹ Here and throughout the article, phonological is of course referring to the phonetics and phonemics of SLs, in which visual form is abstracted into units of meaning, or phonemes, and we are not using phonological to mean the science of speech sounds.</footnote>

<footnote n="2">² The GestualScript team, based at the De-sign-e lab of the École Supérieure d'Art et de Design (ÉSAD) d'Amiens, is an interdisciplinary group made of linguists (D. Boutet†, C. Danet, C. S. Bianchini, L. Chevrefils, C. Thomas), designers (P. Doan, M. Rébulard, A. Contesse), and a computer scientist (J-F. Dauphin). The team's research was partly funded by the French DGLFLF and the Department of Culture.</footnote>
transcribing SLs in order to analyze them systematically. These two distinct perspectives yield different yet complementary principles of graphic representation.

## 1.1 SignWriting (SW): a representation system for recognizing SL

In 1974, Valerie Sutton conceived SW, inspired by her previous DanceWriting (1966-74) work and driven by the linguistic research carried out at the University of Copenhagen (Sutton, 2020). This system is aimed at both the teaching and the everyday practice of SL, and is characterized by an anthropomorphic representation of the sign in an attempt to offer a proxy of reality.

First, SL signs are represented by distinct graphic units (graphemes) that correspond to the minimum units that carry meaning in the structure of the language (phonemes), and which take up the main formal characteristics (shapes of the hand, eyes, arms, etc.); this is a so-called phonological level of deconstruction. Next, these graphemes are arranged analogically to the sign space in a thumbnail called a “vignette” (Fig. 1). They take up the global spatial organization of the SL sign to reproduce its image, like a transfer from reality. This is a so-called logographic level of construction where the different graphemes are brought together to form a unified sign representing a lexical unit.

This makes SW a system with a pictographic tendency since the vignettes reproduce a schematic, stylized, and above all, unified version of the SL sign, thus allowing the user to focus more on the text meaning than on the language structure. This representation, however, can also be considered “alphabetical” since each vignette can be split into glyphs which relate more to phonemes.<fnref n="3" />

<figure><img src="image_3.png" /><figcaption>Fig. 1. Organization of SW glyphs inside a vignette</figcaption></figure>

These glyphs are then arranged non-linearly, leaving the writer considerable expressive freedom, both in the choice of several glyphs that are almost synonymous and of their location. If the only limit to this freedom is keeping the legibility of the thumbnail, this results in a great variability from one writer to another, which limits data comparisons (e.g., when searching for inter-annotator agreement) (Fig. 2).

<figure><img src="image_4.png" /><figcaption>Fig. 2. Freedom in placing some SW glyphs (e.g., movement arrows) may make difficult data comparisons</figcaption></figure>

SW is organized in categories and sub-categories (e.g., configuration of the hand, its movement, its dynamics and coordination, etc.); each containing basic glyphs (about 500 in total) to which specific rules apply, a process that generates nearly 37,000 “conjugated” glyphs.

The description of the elements present in a category or subcategory calls on several frames of reference (FOR, §3.1.2). For example, the movement of the hands is described in an environmental FOR (the movements are directed towards the imaginary walls of a room, the horizontal axis corresponds to

<footnote n="3">³ The question of the exact nature of SW (alphabetic or featural; phonological or phonetic) is still open but not the subject of this article; for further discussion see Bianchini, 2012; 2016.</footnote>
the floor, the vertical one to the height of the walls; Fig. 3a), but those of the head and the body are described in a FoR centered on the speaker (the movements are directed towards the sides of the signer, the horizontal axis corresponds to the shoulders, and the vertical axis to the signer's height; Fig. 3b).

<figure><img src="image_5.png" /><figcaption>Fig. 3. (a) the environmental FOR is used to code hand movement; (b) speaker-centered FOR is used to code head movement; if hand and head movement is coded in the same vignette, two FORs are present. These can be: (c) collinear, if their axes are superimposed; or (d) non-collinear, involving fragmentation of the representation space</figcaption></figure>

The presence of several FORs within the same vignette generates a fragmentation of the sign representation space whenever the axes of the different FORs are non-collinear (not superimposable). If standing, the signer's horizontal axis corresponds to the environmental horizontal axis (the two FORs are therefore collinear; Fig. 3c), but if bent to the side, the signer's shoulders will no longer be parallel to the floor and therefore the horizontal axis will no longer correspond to the horizontal plane of the room (Fig. 3d).

The flexibility of notation and the large number of glyphs make SW an asset for representing many phenomena. Thanks to its visual evocational power, it is the system most used by educators around the world; however, in the absence of a ductus – a defined procedure specifying the number of strokes, the direction, and the sequence for drawing the various symbols – writing SW is much more complex than reading SW. Moreover, this makes it difficult for linguists to obtain inter-annotator agreement as well as the ability to query the vignettes in a database. However, apart from the drawing of SW by hand, there is also online input software (SignMaker<fnref n="4" />) available and the system has been coded under the Unicode standard since 2010.

## 1.2 HamNoSys: a representation system for analyzing SL

Directly focused on researchers, the Hamburg Notation System (Prillwitz et al., 1989), a.k.a. HamNoSys, is a transcription system based on phonological principles, i.e. each parameter is broken down into phonemes which are represented by glyphs. This approach is an evolution of the one adopted by Stokoe Notation (Stokoe, 1960), the first SL notation system.

Compared to Stokoe Notation, HamNoSys offers a more detailed description of SL phonemes and increases the number of examined parameters (e.g., non-manual parameters, locations outside the signature zone, etc.). Phonemes are represented by around 210 more or less iconic symbols, but while some have their own symbol, others are obtained by composing a basic form with diacritics (e.g., the hand configuration, the movement, etc.). This phonographic system is intended to be inter operational, and therefore aims at international use, compatibility with standard computer display and indexing tools, extension capacity, ergonomic syntax according to the principles of compositionality, syntactic efficiency (e.g., the principles of symmetry), and an iconicity of symbols (for ease of memorization) (Hanke, 2004).

The graphemic equation puts the sign information in linear order from left to right, according to a strict syntax (Fig. 4).

<footnote n="4">⁴ SignMaker (http://www.signbank.org/signmaker.html); for an analysis of the interface see Bianchini et al., 2012.</footnote>
<figure><img src="image_6.png" /><figcaption>Fig. 4. Organization of HamNoSys glyphs in an equation</figcaption></figure>

Like SW, HamNoSys changes its point of view. For example, the glyphs representing the hand orientation and movement are related to three perspectives, one from the signer's point of view, the others from above or from the right (Fig. 5, from Hanke, 2004, p. 2).

<figure><img src="image_7.png" /><figcaption>Fig. 5. Multiple points of view within HamNoSys</figcaption></figure>

To use HamNoSys, it is possible to download a font and a dedicated virtual keyboard. HamNoSys is also coded under the Unicode standard. SL signs are encoded in a fully linearized typographic form, which gives the system great flexibility and compatibility with computer tools for displaying and indexing data. However, HamNoSys can be complex to use, especially during the decryption phase, due to the amount of parameters to be processed and the way the characters are composed.

## 1.3 Conclusions: an external perspective of SL representation

This short presentation of SW and HamNoSys has shown the advantages associated with the two main modes of representation that characterize them. The phonographic approach uses a limited inventory of signs corresponding to the SL phonological structure to transcribe them in an efficient and detailed manner; conversely, the logographic approach offers a synthetic and evocative graphic representation by visually transposing the semiotic dimensions of a corporal FOR inherent to all SLs.

It becomes clear that a semiography (linguistic sign notation that refers to the semantic level of a language) that could combine phonographic and iconic trends would be advantageous. On one hand, the phonological structuring makes it possible to isolate the distinctive elements, thus providing an efficient and functional system necessary for transcription. On the other hand, the synthetic and evocative graphic representation preserves the semiotic relationships intrinsically offered by the different spatial and bodily references of these visuogestural languages.

It is worth noting that whatever the pictographic or phonological dominant, the systems of SL representation resort in general to a form of visual iconicity. Indeed, unlike VLs which must use graphic conventions to represent sounds, SLs can find in writing the figurative dimensions directly emanating from the visuogestural modality. Translated graphically, the articulated structure of the body and the forms it produces in space constitute an image that recalls the SL sign all at once. The result is a
remarkably natural spelling, so to speak, but it remains important to ask questions about its strengths and weaknesses.

Indeed, this immediate readability means that the existing systems systematically approach the representation of SL as a projection of perceived shapes on a 2-dimensional plane. Although these languages are deeply rooted and subject to the articulators of the body and their own geometry, the body is viewed from an external point of view where only its apparent parameters are described. The problem here is not so much to have resorted to an external perspective or projection, but to have ignored the intrinsic characteristics of the SL sign. Without them, it is impossible to know what a representation really corresponds to, given the superficiality of the projection. In fact, signs are described according to what the recipient sees and not what the speaker's body is doing.

Therefore, SW and HamNoSys must multiply the points of view to account for a gestural phenomenon. These external views changing from one sign to another (HamNoSys) or even joined in the same description (SW) may lead to misunderstandings in sign reading and inconsistencies in analysis.

The linguistic distinction in manual and non-manual parameters produces dissociation between the different segments of the body, which disconnects them from their bodily transformations. Movement, which is the most reluctant parameter to be used in linguistic description due to its complexity (Boutet, 2018) is then simply considered a trajectory of the hand, yet this manual trajectory - as a trace left behind – cannot by itself contain all the meaning conveyed by the signer's bodily comportment.

# 2 TYPANNOT

## 2.1 Approach and goals

The GestualScript team believes that meaning in SL is driven by the signer’s own activity and that this activity is fundamentally defined by the many ways in which the body can be mobilized and experienced to promote an ongoing dynamic of signification (Varela et al., 1993; Theureau, 2004; Poizat et al., 2013). Although SL gesture refers to cultural and linguistic forms, part of its meaning is fundamentally undetermined and arises through the non-linear, open dynamics of activity. This means that gesture is personally lived and understood at the level of a body that can freely transform, modulate, and interact with those cultural and linguistic forms within the limits of what is possible in terms of movement and signification. TYPANNOT is a novel typographic system, which allows researchers to represent those transformations, modulations, and interactions at the articulatory level, at the point of skeletal joints. Such a musculoskeletal description makes it possible to investigate the semantic processes that arise from elementary gestural phenomenon and that would otherwise be difficult to distinguish. This approach involves a radical shift of perspective that has profound consequences in the way the different gestural components are perceived and represented.

The representation framework of TYPANNOT is based on the kinesiologic model presented in section (§3.2). It distinguishes the different articulatory domains that provide intrinsic representations for the five parameters that structure SL. Four are static articulatory parameters (HS, LOCINI, MOUTHACTION and EYESACTION<fnref n="5" />) and one is a dynamic parameter (MOV) that describes the way an articulatory parameter is being transformed. This representation framework must also meet the practical aspects of transcription which implies processing information in the form of viewable, transferable and searchable textual data. In order to translate this representation framework into a viable typographic system, four requirements have been identified, which guide the design process: genericity, readability, modularity and inscribability.

<footnote n="5">⁵ MOUTHACTION and EYESACTION are two parameters for describing the posture of the mouth and that of eyes and nose.</footnote>
## 2.2 Design principles

### 2.2.1 Genericity

The first requirement directly stems from the phonological transcription approach using an articulatory model of the human body. For each of the articulatory parameters, gesture is deconstructed into discrete elements representing four layers of information (Fig. 6):

* Layer 1: the SL parameter that the transcription refers to (e.g., handshape);

* Layer 2: the different parts that compose the parameter (e.g., thumb);

* Layer 3: the different variables associated with each part (e.g., angle);

* Layer 4: the values assigned to those variables (e.g., open).

Each layer has a limited set of characteristics that defines it, creating individual bricks of information. Once defined, these characteristics form the generic components of the TYPANNOT transcription system. Symbolic graphic representations can be assigned to them and later encoded into a font to perform like letters.

<figure><img src="image_8.png" /><figcaption>Fig. 6. This transcription of a mouth action has been set in generic form and colorized in order to distinguish the four layers of information. The SL parameter (layer 1) is written in black. The parts (layer 2) are written in orange. The variables (layer 3) are written in green. The values (layer 4) are written in blue.</figcaption></figure>

While there might be hundreds of thousands of possible configurations for a parameter (261 million possible configurations for the handshape alone), TYPANNOT requires only a few generic components to describe them all. The systematic organization of the information into four layers also gives the transcription a robust syntax that ensures it can be consistently produced, manipulated, and searched. Finally, through the principle of genericity, TYPANNOT allows annotators to perform queries and comparisons throughout different phonological levels, involving a combination of features or targeting a single one. This kind of deep querying of data is impossible to perform with other SL representation systems.

### 2.2.2 Readability

The TYPANNOT phonological approach aims to provide a discrete and low-level representation of gestures. From a typographical point of view, this is achieved at the cost of linearity. While the generic design principle involves methodically decomposing gestures into a suite of individualized pieces of information, it breaks down the only visuo-spatial guiding perspective that would allow users to read gestures in an intuitive and instantaneous way: the body space, in other words a unified representation of the body. For a language that is fundamentally visual in terms of perception, it is ironic that its representation needs to distill it to its lowest distinctive components, thus making its transcription unreadable. Although logographic systems like SW exist and show how readability in SL can be achieved through a spatialized representation of the parametric components, none are able to retain a high level of discreteness while doing so.

This trade-off limits the main function of a transcription system: analysis. To be relevant to the principles of both genericity and readability, we believe that a SL transcription system needs to be able
to display the same information in two formats: 1) a *generic* form that shows the distinct bricks of information organized through robust linear syntax that allows deep research into the gesture components; 2) a *composed* form that translates and integrates the different phonological components into a recognizable form: the image of the signed body. Progress in font encoding technologies (i.e., Opentype<fnref n="6" /> features) and typographic functionalities (i.e., contextual ligatures) allows us to design a system that gives users the ability to seamlessly display one form or the other while retaining data integrity. For each of the articulatory parameters, we define a specific graphic formula that translates the generic pieces of information into a unified and visually explicit “composed” glyph. For example, the initial location (LOCini) parameter refers to the structure of the upper limbs. This structure is made out of three parts (arm, forearm, hand) that are articulated according to various variables (flexion/extension, adduction/abduction, etc.) and their possible values (neutral, +1, +2, etc.). Displayed in the *generic* form (Fig. 7), the transcription looks like a string of symbols following a linear syntax. Displayed in the *composed* form (Fig. 8), the transcription looks like an articulated structure with joints (shoulders, elbows, wrists) and segments (forearms) forming an expressive figure with the head (triangle). The last segment, the hand, is shown on each side in order to appear bigger and more readable.

<figure><img src="image_9.png" /><figcaption>Fig. 7. LOCini displayed in the generic form (left side only)</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>Fig. 8. LOCini displayed in the composed form (left and right sides)</figcaption></figure>

While the logographic composed form has no analytic function, it reflects the ongoing ethical commitment of our team to provide accessible representation tools for both linguists and signers.

### 2.2.3 Modularity

Designing a typographic system that is both phonological and logographical means that we have to maintain strict equivalence between the two forms. This equivalence can be achieved through using a modular design approach. *Composed* forms are basically projections of intrinsic articulatory characteristics following an allocentric perspective<fnref n="7" />. Graphic modules symbolizing the different parts (i.e., fingers and thumb) of an articulatory system (i.e., the hand) are transformed and assembled according to articulatory characteristics (i.e., form, angle, contact, etc.) inside a framework that systematically replicates the spatial organization of the body (Fig. 9).

<footnote n="6">⁶ OpenType is a vectorial font format that allows encoding any character associated with Unicode, regardless of the platform (Mac, Windows, Android, etc.); OpenType fonts can have advanced typographic features that allow handling complex writing and typographic effects like ligatures.</footnote>

<footnote n="7">⁷ In an allocentric perspective, the position of a body part is defined relative to the position of other body parts (e.g., the position of the hand depends on the position relative to the forearm, which depends on the position relative to the arm). In an egocentric perspective, the position of an element depends on the orientation of the viewer's body (e.g., the hand is in front, on the left and up).</footnote>
<figure><img src="image_11.png" /><figcaption>Fig. 9. 3 composed HS glyphs showing variation in their construction</figcaption></figure>

This modular design principle helps us solve the question of equivalence, and more importantly, allows us to automatize the glyph creation process. To this end, the articulatory approach is synonymous with massive combinatorial possibilities and we are now facing the problem of quantity. For example, the articulatory system of the hand alone can give rise to hundreds of thousands of configurations and thus requires the production of equal amounts of composed glyphs. Manually designing that many glyphs is, for many reasons, impractical<fnref n="8" />. Thanks to our modular framework and scriptable font design environment (i.e., Robofont<fnref n="9" />), we are able to code the module’s integration process in order to generate all the composed forms.

### 2.2.4 Inscribability

The TYPANNOT project aims at providing a tool for the representation of SLs, but also explores the possible relations between the annotator or signer and a written form. While vocal writing systems use conventional graphic principles, SL writing or transcription systems have the unique opportunity to engage in a dialogue between writers and the constitutive dimensions of their own language: the human body. Through the intrinsic perspective of the articulatory models and a typographic system that combines phonographic and logographic dimensions, GestualScript believes that annotators can develop an intuitive bond with their transcription, not by describing “what SL looks like” but by recognizing “how SL happens” and describing it from the inside. Because it is not a familiar way to perceive gesture, this shift in perspective involves the creation of specific input interfaces that promote interactions that elucidate the articulatory and dynamic principles behind it (gesture). While designing those interactive input interfaces (see § 2.4) we are aiming at facilitating the process of incorporating and assigning the transcription systems (Poizat et al. 2013a).

## 2.3 Corpus and user-driven font systems

OpenType fonts can contain up to 65,536 glyphs. With TYPANNOT, following the principle of modularity (§ 2.2.3), generating all morphologically possible combinations of a parameter's elements greatly exceeds the maximum capacity of font glyphs. For example, the automatic composition of TYPANNOT handshapes creates 291,600 glyphs.

To create TYPANNOT fonts, GestualScript has to decide what criteria should be used to reduce the number of possibilities by selecting the most appropriate and relevant glyphs, keeping in mind language evolution. A bottom-up approach<fnref n="10" /> was chosen as the operating principle:

* First, a character set was created using the 237 handshapes identified by Eccarius and Brentari (2008) in their corpus made of confirmed configurations present in lexicons of 9 SLs (Hong Kong, Japanese, British, Swedish, Israeli, Danish, German, Swiss-German and American SL).

* A further development consisted in extending the character set to include a larger sample of signs, thanks to the addition of the configurations listed in the inventory of SW (Sutton, 1995), plus some variants sought for completeness.

* A third, future step will expand the character set in a participative way. By using TYPANNOT, linguists from every background will transcribe handshapes that haven't got yet a composed form. The program will automatically identify and collect those unknown forms in order to plan regular updates of the character set (§ 2.4.3).

<footnote n="8">⁸ One reason among others: at the improbable rate of one compounded glyph per second, it would take 261,000,000 seconds to encode all handshapes, which would require working more than 8 years, 24/7.</footnote>

<footnote n="9">⁹ Robofont is software for typeface creation, which can automatically generate contextual ligatures from graphic modules and instructions on their layout. For the development of TYPANNOT, Frederik Berlaen, developer of Robofont (https://robofont.com), has kindly provided GestualScript with a license to use his software.</footnote>

<footnote n="10">¹⁰ The selection is not based on given linguistic rules but on the occurrences found in various linguistic corpus.</footnote>
This updating procedure will also help us identify and register new handshapes in SLs that are less studied, expending their understanding.

## 2.4 Input systems

Setting up a complete and comprehensible typographic system for SLs was no easy task. The methods described above were essential in the completion of TYPANNOT's goal to offer an efficient solution to enhance linguistic research on SL. Yet, the typographic system by itself is not sufficient. In order to truly come into being, a custom tool that allows researchers to use TYPANNOT to its fullest capacities was needed. Therefore, we are currently shaping the TYPANNOT Keyboard into a digital interface, which will offer several input devices to fit a wide spectrum of transcription approaches.

### 2.4.1 Enhancing knowledge through technology and design efficiency

Creating a digital interface to make TYPANNOT fully accessible goes far beyond the sole possibility of combining glyphs together in order to inject them into office software (e.g., Word, PowerPoint, etc.) or multimodal transcription software (e.g., ELAN).

Such an interface has the responsibility of ensuring that users will understand and use TYPANNOT in a coherent and consistent manner. It is true that a well thought out user experience is always essential for any given tool. But in this case, it goes beyond the necessity of user-friendly software. TYPANNOT, as a new typographic system, needs to be discovered, understood and used in a consistent way. It is an essential part of the process to ensure that transcriptions using TYPANNOT can be understood and used in cross referencing research.

That means that the structure and the interface design have to be engineered to give users key pieces of information about TYPANNOT itself: information on the structure of the typographic system, on the value of each glyph, on how to combine them properly, and what the results signify.

This task can be achieved in various ways. Some are very tangible, like a quick overview of the software or a series of tutorial videos and exercises to display the full potential of the interface. Others, equally important, are less tangible, like the overall interface design and interactive feedback to help users understand what they can do and how to do it (Fig. 10).

<figure><img src="image_12.png" /><figcaption>Fig. 10. Home page of TYPANNOT Keyboard: the interface design guides the user on what can be done and how it should be done.</figcaption></figure>

### 2.4.2 Opening possibilities for present and future research

To be efficient, the design work of the TYPANNOT Keyboard has to take into account the TYPANNOT typographic system and different user profiles. Our opinion is that the latter is the most important aspect. Understanding the user's thought process and motivation is essential. And this alone is a substantial task. The TYPANNOT Keyboard is not intended to shape the direction of researchers' work. Our goal is to offer a flexible tool that enhances research capabilities, while retaining as much information as possible to broaden our understanding of SL. To this end, the TYPANNOT Keyboard
and its interfaces are being developed to fit all research methodologies, all types of focus and specializations, and of course, all SLs and gestural actions.

From the beginning, the TYPANNOT Keyboard has been designed to be a virtual keyboard that can be used on top of any given software. Two different interfaces were developed, each offering its own transcription experience. These interfaces have three main features in common: an interactive 3-dimensional representation of the parameter, the corresponding glyph in TYPANNOT Font, and an input device.

The Parametric Interface (Fig. 11) has TYPANNOT's 4 layers of information (parameter, part, variable, value) as the input device. Glyphs are composed by selecting and adding values. It is a very simple way to compose glyphs that ensure a perfect comprehension of the typographic system.

<figure><img src="image_13.png" /><figcaption>Fig. 11. Parametric interface of the TYPANNOT Keyboard for HS coding</figcaption></figure>

The Gestural Interface (Fig. 12) uses motion capture devices (Leap Motion, Neuron Perception, Brekel Pro Face 2) to offer an effortless transcription process. This means that the annotator's own body is used to transcribe, directly reproducing the handshape, body position or facial action. This offers a very intuitive input system that truly connects with the nature of SL.

<figure><img src="image_14.png" /><figcaption>Fig. 12. Gestural interface of the TYPANNOT Keyboard for HS coding</figcaption></figure>

### 2.4.3 Research sourced typographic library

Beyond a learning tool and an input device, the TYPANNOT Keyboard is also the answer to the technological limitations of OpenType fonts. In its initial version, the keyboard will be loaded with 990 glyphs corresponding to our fundamental set. But 990 is not exhaustive, and researchers will inevitably need more glyphs. When users compose a glyph needed for their own research and not yet included in the glyphic library, the TYPANNOT Keyboard will offer them the opportunity to request the addition of the new glyph. On a regular basis, the TYPANNOT library will be updated, including all requested glyphs. With this open sourced process, TYPANNOT will be the research-based font that includes all glyphs from all research around the globe.
# 3 The kinesiological approach

## 3.1 The body at the center of linguistic analysis

It is further important to recognize the role of movement in the expression of meaning in both SL and gesture, and therefore, to recognize the importance of movement in the development of our research. While phonological studies endeavor to faithfully represent the other manual and non-manual components in order to analyze them, any attempt to fix movement seems to go against the very essence of its ephemeral nature. The strong physical anchoring of movements generates a great complexity of representation; this creates analytical difficulties, which contribute to the marginalization of movement research relating to research about SLs and gestures in all types of communication. In turn, this results in a poor understanding of the nature and meaning of movement, all of which ends up reinforcing the complexity of movement representation: it is a vicious circle<fnref n="11" />.

Besides being marginalized, the study of movement that does exist is almost exclusively focused on the activities of the hand, whose capacity to produce meaning is never in doubt. But while researchers are increasingly interested in the participation of the face and the trunk in the production of meaning, the arm and forearm remain confined to the role of simple connecting segments.

Breaking the ruts created by these common trends - i.e., the focus on the hand and the marginalization of the movement - requires a radical change of approach, which is precisely what Dominique Boutet<fnref n="12" /> (DB) proposes through the kinesiological approach developed in his own research (Boutet, 2018). Taking into account the physical and physiological mechanisms governing the anatomical constraints of human movements, DB seeks both to restore the body capacities as a vehicle of meaning (see §3.2) and to show that it is possible to describe (and analyze) the movement faithfully and efficiently. For this, it is necessary to make innovative choices: include all of the upper limbs, change the frame of reference, and abandon Euclidean geometry.

### 3.1.1 Considering the upper limb

The hand is often considered as the only articulator carrying movement and, consequently, meaning (see Intro and §1). However, this is not an entity independent of the rest of the body. It is attached to the forearm, which is attached to the arm, itself linked to the trunk. The hand is therefore the most distal<fnref n="13" /> end of a chain of segments (SEG) comprising the forearm as well as the arm, and this concatenation necessarily generates a series of physiological constraints and limitations on the freedom of movement of each of these SEGs.

In the approach proposed by DB, movement is carried by all the SEGs of the upper limb, considered as an articulatory system. The movements and postures of each of these SEGs are described according to principles governed by biomechanics (Kapandji, 1997). Each SEG is associated with so-called degrees of freedom (DOF), which correspond to the rotation of a SEG around an axis located at the level of the proximal adjacent SEG. Thus, the hand will be described in relation to its position relative to the forearm; the latter will be linked to the arm, which in turn will be described in relation to the trunk. These axes mainly pass through the joints (wrist, elbow, shoulder), but they can also cross bones longitudinally (ulna + radius, humerus).

The upper limb is therefore an “infrastructure which underlies all the possible movements” (Boutet 2010:2) of the hand, and constitutes an articulated whole with inseparable parts, all having a precise role in the unfolding of the sign.

<footnote n="11">¹¹ Getting out of this loop is a very topical issue, practical but also theoretical. Indeed, the deepening of the analysis of movement is perceived as a possible response to the debate which animates research in SL on the distinction between co-verbal gestures and purely linguistic phenomena (Schembri et al. 2015; Goldin-Meadow and Brentari 2017).</footnote>

<footnote n="12">¹² Dominique Boutet was coordinator of the GestualScript team from its beginnings in 2008 until 2020, when he succumbed to the COVID-19 pandemic. Parallel to his commitment to the representation of SLs, he developed the kinesiological approach to human gestures, which has greatly influenced the ongoing work carried out by the GestualScript team.</footnote>

<footnote n="13">¹³ Distal and proximal are concepts indicating the position of a Seg relative to another Seg and to the body: a Seg is distal if it is located further from the body in relation to another Seg (the hand is distal in relation to the forearm which is distal to the arm); a Seg is proximal when closer to the body than another Seg (the arm is proximal to the forearm which is proximal to the hand).</footnote>
3.1.2 Changing the frame of reference (FoR)

An articulatory approach has a profound effect on how movement is inscribed in the representation space. Traditionally, the description of movement is done in a space defined by Cartesian planes: horizontal, vertical and sagittal. The point where these planes intersect defines the frame of reference (FoR), which can be absolute, relative and intrinsic (Levinson, 1996; see Fig. 13). The first is centered on the surrounding space (the choice of a specific location on the space giving rise to several subtypes of relative FORs); the second on the body of the signer (again, different subtypes are possible); the third is centered on an object and is defined on the basis of its inherent characteristics.

<figure><img src="image_15.png" /><figcaption>Fig. 13. Absolute, relative or intrinsic frame of reference (Levinson, 1996).
Depending on the adopted FoR center O,
there are different descriptions of the position of A (cork) relative to B (bottle)</figcaption></figure>

The analysis of the SL representation systems (see §1) shows that they adopt FORs which can be
relative or absolute (never intrinsic), but also that within the same system, different FORs can be
adopted, sometimes on a case-by-case basis.<fnref n="14" /> In these events, these descriptive instabilities give rise
to fragmentations of the description space even if looking just at the movement, thus generating the risk
of inconsistencies in the analysis of signs (see above §1).

Extending the analysis to all the SEGs of the upper limb, the kinesiological approach risks being confronted with a multiplication of the difficulties of representation and analysis, unless it adopts a coherent system of registration in a single typology of FOR. The choice has been to abandon the projection of SEGs on planes in favor of a parameterization of the SEGs in their own respective space. This is allowed by the use of intrinsic FORs. The description of each SEG is then centered on an object (i.e., the proximal SEG adjacent to the analyzed SEG) and is defined on the basis of the intrinsic characteristics of this same object (which are in fact equivalent to the DOF of the analyzed SEG). The various DOFS are identified by the name of their poles (or joint stops): abduction (ABD) on one side and adduction (ADD) on the other, flexion (FLX) and extension (EXT), pronation (Pro) and supination (SUP), internal rotation (RIN) and external rotation (REX).

More concretely (see Fig. 14), the hand - whose FOR is defined in relation to the forearm - is affected by 2 DOF whose poles are FLX-EXT and ABD-ADD, both passing through the wrist joint; a 3rd DOF is present, PRO-SUP, which goes through the ulna and the radius (bone that can cross). This latter DOF could be considered to affect the forearm, but since the “result” of this movement is visible on the hand, it was decided to include it in the description of the hand. The forearm - more proximal than the hand and more distal than the arm - is affected by 2 DOF, i.e. FLX-EXT which passes through the elbow, and RIN-REX which is due to the possibility of the head of the humerus to rotate in the scapula. Here too, although RIN-REX is located on the upper arm, it is assigned to the forearm because its result is visible there. Finally, the arm - the most proximal of the SEGs and which is described in relation to the trunk - is affected by 2 DOF, i.e. FLX-EXT and ABD-ADD, both passing through the shoulder joint.

<footnote n="14">¹⁴ In her thesis, C.S. Bianchini (2012) offers a detailed analysis of all the FoRs present in SW, showing that the FoR can vary within the same parameter.</footnote>
<figure><img src="image_16.png" /><figcaption>Fig. 14. Listing of the degrees of freedom (DOF) of the segments (SEG) of the upper limb</figcaption></figure>

The FOR used by the kinesiological approach is therefore unique, because in fact it is an intrinsic FOR, but it is combinatorial too, since it takes into account the fact that in an articulatory system each SEG depends on its proximal SEG. This innovative choice has also the advantage of being ready for the envisaged technological requirements: motion capture systems (MoCap) are gaining in importance in gestural studies and some of these technologies are based on intrinsic FORs, but the classical representation systems used to analyze them are still based on relative or absolute FORs, thus requiring a conversion. The kinesiological approach allows direct access to data, minimizing biases related to said FOR conversion, thus facilitating not only the understanding of the movement but also its representation: this should make it possible to break the vicious circle discussed in §3.1.

The change of FOR makes it possible to focus on the possibilities and limits of the movements specific to each of the SEGs, but it also generates other modifications: bypassing Cartesian planes for the description of SEGs requires finding a geometry that can take into account elements which no longer fit into these plans.

### 3.1.3 The transition to a non-Euclidean geometry

In 1934, Ernest A. Codman affirms that if the arm is completely raised, it is both in complete REX and complete RIN (Codman, 1934). Neither its author nor the specialists in movement and physiology who looked into it (Pearl et al., 1992) succeeded in explaining this fact. It is ultimately the abandonment of Euclidean geometry in favor of a non-Euclidean geometry which allows understanding the existence of diadocal movements<fnref n="15" /> (Mac Connail, 1953) and therefore resolving this alleged paradox.

Euclidean geometry draws its forms on planes and is based on 5 postulates, the last of which states (simplifying) that “given a straight-line *d* and a point *P* located outside it, there is one and only one straight line *d'* passing through *P* and parallel to *d'* (Fig. 15). The non-Euclidean geometry proposed by Gauss, a.k.a. spherical geometry, rejects this postulate, asserting that “there exists an infinity of lines passing through *P* which are parallel to *d'* (Fig. 16). This is possible if, and only if, we abandon the

<footnote n="15">¹⁵ The Codman's Paradox is the result of a “diadocal movement”, i.e. an involuntary movement which, on a SEG with 3 DOF (like the hand or the arm, if considering also the hidden DOF carried by the humerus), affects a DOF when the other two moves consecutively.</footnote>
representation on the plane in favor of a representation on the sphere. A line (spherical) will be a circle drawn at the “equator” of a sphere; a point will be a pole where several spherical lines intersect. The consequence is such that - unlike the principles stated by Euclidean geometry - in spherical geometry, the curve (or spherical line) is a very simple plot and the plane straight line is a complex figure.

<figure><img src="image_17.png" /><figcaption>Fig. 15. The postulates of Euclidean geometry</figcaption></figure>

<figure><img src="image_18.png" /><figcaption>Fig. 16. In spherical geometry, Euclid's 5th postulate is not respected</figcaption></figure>

Coming back to the description of the body, the most proximal end of a SEG (e.g., the elbow for the forearm) constitutes the center of a (portion of) sphere; the movement of the different DOFs draw at the most distal end of the SEG (e.g., the wrist for the forearm) “straight” spherical lines which intersect at the poles of this sphere. The use of the spherical geometry, associated with the multiple intrinsic FORs, thus allows to describe in a simple way what is anatomically simple - that is to say the production of curves - and in a complex way what is complex for the body, i.e. the straight lines (Fig. 17). This therefore contributes to the creation of a faithful and efficient description of the movement.
<figure><img src="image_19.png" /><figcaption>Fig. 17. Segment(s) performing a simple (1 DoF) or complex (several DoFs) anatomical gesture.</figcaption></figure>

## 3.2 The body as a generator of meaning

In the classical approach to SL analysis, the hand is seen as the articulator which, replacing the mouth, conveys meaning. The kinesiological approach, with its consideration of the entire upper limb in a non-Euclidean geometry and with intrinsic and multiple FORs, questions the validity of this idea.

For example, the co-verbal gesture “no” can be done, at least in France, by standing with the arm alongside the body, the forearm slightly bent, and the hand extended; it is then possible to perform repeated movements of ABD and ADD of the hand (a “little no”). But to support the disagreement, it will then not only be the hand that will be in motion, but also the forearm, and why not, the arm (a “big no”). Visually, these two realizations are very different, but nobody doubts that they convey the same meaning “no”: how is that possible?

To answer this question, the kinesiological approach proposes to search for the structural invariants of the articulatory dynamics which underlie the creation of signs and which are hidden by purely visual differences. Once again, innovative choices and new concepts become necessary: a) proposing a new notion of movement and temporality; b) restructuring the classic parameters of SL analysis.

### 3.2.1 Movements and temporality

The search for invariants begins with understanding the different typologies of movement. DB suggests distinguishing proper movement from displacements and transfers. It is a question of proper movement when a SEG initiates movement, that is, at least one of its DOFs performs a rotation. If this SEG has SEGs more distal than itself, these - driven by the proper movement - displace in space, without even their DOF moving. Finally, in special cases, there may be an inertial transfer of movement: the rotation of a DOF then engages the variation of a DOF on a different SEG.

If a movement can propagate between different SEGs, then it is possible to study its flow (Boutet, 2018), that is to say the order in which the different SEGs are set in motion. If a movement begins on the hand and then continues on the forearm, this is a distal-proximal flow; if the reverse is true, then it is a proximal-distal flow; the flow can also be “neutral” if all the concerned SEGs move at the same time, or even “absent”.

To come back to the “big no” and “small no”, they are identified as manifestations of the same sign because they correspond to the same pattern: the hand initiates a repeated movement on the DOF ABD-ADD, which is propagated by following a distal-proximal flow. This reality, however, remains hidden under the superficial manifestation of these two demonstrations. A restructuring of the classic manual parameters (see Intro) is therefore necessary.
### 3.2.2 Restructuring of location and orientation

DB argues (Boutet, 2018) that the search for invariants can be facilitated by restructuring the classic parameters of orientation, location and movement, proposing to replace them by parameters whose names may seem similar, but whose scope will be radically different: the initial location (LOCINI) and the movement (MOV).

The LOCINI makes it possible to fix the position of all the SEGS before the deployment of the sign. Therefore, it brings together the notions of orientation and location, but by extending them to the whole of the upper limb. Concretely, the LOCINI is described through the angles of rotation (in an intrinsic FOR) of all the DOFs of the SEGS (only 7 in total).

The kinesiological approach then makes the hypothesis (Chevrefils, forthcoming) that once the body is installed in a posture, the resulting MOV is simple: the body's tendency to decrease the DOFs to be controlled pushes the SEGS to coordinate (Survey, 1990) and to prefer a distal-proximal flow,<fnref n="16" /> which leads to economy and predictability of movements. The results of a first study involving a few minutes of corpus in three SLs (English, French and Italian) seem promising (Danet et al., 2017). A deepening of this hypothesis, through an accurate analysis of the kinematic data from a MoCap system, is underway.

Therefore, the subdivision between LOCINI and MOV also contributes to understanding the difference between the small and the big “no”: the two are identified as distinct realizations of the same sign because their MOV is the same, despite a difference in the LOCINI and in the MOV amplitude. A difference in the SEG initiating the gesture, in the DOF concerned or in the flow would not have allowed these signs to be identified as “no”. The kinesiological approach thus renders to the whole body (and no longer just to the hand) its function of generator of meaning.

## 3.3 From theory to practice: from movement to TYPANNOT

Although offering a reliable and economical description of the movement, the kinesiological approach requires the handling of many concepts. This novelty could generate the impression that this description system can only be used after following a specific theoretical training, in particular concerning the use of intrinsic FOR which is “experienced” by any speaker, but which is not “recognized” and “perceived” by most of them.

The work of the GestualScript team addresses this point. Its goal is to make all these notions accessible and functional through the creation of TYPANNOT, a transcription system based on typographic principles. TYPANNOT is not only a graphic formalization of a theory, it is the instrument for appropriating the theory itself, drawing its bases from the kinesiological model and its descriptive efficiency, it will allows the constitution of a readable, writable, and searchable corpus (of SL or of co-verbal gestures) readable, scriptable and searchable according to the desired level of granularity.

The passage from the complexity of a theoretical approach to the intuitiveness of a “turnkey” typographic system requires answering several preliminary questions, including a non-exhaustive list of which is: how can such complex phonological descriptions be readable and scriptable? how to increase the descriptive precision of the system without increasing the transcription time, or even reducing it? how to make the transcriptor conscious of their own body so that the notion of intrinsic FOR is understandable?

The answers to these questions go through the definition of different layers of information and construction principles, set out above.

## DISCUSSION

<footnote n="16">¹⁶ Despite the existence of a decreasing inertial slope from the arm to the hand (Dumas et al., 2007) favoring a proximal-distal flow, the communicational aim of movement in SL would reverse this trend: the flow of movement would then be predominantly distal-proximal (Chevrefils, forthcoming). The preliminary study of a corpus of three SLs seems to confirm this trend (Danet et al., 2017), the causes of which are still under investigation.</footnote>
TYPANNOT is a typographic writing system intended for linguists needing to study and transcribe SLs. It follows a musculoskeletal articulatory approach that changes the conventional perspective from which gestures are observed. This perspective allows researchers to investigate how corporal activity fully determines the construction, modulation, and transformation of meaning of the signs in SLs.

Further, the phonological articulatory approach of TYPANNOT, in which the joints of the skeleton are represented in order to distinguish the abstract phonetic units that correspond to meaning in SLs and movement, it possible to transcribe gestures using the same corporal parameters. This way, TYPANNOT is also a tool for investigating other forms of corporal expression, such as co-verbal gesturality, which refer to gestures made while talking.

Also, because our writing system describes the morphological components of SLs at the articulatory level, TYPANNOT indexes SL using elementary characters. This makes it possible to refer to SL signs using simple morphological and gestural features rather than translating them into a vocal language, as is systematically the case with online SL dictionaries.

Another aspect of the TYPANNOT system is its ability to transcribe the dynamics of the articulatory system. Such transcription possibilities can be coupled with motion capture technologies to explore new ways of inputting, recognizing, and reproducing SL signs. When a stream of recorded gestures can be directly recognized to automatically generate a transcription (input), this transcription can also generate the 3D animation of a signing avatar (output).

Finally, although not the main goal of this project, TYPANNOT is fitting to the contribution of the development of a writing system for SLs by giving signers a new form of expression that is based on the human body itself, the very center and origin of all SL expression.
**Bibliography**

Augustus, Robert A., Elsie Ritchie and Suzanne Stecker (2013). "The Official American Sign Language Writing Textbook". si5s.org & ASLized.org.

Bébian, Auguste (1825). "Mimographie ou Essai d'Écriture Mimique, Propre à Régulariser le Langage des Sourds-Muets". Paris: Louis Colas.

Bianchini, Claudia S. (2012). "Analyse Métalinguistique de l'Émergence d'un Système d'Écriture des Langues des Signes : SignWriting et son Application à la Langue des Signes Italienne (LIS)". Thèse de Doctorat : Sciences du langage. Université de Paris 8 Vincennes & Università degli Studi di Perugia.

Bianchini, Claudia S. (2016). "Regard sur la Nature de SignWriting (SW), un Système pour Représenter les Langues des Signes (LS)". In: *Dossiers d'HEL* 9, pp. 404-421.

Bianchini, Claudia S., Fabrizio Borgia and Maria De Marsico (2012). "SWift - A SignWriting Editor to Bridge Between Deaf World and e-Learning". In: *Proceedings of the International Conference on Advanced Learning Technologies (ICALT), IEEE - Institute of Electrical and Electronics Engineers*, pp. 526-530. doi: 10.1109/ICALT.2012.235.

Boutet, Dominique (2010). "Structuration Physiologique de la Gestuelle : Modèle et Tests". In: *Revue de Linguistique et de Didactique des Langue* 42, pp. 77-96.

Boutet, Dominique (2018). "Pour une Approche Kinésiologique de la Gestualité : Synthèse". Habilitation à Diriger des Recherches, Université de Rouen Normandie.

Boutet, Dominique, et al. (2018). "Systèmes Graphématiques et Écritures des Langues Signées". In: *Signatures : (Essais en) Sémiotiques de l'Écriture*. Ed. by J.-M. Klinkenberg and S. Polis. *Signata* 9, pp. 391-426.

Braffort, Annelies (1996). "Reconnaissance et Compréhension de Gestes, Application à la Langue des Signes". Thèse de Doctorat. Université de Paris 11 Orsay.

Brugeille, Jean-Louis (2007). "L.S.F. Numérique", *Journée Professionnalisante sur la Traduction - Traduction et Métiers Émergents : Traducteur en Langue des Signes*. Université de Toulouse Le Mirail.

Chevrefils, Léa (forthcoming). "Vers une Modélisation des Constituants Gestuels des Signes : Capture de Mouvement et Transcription Formelle d'un Corpus de Langue des Signes Française". Thèse de Doctorat: Sciences du langage. Université de Rouen Normandie.

Codman, Ernest (1934). "The Shoulder: Rupture of the Supraspinatus Tendon and other Lesions in or about the Subacromial Bursa". Boston: R.E. Kreiger.

Cuxac, Christian (2001). "Les langues des Signes : Analyseurs de la Faculté de Langage". In: *AILE - Acquisition et Interaction en Langue Étrangère* 15, pp. 11-36.

Danet, Claire et al. (2017). "Structural Correlation Between Location and Movement of Signs: Lacking Motion Economy for Co-Speech Gestures". In: *Language as a Form of Action*. Rome: CNR.

Dumas, Raphaël, Laurence Chèze and Jean-Pierre Verriest (2007). "Adjustments to McConville et al. and Young et al. Body Segment Inertial Parameters". In: *Journal of Biomechanics* 40(3), pp. 543-553.

Eccarius, Petra and Diane Brentari (2008). "Handshape Coding Made Easier: a Theoretically Based Notation for Phonological Transcription". In: *Sign Language & Linguistics* 11(1), pp. 69-101.
Fusellier Souza, Ivani (2001). "La Création Gestuelle des Individus Sourds Isolés : de l'Édification Conceptuelle et Linguistique à la Sémiogenèse des Langues des Signes". In: *AILE - Acquisition et Interaction en Langue Étrangère* 15, pp. 61-96.

Goldin-Meadow, Susan and Heidi Feldman (1975). "The Creation of a Communication System: a Study of Deaf Children of Hearing Parents". In: *Sign Language Studies* 8, pp. 225-234.

Goldin-Meadow, Susan and Carolyn Mylander (1983). "Gestural Communication in Deaf Children: the Non-Effects of Parental Input on Language Development". in: *Science* 221, pp. 372-374.

Goldin-Meadow, Susan and Carolyn Mylander (1984). "Gestural Communication in Deaf Children: the Effects and Non-Effects of Parental Input on Early Language Development". In: *Monographs of the Society for Research in Child Development* 49(3-4), pp. 1-121.

Goldin-Meadow, Susan (1998). "The Development of Gesture and Speech as an Integrated System". In: *The nature and functions of gesture in children's communications*. Ed. by J.M. Iverson and S. Goldin-Meadow. *New Directions for Child Development* 79, pp. 29-42.

Goldin-Meadow, Susan and Diane Brentari (2017). "Gesture, Sign, and Language: the Coming of Age of Sign Language and Gesture Studies". in: *Behavioral and brain sciences*, pp. 1-60. Cambridge: University Press.

Guitteny, Pierre (2007). "Langue des Signes et Schémas". In: *TAL -Traitement Automatique des Langues*, 48(3), pp. 1-25.

Hanke, Thomas (2004). "HamNoSys: Representing Sign Language Data in Language Resources and Language Processing Contexts". In: *Workshop on the Representation and Processing of Sign Languages on the Occasion of the Fourth International Conference on Language Resources and Evaluation (LREC Lisbonne)*, pp. 1-6.

Haouam-Bourgeaois, Nadia (2007). "La Signographie: Master n°1". Les Essart-le-Roi: Éditions du Fox.

Jouison, Paul (1995). "Écrits sur la Langue des Signes Française (LSF)" (édition critique établie par Brigitte Garcia). Paris: L'Harmattan.

Levinson, Stephen C. (1996). "Frames of Reference and Molyneux's Question: Crosslinguistic Evidence". In: *Language, Speech, and Communication: Language and Space*, pp. 109-169. Ed. by P. Bloom, M.A. Peterson, L. Nadel and M.F. Garrett. Cambridge MS: The MIT Press.

MacConaill, Michael A. (1953). "Movements of Bone and Joints: 5 – the Significance of Shape". In: *The Journal of Bone and Joint Surgery* 35B(2), pp. 290-297. doi: 10.1302/0301-620X.35B2.290

Pearl, Michael L. et al. (1992). "Codman's Paradox: Sixty Years Later". In: *Journal of Shoulder & Elbow Surgery* 1(4), pp. 219-225. doi: 10.1016/1058-2746(92)90017-W

Poizat, Germain, Deli Salini and Marc Durand (2013). "Approche Énactive de l'Activité Humaine, Simplexité et Conception de Formations Professionnelles". In: *Éducation, Sciences & Society* 4(1), pp. 97-112.

Prillwitz, Siegmund et al. (1989). "Hamburg Notation System for Sign Languages: an Introductory Guide". Hamburg: Signum Press.

Sandler, Wendy and Diane Lillo-Martin (2006). "Sign Language and Linguistic Universals". Cambridge: Cambridge University Press.
Schembri, Adam, Caroline Jones and Denis Burnham (2005). "Comparing Action Gestures and Classifier Verbs of Motion: Evidence from Australian Sign Language, Taiwan Sign Language, and Nonsigners' Gestures Without Speech". In: *The Journal of Deaf Studies and Deaf Education* 10(3), pp. 272-290.

Slobin, Dan I. et al. (2001). "Sign Language Transcription at the Level of Meaning Components: the Berkeley Transcription System (BTS)". In: *Sign Language & Linguistics* 4(1-2), pp. 63-104.

Stokoe, William (1960). "Sign Language Structure: an Outline of the Visual Communication Systems of the American Deaf". In: *Journal of Deaf Studies and Deaf Education* 10, pp. 3-34.

Sutton, Valerie (1995). "Lessons in SignWriting". La Jolla CA: DAC.

Sutton, Valerie (2020). "SignWriting History". SignWriting Official Website http://www.signwriting.org/library/history/index.html (accessed 9/20)

Theureau, Jacques (2004). "*Le cours d'action : méthode élémentaire*". Toulouse: Octares.

Turvey, Michael T. (1990). Coordination. American Psychologist, 45, 938-953.

Varela, Francisco, Evan Thompson and Eleanor Rosch (1993). "*L'Inscription Corporelle de l'Esprit : Sciences Cognitives et Expérience Humaine*". Paris: Éditions du Seuil.