# Presentation: 244445

## Mapping

- Workshop on Universal Reporting Parameters for the Speech
- of Individuals with Cleft Palate
*Washington DC, April/May 2004*
- Tara Whitehill, Hong Kong

## Proposed Three-Stage Plan

**Evaluating****:**  The usual speech evaluation by the local speech specialist in cleft palate.
**Mapping:**   Conversion from the local speech evaluation to the universal reporting measures.
**Reporting:*****  ***Universal parameters that describe the cardinal features of speech in individuals with cleft lip/palate and related craniofacial anomalies regardless of the individual’s primary language.

## Already covered

- Evaluation (range of protocols and procedures; key issues)
- Parameters (proposal; key issues)
- Focus: mapping

## Fanny LAW Sze Kuen
International Parameters for Cleft Palate Speech: A Pilot Study

Honours dissertation
Bachelor of Science (Speech & Hearing Science)
University of Hong Kong 
May 2004
Supervisor: Dr. T. L. Whitehill

DRAFT - in progress

*International Parameters for Cleft Palate Speech: A Pilot Study*
- Honours dissertation
- Bachelor of Science (Speech & Hearing Science)
- University of Hong Kong
- May 2004
- Supervisor: Dr. T. L. Whitehill
*DRAFT - in progress*

## 6 sets of client forms from 5 centres 
5 clients each set = 30 client forms

Original forms only (no recordings, no additional data such as transcriptions, unless provided on original form)

Two groups of “mappers”
5 “experts” - members of WG who supplied forms
5 students - little/no previous experience cleft

- 5 clients each set = 30 client forms
- Original forms only (no recordings, no additional data such as transcriptions, unless provided on original form)
- Two groups of “mappers”
- 5 “experts” - members of WG who supplied forms
- 5 students - little/no previous experience cleft

## Used “current” version of parameters and guidelines 
Students given session to familiarize/
   clarify terminology
Mapped from client forms onto parameters form
Identifying information masked (client, examiner,          centre)
Provided feedback on ease/difficulty of mapping each parameter
Either for client form or by set (centre)
Weighted scores - not included in this study

- Students given session to familiarize/
-    clarify terminology
- Mapped from client forms onto parameters form
- Identifying information masked (client, examiner,          centre)
- Provided feedback on ease/difficulty of mapping each parameter
- Either for client form or by set (centre)
- Weighted scores - not included in this study

## “Outcome measures”

- Agreement
- Within expert group
- Within student group
- Across all mappers
- Feedback (ease/difficulty)

## Why use students?

- In future, unlikely mappers will be naïve
- However, “expert” group was restricted to individuals who had developed and debated the parameters

## Agreement

- A big problem was “missing data”
- OK if client form stated MD (or WNL or NAD), but what if didn’t? - assume OK? tick MD?
- Calculated
- 	(a) absolute agreement (Condition 1)
- 	(b) disregarding “missing data” 							(Condition 2)

## Table 1. Examples to illustrate how agreement was defined

## Primary ParametersHypernasality

- 				Agreement
- Experts     Condition 1 - 60%
-                      Condition 2 - 63%
- Students   Condition 1 - 73%
-                      Condition 2 - 73%
- All		  Condition 1 - 57%
-                      Condition 2 - 60%
- Not much missing data

## Primary ParametersHypernasality

- Feedback/Issues to consider
- Some difficulties when no. scale points differed
-      (e.g., client form - 7 pt. scale; parameters form - 4 pt. scale)
- In such cases, mappers reported difficulty, and agreement decreased
- Implication: explicit instructions?
- Another difficulty: “mild to moderate” or “moderate to severe” (students selected less severe and experts more severe!)
- “Set B” - definition of “mild” = “evident but acceptable”. One expert rater mapped onto WNL.

## Primary ParametersHyponasality

- 				Agreement
- Experts     Condition 1 -  53%
-                      Condition 2 - 96%
- Students   Condition 1 -  87%
-                      Condition 2 - 100%
- All		  Condition 1 -  50%
-                      Condition 2 - 97%
- Excellent, when missing data is accounted for

## Primary ParametersHyponasality

- Feedback/Issues to consider
- Missing data
- Otherwise, excellent
- (binary scale)

## Primary ParametersAudible Nasal Emissionwith/without Nasal Turbulence

- Level One (absent, present, MD)
- Experts   Condition 1 -  63%
-                      Condition 2 - 87%
- Students  Condition 1 -  77%
-                      Condition 2 - 80%
- All	      Condition 1 -  57%
-                      Condition 2 - 70%

- Level Two (subcategories)
- Experts   Condition 1 -  23%
-                      Condition 2 - 60%
- Students  Condition 1 -  30%
-                      Condition 2 - 38%
- All	      Condition 1 -  17%
-                      Condition 2 - 27%

## Primary Parameters Audible Nasal Emissionwith/without Nasal Turbulence

- Feedback/Issues to consider
- Low agreement for subcategories
- “phoneme specific” - e.g. “audible for plosive and /s/”
- Confusion about pervasiveness e.g. “mild but consistent”, “phoneme specific and frequent”
- Results of mirror test (if not audible in speech)
- (NE w/wout nasal turbulence - not raised here)

## Primary ParametersWeak Oral Pressures

- 				Agreement
- Experts     	Condition 1 -  17%
-                      	Condition 2 - 75%
- Students   	Condition 1 -  13%
-                      	Condition 2 - 62%
- All		Condition 1 -  0%
-                      	Condition 2 - 46%

## Primary Parameters Weak Oral Pressures

- Feedback/Issues to consider
- All 9 mappers found this parameter difficult to map
- Primarily because of lack of data on client forms - no form had specific category for this
- Mappers needed to derive the information from other information; particularly difficult for students
- Also, students unclear about definition

## Primary Parameters Substitution Errors

- Level One (absent, present, MD)
- Experts   Condition 1 -  57%
-                      Condition 2 - 73%
- Students  Condition 1 -  73%
-                      Condition 2 - 85%
- All	      Condition 1 -  57%
-                      Condition 2 - 70%

- Level Two (subcategories)
- Experts   Condition 1 -  10%
-                      Condition 2 - 30%
- Students  Condition 1 -  10%
-                      Condition 2 - 15%
- All	      Condition 1 -  10%
-                      Condition 2 - 20%

- Agreement not bad for level one, but very poor for subcategories; missing data was not the main problem

## Primary Parameters Substitution Errors

- Feedback/Issues to consider
- All forms included level one (present/absent/MD)
- Subcategories: most forms had patterns, but few included frequency information
- Some patterns which expert listeners considered ‘related to cleft palate’ not included here (e.g. nasal substitution/realization’, ‘double articulation’)
- Confusion between this parameter and ‘Developmental Delay or Other Articulation/Phonological Errors’ (even for experts) e.g. omission, backing /t/ -> [k].

## Secondary ParametersErrors related to dentition, occlusion, palatal vault configuration

- 				Agreement
- Experts     	Condition 1 - 13%
-                      	Condition 2 - 69%
- Students   	Condition 1 - 30%
-                      	Condition 2 - 59%
- All		Condition 1 -  7%
-                      	Condition 2 - 53%
- Better when missing data taken into consideration, but still pretty poor

## Secondary ParametersErrors related to dentition, occlusion, palatal vault configuration

- Feedback/Issues to consider
- No specific category for this on client forms; information had to be extracted from other information
- Some forms (Sets B, E, F) included categories such as ‘lateralization’, ‘palatalization’, ‘dentalization’ - but still not clear whether these due to organic factors or “Other Articulation/Phonological Errors” - even for experts
- Errors related to hearing problem? fistula?

## Secondary ParametersDevelopmental Delay or Other Articulation/Phonological Errors

- 				Agreement
- Experts     	Condition 1 - 16%
-                      	Condition 2 - 72%
- Students   	Condition 1 - 30%
-                      	Condition 2 - 50%
- All		Condition 1 - 10%
-                      	Condition 2 - 43%
- Again, better when missing data taken into account, but still pretty poor

## Secondary Parameters Developmental Delay or Other Articulation/Phonological Errors

- Feedback/Issues to consider
- Mappers needed to make judgement regarding whether errors were related to cleft or not
- Confusion with both “Substitution Errors” and “Errors related to dentition..”
- Client forms rarely specified cause or explanation of error patterns - mappers needed to judge
- Special cases e.g. “no oral consonants”

## Secondary ParametersVoice/Laryngeal Disorder

- 				Agreement
- Experts     	Condition 1 - 56%
-                      	Condition 2 - 90%
- Students   	Condition 1 - 63%
-                      	Condition 2 - 97%
- All		Condition 1 - 60%
-                      	Condition 2 - 96%
- Agreement was good, especially when missing data was taken into account

## Secondary Parameters Voice/Laryngeal Disorder

- Feedback/Issues to consider
- Similar to hyponasality - mappers were familiar with term
- When data was provided on client form, it was clear
- Binary choice; no subcategories

## Global RatingSeverity/Intelligibility

- Not decided at time of this study
- Only 2 of the 6 sets included such data

## Conclusions

- Reliability lower among students than experts
- Experts were those who developed form
- Who will be mapping in the future?

## Slide 29