The development of a new rating scale for the perceptual assessment of tracheoesophageal voice quality outcome following total laryngectomy

Abstract

Perceptual assessment of voice in people with surgical voice restoration (SVR) is essential to evaluate surgical and other interventions aimed at delivering optimal voice quality. Currently there are no tools to measure this that do not have issues of validity and reliability. This work describes the development and trialling of investigatory versions of three scales to address this situation: a) the Sunderland Tracheoesophageal Perceptual Scale (SToPS) for professional raters, b) the Naïve Rater Scale for non-specialist raters and c) the Patient and Carer Scale. In the final testing of the pilot version 55 speakers using tracheoesophageal voice were evaluated by twelve Speech and Language Therapists (SLT’s) and ten Ear, Nose and Throat (ENT) surgeons, divided into experienced or not at assessing voice. Ten naïve raters assessed the voice stimuli within a test-retest design. Forty tracheoesophageal speakers and thirty-seven carers attended an interview to rate their own or their relative’s voice. Inter rater agreement was then calculated between SLT, ENT, naïve, patient and carer groups with weighted kappa co-efficients Strength of agreement values (Landis and Koch 1977) were compared to profession and expertise. Expert SLT’s achieved “good” agreement for nine of fourteen parameters. Naïve judges attained “good” levels of inter and intra-rater agreement for the parameters Overall Grade and Social Acceptability. The greatest inter group consensus was for patients and carers, with “good” agreement for Intelligibility, Volume and Wetness. The only other “good” agreement was between naïve/ENT and naïve/ SLT groups for Overall Grade. The scales are ready for clinical use with the proviso that future work will determine whether it is possible to enhance agreement so less experienced judges can achieve “good” levels of agreement for more parameters and examine which perceptual parameters might be more prominent or vital for outcomes for different groups.