Assessing simultaneous interpreting :a study on test reliability and examiners' assessment behaviour

Abstract

A substantial amount of research work has been done on the quality assurance of conference interpreting, yielding useful guidelines for the selection and training of interpreters. However, the field of assessment in interpreter training within the educational context is still under-researched. Many interpreter trainers and researchers have pointed out some urgent issues to be addressed in the assessment of interpreting. Among them, the issues surrounding the test validity and reliability are most in need of clarification. This study tackles this complex subject by firstly exploring what the examiners are really paying attention to when assessing student interpreters. Thirty examiners, who are mostly based in Taiwan – some with substantial interpreting experience and some with less, were invited as study subjects to participate in a simulation of simultaneous interpreting examination. This research study adopted a multi-strategy approach in collecting and analysing research data – quantitative and qualitative. Inconsistencies and fuzziness are two themes identified in the study findings in terms of the examiners’ judgements and their use of assessment criteria. The examiners might appear to be using the same assessment criteria, but there were variations in the way how they were used. This study explores, discusses, and clarifies the intricate relations of various components and factors in the interpreting examinations. Based on the study findings, a conceptual model is proposed as a framework for describing the test constructs of the interpreting examinations, and for understanding how the examiners apply the assessment criteria in order to improve the assessment instruments and examination procedures. At the end, implications of the study method are discussed and suggestions are made for future studies in this area.