Consistency in performance in the group oral discussion test: an interactional competence perspective

Language teaching institutions may require a dependable test of conversational ability to assess student development and encourage communication in the curriculum. The group oral test (GOT) is a format in which three or four test-takers discuss a prompt for up to ten minutes that has the potential to do this. Using Interactional Competence (IC) as its foundation and a mixed-methods research (MMR) framework, this dissertation traces 53 Japanese university students’ performance over three administrations in two years to investigate the test’s appropriacy as a tool of assessment for their conversational ability. The quantitative phase of the study measured the participants’ performance using indices of complexity, accuracy, fluency, vocabulary and interactive functions and found varying developmental patterns: most indices showed significant improvement only in the second administration, while others showed gains in just the third administration, and some improved in each successive administration. Overall, gains made in the second administration had larger effect sizes than in the third administration. However, the scores awarded showed that students only improved significantly in the second administration. The qualitative phase of the study adapted Young’s Interactional Competence (IC) framework (2009, 2011) to investigate a subsample of eight test-takers to investigate the extent that students’ performances were represented by the quantitative indices and their scores. The qualitative analysis illuminated the consistent and variable elements in the students’ performances, and revealed the difficulties the raters had of scoring individuals independently of their IC displayed relative to the group’s performance: higher level test-takers could be scored overgenerously for being the best in their group, and lower level students rewarded for being tested with supportive group members. Among the implications for the GOT are that students of similar abilities should be grouped together, rating scales need to reflect the higher order conversational skills identified by this study, and the raters need to be aware of situations in which their interpretation of the rating scales may be compromised.