01whole.pdf (2.68 MB)
Download file

Developing and validating in-house, standardised, English reading and listening placement/streaming and achievement tests aligned to the Common European Framework of Reference Levels A2–B1 at a Japanese university

Download (2.68 MB)
thesis
posted on 28.03.2022, 18:23 authored by Jack Victor Bower
The research project presented in this thesis investigates one aspect of redesigning a compulsory English as a foreign language curriculum at a Japanese university to match intended target CEFR proficiency levels. Specifically, this thesis focuses on the creation and validation of institutional standardised tests of English reading and listening proficiency intended to be aligned to levels A2 and B1 of the CEFR. Kane’s argument-based approach to test validation, which firstly elaborates an Interpretation/Use Argument (IUA), and secondly uses a validity argument to evaluate evidence gathered to support the IUA, is used as a framework for the validation of the tests in question, known as the Bunkyo English Tests (BETs). Inferences in the IUA are also drawn from Chapelle, Enright, and Jamieson (2008), and the warrant for the final inference is drawn from Bachman and Palmer (2010). To substantiate the validity argument, data were collected from a variety of sources, including test results and test specifications, course outlines, surveys administered to teachers and students, interviews with teachers and university senior administrators, student course grades and assessment results, and student results from two other standardised tests, the Oxford Online Placement Test® and the Test of English for International Communication (TOEIC®). Results indicate that the BETs seem to have functioned sufficiently as course placement tests, but not as class streaming tests to divide classes within courses. Further analysis shows that the tests did not function effectively as achievement tests aligned to CEFR levels A2 and B1. This study demonstrates that Kane’s approach to test validation is viable for small-scale, in-house testing programs in the development phase, as it facilitated the selection of validity evidence, the analysis of which exposed areas of weakness in the tests and the test specifications, indicating clear avenues for improvement. Results also point to the need for further validation research on in-house tests which aim for CEFR alignment.

History

Table of Contents

1. Introduction to the study -- 2. Literature review : an overview of validity and validation in language testing -- 3. Literature review : empirical investigations relevant to the current study -- 4. The Bunkyo English test interpretation/use argument -- 5. Materials and methods -- 6. Results -- 7. Summary, conclusions and reflections -- References -- Appendices.

Notes

Bibliography: pages 269-293 Empirical thesis. "Research carried out at Hiroshima Bunkyo Women’s University in Hiroshima, Japan" -- title page.

Awarding Institution

Macquarie University

Degree Type

Thesis PhD

Degree

PhD, Macquarie University, Faculty of Human Sciences, Department of Linguistics

Department, Centre or School

Department of Linguistics

Year of Award

2018

Principal Supervisor

Mehdi Riazi

Additional Supervisor 1

Rie Koizumi

Rights

Copyright Jack Victor Bower 2018. Copyright disclaimer: http://mq.edu.au/library/copyright

Language

English

Extent

1 online resource (xii, 348 pages) graphs, tables

Former Identifiers

mq:70692 http://hdl.handle.net/1959.14/1266788