Macquarie University
01whole.pdf (3.48 MB)

Assessing the assessments: using an argument-based validity framework to assess the validity and use of an English placement system in a foreign language context

Download (3.48 MB)
posted on 2022-03-28, 17:42 authored by Robert Charles Johnson
Use of a single, standardised instrument to make high-stakes decisions about testtakers is pervasive in higher education. Contrary to longstanding best practices encouraged by researchers, professional organizations, test publishers, and many accrediting bodies, however, few, if any such institutions have endeavoured to meaningfully validate the instrument(s) they use for their specific context and purposes. The current study attempted to address this void by developing and applying an argument-based validation framework for two widely adopted placement assessment methods - a standardised placement test (Accuplacer), and a locally developed and marked writing sample - utilised by a 2-year higher education institution in the Pacific. A hybrid of two validation structures - Kane's interpretive model and Bachman's assessment use argument - was implemented in order to assure a balanced focus on both test score interpretation and test utilization. Various types and sources of evidence informed the study, including instrument outcomes, student course results, institutional practices and policies, test publisher data, and the opinions of stakeholders gathered via focus group interview and questionnaires. Results are argued to provide insights regarding a number of current issues in the literature, including: i) debates regarding the relative strengths and weaknesses of standardised tests and locally developed and marked writing samples for informing placement decisions; ii) the value of locally conducted validation efforts to evaluate the performance and impact of an institution's chosen assessment instruments, and identifying opportunities for improvement; and iii) the need for further argument-based validation studies, particularly those which attend to both test score interpretation and the longneglected area of test utilization, to be carried out wherever assessments are used to make decisions which impact stakeholders.


Table of Contents

1. Introduction to the study -- 2. Literature review: recent advances in test validity and validation -- 3. Literature review: empirical investigations relevant to the current study -- 4. Materials and methods -- 5. Results -- 6. Summary, discussion and conclusion


Includes bibliographical references "Research carried out at the College of the Marshall Islands. Submitted December 20, 2011 to the Department of Linguistics, Division of Linguistics and Psychology, Macquarie University. This thesis is presented for the degree of Doctor of Philosophy in Applied

Awarding Institution

Macquarie University

Degree Type

Thesis PhD


PhD, Macquarie University, Faculty of Human Sciences, Department of Linguistics

Department, Centre or School

Department of Linguistics

Year of Award


Principal Supervisor

Mehdi Riazi


Copyright disclaimer: Copyright Robert Charles Johnson 2012.




1 online resource (x, 194 pages) illustrations, charts, graphs

Former Identifiers

mq:30434 2134447