Macquarie University
01front.pdf (812.53 kB)

A model for translation accuracy evaluation and measurement: a quantitative approach

Download (812.53 kB)
posted on 2022-03-28, 14:31 authored by Junxiong Huang
Translation quality assessment (TQA) has been part of the translating process since Marcus Tullius Cicero (106-43BCE), and earnest studies on TQA have been conducted for several decades, but there has been no breakthrough in standardized TQA. Though the importance of TQA has been stressed, agreement on specific means of TQA has not been reached. As Chesterman and Wagner summarize, "Central to translation [...]," "[q]uality assessment is so complicated - especially if it is to be objective and reproducible" (2002: 80-81). The approaches to TQA published throughout the past millennia, by and large, are qualitative. "Whereas there is general agreement on the requirement for a translation to be 'good,' 'satisfactory,' or 'acceptable,' the definition of acceptability and of the means of determining it are matters of ongoing debate and there is precious little agreement on specifics" (Williams, 2004: xiv). Most published TQA approaches are neither objective nor reproducible. -- My study proposes a model for fuzzy standardized TQA through a quantitative approach, which expresses TQA results in numerical terms in a consistent manner. My model is statistics-based, practice-based and practice-oriented. It has been independently tested by eleven professors from four countries, fifteen senior United Nations translators, and fifty reader evaluators. My contrastive analysis of 23,000 pages of bilingual and multilingual texts has identified the unit of translation - the orthographic sentence in context, which is also verified by the results of an international survey among 66 professional translators, the majority of whom also confirm that they evaluate translations sentence by sentence in context. Halliday and Matthiessen's functional grammar theory, among others, provides my model for quantitative TQA with its theoretical basis, while the international survey, the necessary data. My model proposes a set of six Fuzzy Functional Translation Grammar terms, a grammar concept general enough to cover all grammar units in the translated orthographic sentence. Each term represents one type of error which contains from one to three sub-categories. Each error is assigned a value - the mean of the professional markers' deductions for relevant artificial errors and original errors. A marking scheme with sixteen variables under eight attributes is thus created. Ten marks are assigned to each unit of TQA, the sentence. For easy calculation, an arithmetic formula popularly used in statistics (Ex/n ) is adopted. With the assistance of a simple calculator, the evaluator can calculate the grade of a sentence, a sentence group, and the overall grade for an entire TT, regardless of its length. -- Perfect reliability or validity in any form of measurement is unattainable. There will always be some random error or noise in the data (McClendon, 2004: 7). Since it is the first of its type, I do not claim that my model is perfect. Variation has been found in the results of the testing performed by scholars and professional translators, but further testing based on two "easy" (markers' comment) sentences by the 50 reader evaluators respectively achieves 98% and 100% consistency, which indicates that markers' competence may equal constancy or that proper marker training and/or strict marker examination will minimize inconsistency among professional markers. My model, whose formulas withstand testing at the theoretical level and in practice, is not only ready for application, but it has profound implications beyond TQA, such as use in machine translation, and for other subjects like the role of the sentence in translation studies and translating practice.


Table of Contents

Introduction -- Literature review -- Identification of the unit of translation -- Towards a model for standardized TQA -- Mean criteria of the world -- Creating the mark deduction scheme -- Testing the model -- Applying the model -- Conclusion.


Bibliography: p. 303-317 2007

Awarding Institution

Macquarie University

Degree Type

Thesis PhD


Thesis (PhD), Macquarie University, Division of Linguistics and Psychology, Dept. of Linguistics

Department, Centre or School

Dept. of Linguistics

Year of Award


Principal Supervisor

Christian Matthiessen

Additional Supervisor 1

Canzhong Wu


Copyright disclaimer: Copyright Junxiong Huang 2008. This thesis was digitised for the purposes of Document Delivery. Macquarie University ResearchOnline attempted to locate the author but where this has not been possible; we are making available, open access, selected parts of the thesis which may be used for the purposes of private research and study. If you have any enquiries or issues regarding this work being made available please contact Macquarie University ResearchOnline - If you wish to access the complete thesis, on receipt of a Document Supply Request, placed with Macquarie University Library by another library, we will consider supplying a copy of this thesis. For more information on Document Supply, please contact




317 leaves

Former Identifiers

mq:8516 1353053

Usage metrics

    Macquarie University Theses


    Ref. manager