01whole.pdf (5.86 MB)
Download file

Document restructuring to promote semantic coherence in topics

Download (5.86 MB)
thesis
posted on 29.03.2022, 00:40 by Fiona Martin
This thesis examined whether simple preprocessing of documents such as lemmatising text, or removing or weighting certain parts of speech, could generate better quality topics, faster, using Latent Dirichlet Allocation (LDA) topic modelling. Past work has generally attempted to improve topic modelling performance by making changes to the topic modelling algorithm itself. This study examines the simpler option of transforming the input documents to the LDA algorithm. Topic quality was assessed on a range of measures that examined both topic interpretability, and how well the topics represented the source documents. The results indicate that topic quality was improved, and the time to generate the topics was less, if the input documents were reduced to only nouns, or nouns and adjectives, when the numbers of topics to be generated was 200 or 500 topics. This study also found that even when the number of topics to generate was not large, input documents could be reduced to select parts of speech to speed the generation of topics, with no loss of topic quality. The implications of these results are that very large data sets may benefit from being lemmatised and reduced to simply the nouns prior to topic modelling.

History

Alternative Title

Topic coherence after document restructuring.

Table of Contents

1. Introduction -- 2. Background -- 3. Literature review -- 4.Method -- 5. Results -- 6. discussion -- 7. Conclusion.

Notes

Spine title: Topic coherence after document restructuring. Theoretical thesis. Bibliography: leaves 52-56

Awarding Institution

Macquarie University

Degree Type

Thesis MRes

Degree

MRes, Macquarie University, Faculty of Science and Engineering, Department of Computing

Department, Centre or School

Department of Computing

Year of Award

2015

Principal Supervisor

Mark Johnson

Rights

Copyright Fiona Martin 2015. Copyright disclaimer: http://mq.edu.au/library/copyright

Language

English

Extent

1 online resource (x, 56 leaves) tables

Former Identifiers

mq:69672 http://hdl.handle.net/1959.14/1256589