Macquarie University
01whole.pdf (7.82 MB)

Collaborative agent-based information processing for intelligent human computer interaction (iHCI) in mixed reality

Download (7.82 MB)
posted on 2022-03-28, 22:17 authored by Charles Zhenzhong Liu
The primary goal of this thesis is to explore collaborative agent-based information processing solutions for intelligent human-computer interaction (iHCI) systems in mixed reality applications. The thesis mainly focuses on the following research questions: 1) What are the major components of an iHCI framework for mixed reality? 2) How can we simulate human's visual processing in iHCI to differentiate the target of interest in mixed reality? 3) How can we design a knowledge-based information system to model human experience in mixed reality scene fusion? 4) How can we optimize the processes to improve the joint performance of human and computer in an iHCI system? 5) How can we manage dispersed, distributed but collaborative information processing in an iHCI system using an agent-based system architecture? 6) How can we guarantee the information security in an iHCI system for mixed reality? In the thesis, we propose an agent-based iHCI system framework for collaborative information processing. We investigate the use of a number of methods, such as context-based pattern analysis, target-of-interest differentiation, user experience estimation, subspace learning, confidential data exchange, etc., as parts of a modular system. Each agent in the iHCI system is designed with specific functionalities to support the system. We validate the system using a proof of concept for each module. The synthesis of the modules stand as a showcase to demonstrate the feasibility of the solution. The main contributions of this research include: 1) a framework of an iHCI system; 2) an agent-based collaborative information processing architecture; 3) a modular system for mixed reality fusion and novel methods 4) for target of interest (TOI) differentiation using enhanced matting and recursive learning, 5) for dynamic layering with motion detection and adaptive learning, 6) for modeling context-based awareness and pattern analysis for TOI differentiation, 7) for quality of experience and quality of service using data-aware computing and evaluation of user-experience, 8) for user identification based knowledge database management, 9) for system security using communication and detection methods such as onfidential tunnel and data-aware masquerading detection. Our findings state that the proposed framework can provide an agent-based solution to design and implement an iHCI system. The thesis answers the six research questions by developing novel methods as stated in the main contributions above. These research results can also be used for improving the design and implementation of iHCI systems for mixed reality applications.


Table of Contents

1. Introduction -- 2. Computing awareness in iHCI -- 3. The design of interactive mixed reality systems -- 4. Agent-based design system -- 5. Target of differentiation with knowledge based adaptive learning -- 6. Motion analysis based target-scene differentiation -- 7. Context based computing for scenario analysis -- 8. User experience and system performance -- 9. User-identification based database management -- 10. Confidential data exchange with cryptographic tunnel -- 11. System synthesis and implementation with an application of mixed reality scene diffusion -- 12. Conclusion and future works -- Bibliography.


Bibliography: pages 306-366 Thesis by publication.

Awarding Institution

Macquarie University

Degree Type

Thesis PhD


PhD, Macquarie University, Faculty of Science and Engineering, Department of Computing

Department, Centre or School

Department of Computing

Year of Award


Principal Supervisor

Manolya Kavakli

Additional Supervisor 1

Scott McCallum

Additional Supervisor 2

Leonard G. C. Hamey


Copyright Charles Zhenzhong Liu 2018. Copyright disclaimer:




1 online resource (xxvii, 366 pages) colour illustrations

Former Identifiers