Collaborative agent-based information processing for intelligent human computer interaction (iHCI) in mixed reality

The primary goal of this thesis is to explore collaborative agent-based information processing solutions for intelligent human-computer interaction (iHCI) systems in mixed reality applications. The thesis mainly focuses on the following research questions: 1) What are the major components of an iHCI framework for mixed reality? 2) How can we simulate human's visual processing in iHCI to differentiate the target of interest in mixed reality? 3) How can we design a knowledge-based information system to model human experience in mixed reality scene fusion? 4) How can we optimize the processes to improve the joint performance of human and computer in an iHCI system? 5) How can we manage dispersed, distributed but collaborative information processing in an iHCI system using an agent-based system architecture? 6) How can we guarantee the information security in an iHCI system for mixed reality? In the thesis, we propose an agent-based iHCI system framework for collaborative information processing. We investigate the use of a number of methods, such as context-based pattern analysis, target-of-interest differentiation, user experience estimation, subspace learning, confidential data exchange, etc., as parts of a modular system. Each agent in the iHCI system is designed with specific functionalities to support the system. We validate the system using a proof of concept for each module. The synthesis of the modules stand as a showcase to demonstrate the feasibility of the solution. The main contributions of this research include: 1) a framework of an iHCI system; 2) an agent-based collaborative information processing architecture; 3) a modular system for mixed reality fusion and novel methods 4) for target of interest (TOI) differentiation using enhanced matting and recursive learning, 5) for dynamic layering with motion detection and adaptive learning, 6) for modeling context-based awareness and pattern analysis for TOI differentiation, 7) for quality of experience and quality of service using data-aware computing and evaluation of user-experience, 8) for user identification based knowledge database management, 9) for system security using communication and detection methods such as onfidential tunnel and data-aware masquerading detection. Our findings state that the proposed framework can provide an agent-based solution to design and implement an iHCI system. The thesis answers the six research questions by developing novel methods as stated in the main contributions above. These research results can also be used for improving the design and implementation of iHCI systems for mixed reality applications.