Human-agent teamwork in collaborative virtual environments

There is a growing interest in the use of heterogeneous teams comprised of humans and Intelligent Virtual Agents (IVAs). Human teamwork studies have provided cumulative knowledge about team features and performance. How well this knowledge transfers to human-IVA teams requires further investigation. The development of a Shared Mental Model (SMM) between team members and effective communication of the shared knowledge have been found to improve human teamwork performance. In human-IVA heterogeneous teams, the communication required to develop a SMM is further complicated as each party belongs to different worlds (i.e. real and virtual). Communication is a vital factor in the collaboration between team members. Creating IVAs that are able to communicate with humans in Virtual Environments (VEs) is a challenging research area. When both the IVA and the human user should communicate together while performing a collaborative activity, communication becomes more critical and the challenge becomes more difficult. Moreover, humans may differ in how they produce and perceive communication acts according to their personality traits. The main aim of my PhD is to study the factors that tend to improve team performance and foster collaboration between humans and IVAs in VEs. To understand the requirements of human-IVA collaboration in VEs, we present the design of a framework based on Activity Theory called Multi-Agent Collaborative VirtuaL Learning Environment (MACVILLE), which is a framework to understand the nature of collaboration in human teams. The MACVILLE framework indicated the importance of communication for collaboration in the VE. In addition, the proposed framework demonstrates the need to extend the design of an IVA to include collaborative and social abilities. To address this essential extension we propose an agent architecture that handles two-way human-agent collaboration. To support human-IVA communication in VEs, we present Human-Agent Teamwork Communication Model (HAT-CoM). HAT-CoM was designed based on Speech Act Theory (SAT), which is a methodology to understand the structure of human speech. HAT-CoM was implemented and integrated into our agent architecture. VII To evaluate the impact of HAT-CoM on developing/breaking a SMM between a human and an agent, a study was conducted with 66 undergraduate students. The evaluation was conducted by analytical and inductive means. The analytical evaluation aims at investigating the impact of HAT-CoM components, i.e. verbal and non-verbal, on the development of SMM features, i.e. knowledge about the task and the team. The inductive evaluation aims at verifying the development of a SMM via HAT-CoM through tracking the changes in the designated outcomes of the SMM. The outcomes of the SMM are anticipating a teammate’s decisions, reduced explicit communication, match in cognitive perspective, competence in decision-making (ease of flow of decisions) and involvement in the shared task. Another aim of the study was to investigate the impact of an implausible or unreasonable request on the SMM. The results show that HAT-CoM is effective in assisting the human and agent teammates to develop a SMM. In addition, the results show that an implausible request breaks the developed SMM. A second study was carried out to investigate the impact of the IVA’s multimodal communication on the development of a SMM between humans and IVAs. Moreover, this study aimed to explore the impact of the developed SMM on the human’s trust in the IVA’s decisions and the human’s commitment to honour his promises to an IVA. The result showed that there is a significant positive correlation between the developed SMM and the human’s trust in the IVA’s decision and the human’s commitment to honour his/her promises (the establishment of the social aspect of teamwork). Additionally, the results showed a collective effect of all of these aspects on human-agent team performance.showed that IVA multimodal communication plays a crucial The two conducted studies showed that IVA multimodal communication plays a crucial role in the development of a SMM between humans and IVA; nevertheless, humans may differ in how they produce and perceive communication acts according to their personality traits. To investigate how different IVA personalities affect multimodal communication and development of a SMM, a third study was carried out. In this study, we seek to understand how people trust an IVA teammate. The study considers two facets of trust: personality and cognition. Results indicated that cognitive-based facets played a more dominant role in establishing trust than personality-based facets. Additionally, the results showed that human trust in the IVA had a significantly positive influence on human-IVA team performance. The results of the three conducted studies stressed the importance of IVA multimodal communication on the development of trust and commitment in human-IVA teamwork. Trust and commitment were found to contribute positively to the development of a SMM and hence team performance. Additionally, personality traits were found to influence human perception of IVA multimodal communication.