Towards adaptive context management for intelligent Conversational Question Answering systems

<p dir="ltr">Conversational Question Answering (ConvQA) systems are designed for facilitating natural and dialogue-based interactions between users and machines via enabling users to inquire a series of related questions within a cohesive conversation flow. Unlike traditional Question Answering (QA) systems that handle isolated questions, ConvQA systems maintain context throughout a conversation and provide responses which naturally adapt to follow-up questions and contextual information, thereby enhancing both relevance and coherence. These systems have become increasingly imperative for a number of interactive applications, e.g., virtual assistants (Siri, Alexa, and Google Assistant), customer service chatbots, and generative AI agents, wherein understanding context and history of a conversation is crucial for delivering accurate responses. </p><p dir="ltr">Nevertheless, these systems often face considerable challenges in managing and utilizing the conversational history, particularly, in multi-turn conversations. As conversations extend over multiple conversational turns, the existing ConvQA systems struggle in balancing the inclusion of relevant context while adhering to the token limits of Large Language Models (LLMs), thereby leading to a decline in relevance and coherence of the responses. Therefore, effectively optimizing context utilization is indispensable for managing conversational history. Retaining too much irrelevant conversational history could lead to incoherent or incorrect responses, whereas, over-pruning past conversational history may result in the loss of essential information. Striking the right balance between preserving relevant context and maintaining model efficacy is a key to ensuring high quality and contextually appropriate responses in multi-turn conversations. </p><p dir="ltr">To address this challenge, this research thesis offers a comprehensive literature review for enhancing a deeper understanding of the ConvQA domain. It systematically analyzes the existing ConvQA systems along with their salient components, i.e., history selection, question understanding, and answer prediction, and further explores the integration of the advanced machine learning techniques, i.e., Reinforcement Learning (RL), Knowledge Distillation (KD), Contrastive Learning (CL), Active Learning (CL), and Transfer Learning (TL), with popular LLMs. The literature review also offers a detailed comparison of various ConvQA approaches by highlighting their respective unique characteristics and presents critical insights into how datasets, i.e., CoQA, QuAC, SQuAD 2.0, CANARD, and QReCC, have been employed to evaluate the accuracy of the ConvQA systems. </p><p dir="ltr">One of the key contributions of this research thesis is envisaging of a framework entitled, Adaptive Context Management (ACM), for managing the conversational history by optimizing context utilization in the ConvQA systems. The said ACM framework introduces three salient modules, i.e., the Context Manager (CM) module, Summarization (SM) module, and Entity Extraction (EE) module. The CM dynamically adjusts and prioritizes conversational history to maintain relevance and coherence within the token limit of an LLM, thereby ensuring that the critical context is preserved for accurate and meaningful responses. It assigns priority to the recent conversational turns since they are more relevant to the current question and retains them in their original form as unmodified context. Meanwhile, the SM module summarizes the older conversational turns by retaining essential information and the EE module extracts key entities from the oldest conversational turns to maintain continuity in responses. Together, these modules enable the framework to deliver contextually aware responses even in the case of extended multi-turn conversations. Through rigorous experiments, the ACM framework demonstrated significant improvements across metrics, i.e., F1, ROUGE-L, ROUGE-1, and BLEU scores, in contrast to the pipeline approach.</p>