We developed a proof of concept (PoC) that makes the client's various knowledge sources searchable using ChatGPT within a MRKL approach (Modular Reasoning, Knowledge, and Language). In this setup, ChatGPT acts as a kind of router, retrieving information from one or more knowledge sources depending on the query. The system's output includes a summary of the search results that answers the original question, while also citing the underlying information sources.
For employees, this can mean significant time savings. Typically, they have access to multiple internal knowledge databases, but searching through them during a call can be cumbersome, time-consuming, and costly.
To quantitatively evaluate the solution, we developed a framework around it. This framework allows us to vary different parameters, such as those in the areas of word embeddings and prompt engineering. Responses are generated for each parameter set and then compared by ChatGPT with answers previously marked as correct. This process is iterated multiple times to fine-tune the parameters for optimal answer quality.
