The overall chatbot services market is predicted to grow from USD 454.1 million of 2016 to USD 1,888.4 million by 2021, at a CAGR of 33.0% with the professional services segment holding the largest market share touching USD 865.2 million in 2021, at a CAGR of 38.1% . Phew !

There are simply too many technologies that really go into making a bot comprehend a question, then reason and finally respond to it. Unlike the human brain that has answers to most questions, intelligent systems have multiple answers to every question, it can comprehend and the cycle of responding with the right answer is often as key as being able to understand. This post is really about what kind of technologies and software architecture powers the QnA element of chatbots.

For a Bot to answer a question the process is somewhat similar to the human brain’s

  • It comprehends the question
  • It creates responses from known facts or it looks for sources to retrieve facts
  • It then identifies or structures the facts to create the most suitable response
  • It waits for the vote of confidence/response from the source of the question
  • It prompts the next course of action and if the interaction ends well , marks this as a positive interaction and learns from it . In case of a negative one , it redirects it to an alternate source

AI based conversational bots are really no different  . For the first part of the interaction , technologies used are Natural Language Processing (NLP) to interpret , sometimes also using speech-to-text technologies also. Then it employs different Information Retrieval (IR) techniques along with semantics to create responses. These then need to be taken through extensive hypotheses filtering cycles to then come up with the best response . Here , the bot relies strongly on Machine Learning (ML) techniques to gather a score on confidence for the final response.

The DeepQA software architecture that has been patented by IBM and of the Jeopardy fame best answers the titled question. Below is an adapted version of DeepQA . For further reading  , click on this publicly available whitepaper link here


DeeepQA is a software architecture for analyzing natural language content in both questions and knowledge sources and is focused on Automated QnA.

DeepQA can be deemed as the reference software architecture for QnA that relies on deep content analysis and evidence-based reasoning. It represents a powerful capability that uses advanced natural language processing, semantic analysis, information retrieval, automated reasoning and machine learning.

  • First the important concepts and relations in the input language are identified, a representation of the user’s information need is built and then through search generates many possible responses
  • The parallel pipeline architecture focuses on hypothesis generation and evaluation.
  • For each possible response, independent and competing threads are spawned that gather, evaluate and combine different types of evidence from structured and unstructured sources
  • Machine Learning techniques are used to learn the weights for each scoring component in order to combine them into a single final score.
  • Then a ranked list of responses each associated with an Evidence Profile describing the supporting evidence and how it was weighted by internal algorithms is created
  • Finally the most confident answer is then created using semantics and sent as an output.

This was perhaps the first intelligent system that has well documented evidence on how it discovers and evaluates potential answers and gathers and scores evidence for those answers in both unstructured sources, such as natural language documents, and structured sources such as relational databases and knowledge bases .

Today we have ready Bot frameworks powered by Watson , Microsoft BOT Framework and Google that have such or quivalent QnA architecture embedded leaving much creative scope for business leaders to design its application and personality and most importantly the end user experience and impact to business.

We all know , how bots can risk the entire experience cycle if not ‘trained’ correctly to answer questions and most often it is seen that a simple human escalation event is what was overlooked in the entire complex wiring of technologies .

At other times it has also been observed that too many intents are crammed into a bot . Bots with a limited set of responsibilities and well defined personalities – ie tone of response , characterization work best when put to direct actor roles.

At this point , let me also stop and talk about the roles a conversational bot can play – actor , assistant and advisor. More on this in a later post that will touch upon the roles they play in the bank.

Stay tuned , more coming up on the AI Primer series .

Leave a Reply