CayBot-19 : Conversational AI Chatbot – Ideation to Execution

Introduction:

I hope you and your families are staying safe during these are unprecedented times. Given the pandemic nature of the COVID-19 virus and the emotions it evokes, all of us are trying to stay informed, as best as we can, as the situation evolves rapidly. To say the internet is inundated with information, would be a gross understatement. Unfortunately, with the deluge of information, comes dis-information as well. It becomes, therefore, all the more important that we have access to truthful and verified news as the pandemic develops. As a technology consultant, I felt it my responsibility to leverage technology we work with, to create a conversational AI chat bot, to help our society and community access live and verified information.

Phase 1:

Inspiration:

On March 20, 2020, I came across an interesting piece of news that said the WHO had launched a WhatsApp Bot to help people navigate through the barrage of information on COVID-19. Exploring the concept further, I wondered if and how we could integrate WhatsApp with such conversational AI Bots.

Source of Data:

I traced the source of of the (verified) live updates that are maintained by Arcgis.com.

Once I had my source zeroed in, I evaluated the Geographical Information System (GIS), a framework for gathering, managing and analyzing data. Since GIS integrates many types of data, I wagered it could help reveal deeper insights into data such as pattern, relationship and so on.

While exploring the site that hosts live geographic data, I did some reverse engineering to find a REST API to get all this data. After few tweaks on the Network Tab, I managed to get what I was after.

https://services1.arcgis.com/0MSEUqKaxRlEPj5g/arcgis/rest/services/ncov_cases/FeatureServer/1/query?f=json&where=(Confirmed%20%3E%200)%20AND%20(Deaths%3E0)%20AND%20(Country_Region=%27INDIA%27)&returnGeometry=false&spatialRel=esriSpatialRelIntersects&outFields=*&orderByFields=Deaths%20desc,Country_Region%20asc,Province_State%20asc&outSR=102100&resultOffset=0&resultRecordCount=250&cacheHint=true

Phase 2: 

Gathering talent:

Now that I had completed my groundwork, I wanted to understand how to create a messaging service and conversational agents that source this data, and make it available to and through WhatsApp. I sat down with my Peers (that’s how we refer to the exceptional talent at PeerIslands!), to explore how we can do our bit in ensuring our society has access to verified information. Parthibarajan Perumal Sekar‌‌‌‌ and Ankita Joshi lent their time, expertise – and patience – to work on this project, and this blog!

The platform:

Our question next was to understand how to integrate a messaging platform with our data sources. Could WhatsApp be our platform of choice? It is currently one of the most-used messaging app., both by individuals and businesses. Would the integration be practical in the first place? We also wanted to explore Google’s cloud platform, with DialogFlow of Google.

Conversational AI Chat Bot:

DialogFlow is a popular and easy-to-use, web-based conversational agent building platform offered by Google. Having evolved from an earlier version called API.AI, DialogFlow supports both voice, and text. It helps us create an experience by providing pre-defined tools to understand two tasks:

  • Understand and generate natural language utterance
  • Manage the conversation

Phase 3:

Setting up DialogFlow:

This was a pretty easy process. All we had to do was:

  1.  Create an account with https://dialogflow.com
  2.  Create basic agents, and save it
  3.  This took us to a page with Intents listing, with two preset Intents:
    • Default Fallback Intent
    • Default Welcome Intent
  4.  We then added our Intents
    • It is important to add an Intent, because there are more ways than one to ask the same question. The process of identifying an Intent is to map the many ways to an Intent
    • Our Intent was to identify what a user or the bot wanted to convey, using buttonsDialog Bot Intents
  5. Next, we added the Training Phrases
    • Training Phrases are example phrases for what end-users’ type
    • For each intent we created several Training Phrases, so when an end-user expression resembles one of these phrases, DialogFlow could match the Intent
    • To explain: consider the Latest Stats Intent. We created a couple of Training Phrases as seen below:Dialog-Bot-Training-Phrase
  6. In the Responses tab, we added the bot’s response to the user, in the Text Response field.
    • We added more responses so that the agent could randomly pick one to make it less repetitiveDialog-Bot-TrainingPhrases
  7. Finally, we tested it using the simulator on the right side of the page.
    • In the Try it now box, we typed stats and pressed Enter (refer image below)
    • Now, an interesting feature by the Dialogue bot is the Fulfillment feature. It dynamically generate responses from the ArcGIS API. When we enable fulfillment for an intent, DialogFlow responds to that intent by calling a service (Web-hook that reads continuous data from ArcGIS) that we’ve defined
    • In our case, we wanted to check the total number of cases in a particular country. Our Web-hook hit ArcGIS Rest API, collected the data, and responded with the total number of cases, in the country selected by the user. Important to note here: each intent has settings to enable fulfillment, and if an intent without fulfillment enabled is matched, DialogFlow uses the static responses defined for the intent

In summary:

  1. End user types an expression (Eg: Corona virus stats for India)
  2. DialogFlow matches end-user expression to an intent and extracts parameters
  3. DialogFlow sends a web-hook request message to our web-hook service. This message contains information about the matched intent, action, parameter and finally the response defined for the intent.
  4. Our service performs actions and reads the COVID-19 lives updates
  5. Our API Service endpoint sends a web-hook response message to DialogFlow. This message contains the response expected for end users:
    "features": [ {
    "attributes": {
    "OBJECTID": 163,
    "Province_State": null,
    "Country_Region": "India",
    "Last_Update": 1584812583000,
    "Lat": 20.5937,
    "Long_": 78.9629,
    "Confirmed": 330,
    "Recovered": 23,
    "Deaths": 4,
    "Active": 303,
    "Admin2": null,
    "FIPS": null,
    "Combined_Key": null }
    }]}
  6. DialogFlow sends the response to end user
  7. Finally end user sees the COVID -19 live data

With our chat bot in place, we needed to integrate all these components, to make this integration meaningful. Twilio helped us achieve this.

Messaging Platform:

Twilio is a cloud communication PaaS component. It allows us to programmatically make and receive phone calls, send and receive text messages, and perform other communication function using its web service APIs. We can integrate any web-hook with Twilio to receive and send data from any other system.

  • Create an account with Twilio https://www.twilio.com/login
  • Go to Programmable SMS section and select WhatsApp channel setup your testing Sandbox
  • Enter any Mobile Number, and Twilio will create a Sandbox code
  • And we are ready. We can setup one-way or two-way messaging
  • Go to Sandbox configuration and configure endpoint URL (my case it was dialog flow Bot URL)

Twilio Sandbox Integration with Dialogue Bots:

Let’s look at the technical architecture of all three systems we’ve spoken about in the preceding paragraphs.

  1. A WhatsApp user sends request to Twilio
  2. Twilio receives the request and passes it to the DialogFlow bot, through endpoint configured in Twilio sandbox configuration. Once the DialogFlow bot agent receives the user’s request, it goes to list of pre-defined intents
  3. After DialogFlow maps the user’s request to one of predefined intents, it sends a response  to user
  4. The DialogFlow agents receive the request, compares it against the intent, picks appropriate responses, and finally sends it back to Twilio. Twilio then sends it to the user
  5. While Twilio is only a messaging platform, the DialogFlow bots are decision makers that also have the capability to integrate web-hooks to obtain data from third party API
  6. Finally, our WhatsApp chat bot is ready! Twilio has given us Sandbox number for testing. This chat bot can now server as a help desk as well!

Here are snapshots of our Chat Bot:

This integration can be made with any social messenger.

I would love to hear your thoughts on this, or if you have questions, I’d be happy to address them.

Co-authors: 

Parthibarajan Perumal Sekar‌‌‌‌ and Ankita Joshi.

References:

  1. COVID-19 live updates: https://www.arcgis.com/apps/opsdashboard/index.html#/bda7594740fd40299423467b48e9ecf6
  2. Getting started with Twilio: https://www.twilio.com/console/sms/getting-started/build
  3. 3. DialogFlow Fulfillment: https://cloud.google.com/dialogflow/docs/fulfillment-overview
  4. 4. DialogFlow Agent design: https://cloud.google.com/dialogflow/docs/agents-design