1. An overview of the common concepts in the area of Chatbots
2. Practical tips on how to define a scalable testing process for teams that are developing Chatbots or similar applications.
3. A demonstration of an automated testing solution that generates sample questions and their corresponding expected answers that our team is using to verify that the cognitive advisor still behaves as expected.

Hristo Gergov is a Test Manager at Musala Soft with 6 years of experience at Software Testing and Quality Assurance for innovative products of IBM, VMware and Deutsche Telekom. Working with the latest cutting-edge technologies, Hristo maintains and expands his expertise in Quality Assurance, Virtualization, Cloud Computing and Internet of Things. His passion is to implement and introduce creative testing approaches and stay tuned with the latest QA trends. Currently, Hristo Gergov coordinates a team of 20 test specialists in Bulgaria and Macedonia at a leading European SmartHome platform - Qivicon (www.qivicon.com). Qivicon is a Gateway-centric platform that controls, combines and automates multiple devices and functions from different brands.

Machines that are performing automated conversations are expected to bridge the gap between companies and end users via the most natural method for human interaction – a dialogue. It is estimated that more than 200,000 chatbots are available today and the expectations are that 2018 will mark a huge step forward in the domain.

Every new and innovative area places challenges that require a lot of creative and out of the box thinking. Chatbots are not an exception to this. They are much more than just applications with predefined questions and answers. Chatbots are highly complex software solutions that combine the advantages of Machine Learning algorithms, Natural Language Understanding and Artificial Intelligence in order to convincingly simulate how a human would behave as a conversational partner.

The user intent, the possible conversation flows, the domain specific terminology and the correct data extractions are things that have to be considered when defining tests for the product. One of the main goals of any chatbot is to provide a scalable way to utilize messaging as a channel for interaction with customers. That is why the product is constantly being enriched with new business logic. In those circumstances test automation is a must have in order to ensure a scalable delivery process. The main focus of the presentation will be on Functional, Non-Functional and Automation Testing and what unexpected challenges those testing types introduce in the context of chatbots.
I will share the experience of our QA practice in a project that is developing a cognitive advisor trained to answer any user questions related to the supply chain area.