Conversational Agents for Intelligent Buildings (2020)

Abstract: We will demonstrate a deployed conversational AI system that acts as a host of a smart-building on a university campus. The system combines open-domain social conversation with task-based conversation regarding navigation in the building, live resource updates (e.g. available computers) and events in the building. We are able to demonstrate the system on several platforms: Google Home devices, Android phones, and a Furhat robot.

## introduction the combination of social chat and task-oriented dialogue has been gaining more and more popularity as a research topic @xcite @xcite . in this paper, we describe a social bot called alana 1 and how it has been modified to provide taskbased assistance in an intelligent building (called the grid) at the heriot-watt university campus in edinburgh. alana was first developed for the amazon alexa challenge in 2017 @xcite by the heriot-watt university team and then improved for the same competition in 2018 @xcite . the team reached the finals in both years. now alana successfully serves as a system core for other conversational ai projects @xcite . in the grid project, several new functionalities have been added to the original alana system which include providing the users with information about: currently, our intelligent assistant is available for users on several google home mini devices distributed in the grid -a large university building with multiple types of users ranging from students to staff, and visitors from business/industry. it is also available on android phones via google actions as part of the google assistant. the system is reconfigurable for other buildings, via a graph representation of locations and their connectivity. it connects to live information about available resources such as computers and to an event calendar. ## system architecture figure 1 presents the architecture of the system. the alana system is an ensemble of several different conversational bots that can all potentially produce a reply to the user's utterance. each bot uses different information resources to produce its reply. example resources are: wikipedia, reddit, many different news feeds, a database of interesting facts etc. additionally, there are also conversational bots that drive the dialogue in case it has stalled, deal with profanities, handle clarifications, or express the views, likes, and dislikes of a virtual persona. the decision regarding which bot's reply is selected to be verbalised is handled by the dialogue manager (dm). asr/tts in the grid project, the audio stream is handled using the google speech api and the system is therefore also available as an action on google assistant on android phones. nlu in the alana system, users' utterances are parsed using a complex nlu pipeline, described in detail in @xcite , consisting of steps such as named entity recognition, noun phrase extraction, co-reference and ellipsis resolution, and a combination of regex-based and deeplearning-based intent recognition. in the grid project, an additional nlu module has been implemented for building-specific enquiries which uses the rasa 2 framework. in the persona bot we use aiml patterns for rapid reconfigurability and control. nlg the nlg strategy depends on the different conversational bots. it ranges from the use of complex and carefully designed templates to automatically summarised news and wikipedia articles @xcite . ## example dialogues general information about the grid building is encoded in persona's aiml templates. it is this bot that responds to questions whose answers are generally fixed over time -for example about opening hours and where one can buy a cup of coffee. for examples of responses generated by persona, see dialogue 1. the ethics bot has been implemented to obtain users' consent for storing and processing transcriptions of their conversations. the request for user consent is omitted in the example dialogues. the directions bot handles requests for locations of rooms and directions to them. it connects to the neo4j graph database where it retrieves the shortest paths. the events bot responds whenever events information is requested for either a specific room or the building in general. the information is stored in the neo4j database as well where the events are assigned to particular rooms. it is retrieved from a university timetable system and updated every day. a: welcome to the grid building! how can i help you today? u: what does grid stand for? a: (persona) grid stands for global, research, innovation and discovery. is there anything else i can help you with? u: what time does this building close? a: (persona) this building never closes. it is open 24/7. did you know that you can have a chat with me about music? also, if you're looking for a specific room, you can ask me how to get there. u: actually, i really need a cup of coffee. do you know where i can get one? a: (persona) you can have a cup of coffee at the cafe on the ground floor. it looks like a tuk tuk. can i help you with anything else? u: what is a tuk tuk? a: (wiki bot) i think that an auto rickshaw is a motorized version of the pulled rickshaw or cycle rickshaw. most have three wheels and do not tilt. they are known by many terms in various countries including auto, baby taxi, bajaj, chand gari, lapa, tuk-tuk, 3wheel or tukxi. u: wow, that's interesting. thanks, goodbye! a: it was lovely talking to you! goodbye! dialogue 1: questions about the grid building. u -user, and a -alana system. the role of the resources bot is to provide the user with information about how many computers are currently free in the grid building's labs. the bot scrapes the data live from a university website. dialogue 2 presents example responses produced by the directions, events, and resources bots. dialogue 3 shows an example of social chat one can have with our system. alana offers conversation about various topics such as music, movies, and sports. it provides current news as well.