# A Modular Framework for Domain-Specific Conversational Systems Powered by Never-Ending Learning
## 4 Experimental Results
### Specific Banking Domain for the Portuguese Language  
The chosen domain for the dialog system is the banking domain, which has a vocabulary with specific terms, and security during its operation is mandatory. The system supports the four main transactions in an internet banking application, which are money transfers, balance checking, extract inquiries, and payments. These operations can be performed, or questions can be asked about them. Thus, the domain-specific classifications needed are operations, which type of question is asked, and the sentiment expressed. Some of the domain-specific slots are account number, bank number, and payment type. In order to recognize these domain-specific classifications and entities, we trained machine learning models using an artificially generated dataset that was generated by the BNF grammar method mentioned in Section 3.1. We manually constructed a generation grammar and then used it to produce a training dataset with 15,000 labeled examples, as well as a test dataset with 600 examples. Both datasets had equal proportions of all classes. We evaluated this by using the validation dataset, measuring the F1 score of the classification, and naming the entity-recognition tasks. The intent model had a 96.08 F1 score, the NER model had an 85.71 F1 score, the operation classifier had a 100 F1 score, the question classifier had a 94.98 F1 score, and the emotion classifier had a 99.46 F1 score. The three ontologies described in Section 3.4 created for the banking domain are shown in Figures 5-7. The task ontology contains the four operations: money transfer (\"transferencia\"), balance (\"saldo\"), extract inquiry (\"extrato\"), and payment (\"pagamento\"), as a child of the root node \"procedimento\". The red nodes are classes and need further disambiguation, choosing which type of payment or money transfer the user wants. This ontology also shows that for the \"interbancaria\" money transfer, the data from the sender \"pessoa_remetende\" and receiver \"pessoa_favorecida\" are mandatory. The domain ontology has a similar graph structure to the task ontology because when the user asks a question about a term in the task ontology, the same spot can be found in the domain ontology to obtain the knowledge to respond to the user's question. In this ontology, there are the properties for each node, such as the maximum value for the money transfer between different banks, which the user could ask for. There is an annotation in each node with its definition so that questions such as \"What is TED?\" can be answered with the correct definition given by the specialist consulted during the creation of the ontology. Finally, the support ontology has information from past operations that can be reused in a future dialog. For example, the yellow node is a money transfer for the same bank (\"intrabancaria\") between two persons: \"usuario_1\" and \"maria_silva\". Thus, if the same user wants to transfer to [PERSON] again, her information can be obtained from the support ontology, and the user does not need to repeat her banking information, such as account number and agency. Therefore, the first time a user makes a money transfer to an unknown person, it is necessary to explicitly inform all the information about the receiver, and with the approval of the user, this new contact can be saved in the support ontology to be used again in the future. Figure 5: Task ontology. The red nodes are classes representing banking operations, procedure (”predecdimento”), money transfer (”transferncia”), interbank transfer (“interbankcia”), and payment (”pagamento”). The green node is a child class representing the intrabank transfer (“interbankcia”), and the beige nodes represents the objects needed by this interbank transfer: favored person (”pessoa favorecia”) and sender person (“pessoa remetende”). The blue nodes are instances of operations: extract inquiry (”extrato”) and balance (“saldo”). The relations shown are needs (”precida de”) and needed by (“e demandado por”). Figure 6: Domain ontology.