Innovative Mind Mapping System Connecting to Smart City IoT Networks
Tags: Translation, Smart Cities, Government, Public Service, Open Data

How it works

To create the semantic network of public services, CEFAT4Cities partners start from a few abstract templates that describe what a public service looks like (who can submit a form to get access to which service, providing which type of proof?) and what the interacting entities look like (are we dealing with an organisation or a citizen?). These abstract templates consist of nodes and links (hence the term “semantic network”) and are provided by the European Interoperability Framework which governs data standards to ensure that data can be used across as many applications as possible.

Next, these templates are used as extraction filters to transform unstructured human natural language (occurring on websites, online forms, etc.) into machine-readable semantic networks which can be utilised in any software application. In short, thousands of pages of raw text are transformed into structural representations that can be used by machines, such as chatbots.

The process runs as follows: data is collected automatically from websites, then only those pages containing public service information are selected. Next, paragraphs describing administrative procedures are extracted and syntactically analysed to identify nodes occuring in the template. Finally, relations between the extracted nodes are identified (the most challenging part of the process) and the information is delivered in a standardised Open Linked Data format.

Note that the process designed can deal with all sorts of raw text in various EU languages.

To achieve all this, multilingual Artificial intelligence techniques such as automated classification are used, topic modelling, clustering, syntactic parsing, machine translation, unsupervised bilingual language induction, shallow parsing, paraphrasing, and question/answer pair generation.

Through a dedicated data schema, the Open Linked Data format used to publish results, is compatible with the FIWARE Context Broker. Any follow-up effort or downstream software application can use this schema to publish or subscribe to the public services content created.

Figure 1 . Architecture

When developing the solution, several challenging issues were discovered. As described earlier, discovering links between nodes (connecting for example an administrative procedure and all the evidence a citizen must provide to fulfil it) proved to be a non-trivial task. A unique solution had to be built, combining syntactic parsing and classification, since no out-of-the-box components existed to do this. Throughout the pipeline, a balance was needed between using monolingual AI models and multilingual AI models using translated data, since many linguistic AI models only exist for a couple of languages.

Finally, often the language itself was problematic. Current AI models excel at “recognising” the meaning of a word when it appears within a larger body of text, but when words occur isolated (for example in a title or a table) recognition and translation become more difficult. In addition, there is also the typical “call-to-action language” used on websites. This sort of language (for example, “Need more info on the necessary formalities to start as a self-employed professional?”) typically packages pieces of declarative information as questions, throwing the systems question/answer pair extraction off balance.