H2020-EUJ-02-2016
H2020 Grant Agreement Number 723076
NICT Management Number 18302

Deliverable D2.11

Feedback and Validation
Report (V2)
Version V1.0
December 31st, 2018

ABSTRACTThis document contains a report on the pilot execution and the validation reportof the use cases and platform. It will summarize the main aspects of the weekly
feedback from the pilot execution and provide a detailed report of the use caseand platform validation process and results.

This work is licensed under the Creative Commons Attribution 4.0 International License.
To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

Deliverable D2.11

Feedback and Validation Report (V2)Disclaimer
This document has been produced in the context of the CPaaS.io project which is jointly funded by theEuropean Commission (grant agreement n° 723076) and NICT from Japan (management number 18302).
All information provided in this document is provided "as is" and no guarantee or warranty is given thatthe information is fit for any particular purpose. The user thereof uses the information at its sole risk andliability. For the avoidance of all doubts, the European Commission and NICT have no liability in respectof this document, which is merely representing the view of the project consortium. This document issubject to change without notice.

Document Information
Editors
Authors
Reviewers
Dissemination
Level
Contractual
Delivery Date
ActualDelivery Date
Keywords

H2020 EUJ-02-2016

Sylvain Prost (TTN), Hiroshi Amano (MSJ)
Sylvain Prost (TTN), Hiroshi Amano (MSJ), Martin Strohbach (AGT), Juan
A. Martinez (OdinS), Antonio Skarmeta (OdinS)
Stephan Haller (BFH)
R
PU
December 31st, 2018
December 31st, 2018

CPaaS.io

Page 2 of 25

Deliverable D2.11

Feedback and Validation Report (V2)

Revision History
Rev.
0.1
0.2

Date
14/11/18
16/11/18

0.3

19/11/18

0.4

12/12/18

0.5

28/12/18

Description
TOC
Contribution to Smart Parking Use
CaseEdits in Tokyo Management ofService Vehicles (5.1, 5.3, 5.4 except
5.2)
Updated EUE description in section
4
Update on Yokosuka

1.0

29/12/18

Review and finalisation

H2020 EUJ-02-2016

CPaaS.io

Contributors
Sylvain Prost (TTN)
Juan A. Martinez (OdinS), Antonio
Skarmeta (OdinS)
Toshihiko Yamakami (ACC)

Martin Strohbach (AGT)
Chiaki Ishikawa (YRP),
Katsunori Shindo (YRP)
Stephan Haller (BFH)

Page 3 of 25

Deliverable D2.11

Feedback and Validation Report (V2)

Table of Contents
1

Introduction............................................................................................................................................................................... 5

2

Waterproof Amsterdam ........................................................................................................................................................ 5

3

4

5

6

7

8

9

2.1

Feedback ........................................................................................................................................................................... 5

2.2

CPaaS.io integration...................................................................................................................................................... 6

2.3

Performance and stability ........................................................................................................................................... 6

2.4

Validation report ............................................................................................................................................................ 7

Yokosuka Emergency Medical Care ................................................................................................................................. 7
3.1

Feedback ........................................................................................................................................................................... 7

3.2

CPaaS.io integration...................................................................................................................................................... 8

3.3

Performance and stability ........................................................................................................................................... 8

3.4

Validation report ............................................................................................................................................................ 9

Event Management – Enhanced User Experience..................................................................................................... 10
4.1

Feedback ......................................................................................................................................................................... 10

4.2

CPaaS.io Integration.................................................................................................................................................... 10

4.3

Performance and stability ......................................................................................................................................... 11

4.4

Validation Report ......................................................................................................................................................... 12

Event Management – Tokyo Management of Service Vehicle ............................................................................ 13
5.1

Feedback ......................................................................................................................................................................... 13

5.2

CPaaS.io integration.................................................................................................................................................... 13

5.3

Performance and stability ......................................................................................................................................... 14

5.4

Validation report .......................................................................................................................................................... 15

Tokyo Public Transportation ............................................................................................................................................. 15
6.1

Feedback ......................................................................................................................................................................... 15

6.2

CPaaS.io integration.................................................................................................................................................... 16

6.3

Performance and stability ......................................................................................................................................... 17

6.4

Validation chart............................................................................................................................................................. 17

Smart Parking – Murcia....................................................................................................................................................... 18
7.1

Feedback ......................................................................................................................................................................... 19

7.2

CPaaS.io integration.................................................................................................................................................... 20

7.3

Performance and stability ......................................................................................................................................... 21

7.4

Validation report .......................................................................................................................................................... 21

Sapporo Visitor Experience ............................................................................................................................................... 22
8.1

Feedback ......................................................................................................................................................................... 22

8.2

CPaaS.io integration.................................................................................................................................................... 22

8.3

Performance and stability ......................................................................................................................................... 23

8.4

Validation report .......................................................................................................................................................... 23

Outlook ..................................................................................................................................................................................... 23

H2020 EUJ-02-2016

CPaaS.io

Page 4 of 25

Deliverable D2.11

Feedback and Validation Report (V2)

Appendix 1 - Enhanced User Experience, Color run feedback form ........................................................................... 25

1 Introduction
Quite a few pilots and use cases accompany the CPaaS project. Even if they are part of the same project,their goals vary quite a lot, ranging from event to infrastructure protection. Nonetheless, they have same
objective, improve citizens' life in cities. As such, the main source of feedback are the users (citizens). Usecases on a city-wide scale are often backed or impacted by governmental organisations, the owner of theuse case should also then listen for feedback from this organisation. This document is summarizing thedifferent feedbacks received for each use cases which include user, stakeholder satisfaction reports; but
also because the use cases work on large scale and/or large amount of data, stability, response time,scalability are metrics that impact the user experience and the pertinence of a projects in the smart city
context.
Each use case will demonstrate how does it gather feedback, identify the main aspect and include it in eachdevelopment cycle. They will also describe any tools and metrics that display the performance and stabilityof the application.

2 Waterproof Amsterdam
2.1 Feedback
In the current state the only shareholder of the use case is Waternet, the company in charge of the public
water system in Amsterdam. There haven't been any field tests conducted yet, so we don’t have any user
feedback for the application. Nonetheless, one important point of feedback is the success of the green
rooftop “De Dakdokters"1: Waternet has decided to finance them into producing and deploying new watertanks, and as they are our partners in the Waterproof application, this will also be beneficial for the workof the CPaaS.io project.
The scope defined by Waternet for the applications is to connect water tanks and through them receive
different kinds of information such as the water level, temperature, etc., but also to be able to command
the water tank to release or retain water. The application is successful in both tasks, although so far onlyin the lab.
The manufacturer of the tanks already has a working solution for their products (API, dashboard, etc.) but
is willing to make changes to its products to either forward the data or migrate the solution to CPaaS. We
have already identified what we need to work on with them, so we can connect them to a bigger network.

1 Cf. http://dakdokters.nl/en/green-roofs/

H2020 EUJ-02-2016

CPaaS.io

Page 5 of 25

Deliverable D2.11

Feedback and Validation Report (V2)

2.2 CPaaS.io integration

Figure 1 - Waterproof CPaaS.io integration

The application makes use of three layers of the CPaaS.io service architecture. The IoT resource layer,
Integration layer, Virtual Entity layer and the SMART City services layer. Finally, FogFlow is used toorchestrate the different services.
Main advantage to this disposition is the separation of the services and the handling of the services. All
the forwarding and mapping is handled inside CPaaS.io and put at disposition of the developer. It also
allows quick deployment and transferability.

2.3 Performance and stability
Our development cycle includes peer reviewing and continuous integration. It allows us to detect potential
bugs and deliver a reliable and performing product. We are also having manual tests in our lab to ensure
end-to-end functionality.
On average The Things Network handle more than twenty-two thousand devices every day and is running
99% of the time. Furthermore, based on results of the first prototype of the version the CPaaS.io platform
can handle more than a hundred requests per second without increased latency in any service of theplatform. It is safe to say that there won’t be any issue in using both of these services.
A remark during the review meeting in Tokyo regarding the cross-region deployment and scalability led
us to rethink the solution architecture. The current solution is fulfilling the main functional requirement.
However, the architecture is monolithic, as such it lacks flexibility, and as such harder to maintain, scaleand evolve. We started to work toward a distributed architecture using FogFlow, which is part of CPaaS.io.
Using the FogFlow service topology and automation we will have better reliability as well as transferability.

H2020 EUJ-02-2016

CPaaS.io

Page 6 of 25

Deliverable D2.11

Feedback and Validation Report (V2)

2.4 Validation report
D2.6 already showed that the validation criteria are covered. In this version we will focus more on the
progress of the integratrion with the CPaaS.io platform.
Prototypeversion
1.0
2.0
3.0

4.0

Components# Components

Documentation

None
FIWARE IoT Broker, LoRaWAN
IoT Bridge, IoT AgentFIWARE IoT Broker, FIWARE
Context Broker, LoRaWAN IoT
Bridge, IoT Agent, Orion Context
Broker
FIWARE IoT Broker, FIWARE
Context Broker, LoRaWAN IoT
Bridge, IoT Agent, Orion Context
Broker, FogFlow, FIWARE app
developer interface

0
3

D2.2 First prototype releaseD2.4 Second prototype release5

D2.8 Third prototype release7

D2.9 Validation of federation andtransferability, D2.10 Final Prototype
release

It took a bit of time before starting the integration with the platform as we first needed to create andexperiment with different device prototypes. Once we had a stable model and field deployment of devices,
we could work on integrating with CPaaS.io. Over the course of time we made four versions with different
devices and UIs. On each iteration we integrated more components to arrive at a total of 7.

3 Yokosuka Emergency Medical CareIn this section, we describe how we are validating the results of Yokosuka Emergency Medical Care
Application.

3.1 Feedback
In the current state, we are designing a use case system of Yokosuka Emergency Medical Care based on
the CPaaS.io architecture, and getting it ready for the first live demonstration/experiment together with
the Yokosuka City Fire Bureau. This took place on January 27, 2018 in conjunction with Ken Sakamura’s
speech and a presentation by a medical doctor on Yokosuka Emergency Medical Care at the 68th Annual
Academic Conference in the Kanto region of the Japanese Association of Acute Medicine that took placeon Hongo campus of the University of Tokyo.
Feedback from this demonstration/experiment was gathered.
••••

The system worked as expected. The dummy in the ambulance could be monitored visually from theterminal for doctors. The dummy vital signals were available, too.
The medical professionals who gathered at the demonstration (they were attending the annual
conference of Association of Acute Medicine) tried the operation.
The general feedback is the appreciation of the ease of use and the inexpensive price of the hardware
devices (good for small city offices with small budget).
There were questions for advanced features such as the guidance for ambulance to reach lesscrowded hospital with suitable doctors, etc.

H2020 EUJ-02-2016

CPaaS.io

Page 7 of 25

Deliverable D2.11

Feedback and Validation Report (V2)

So the system’s operation and purpose were understood by the medical professionals on the spot. We
hope to extend the operation of Yokosuka Medical Care to nearby cities beyond Miura and Yokosuka in
Kanagawa prefecture and other cities in other parts of Japan.

3.2 CPaaS.io integration
Figure 2 shows how the Yokosuka Emergency Medical Care application has been integrated with the u2based CPaaS.io platform. Ambulances in this use case are identified by ucode.
Video transmission function in this use case enables doctors in hospital to see patients’ images or monitors
in real-time. Doctors who can see the images in ambulances are controlled by access control function in
IoT Aggregator.

Figure 2: Integration of the Yokosuka Emergency Medical Care into the CPaaS.io platform. CPaaS.io components used arehighlighted in red.

3.3 Performance and stability
As noted in section 3.1, during the CPaaS.io project, the early prototype has become a daily used system
in Yokosuka city It has taken root in the city's emergency medical care routine, due to its stableperformance. The system also proved to be useful and brought about improved performance, mostly easeof use, usefulness of the transmitted data, at an inexpensive price. In the hope of reaching wider medical
community to promote the ease of use, its inexpensive price tag and its effectiveness, the video
transmission feature of the Yokosuka Medical Care system was demonstrated at an academic conference
at UTokyo and received good praise in line with the earlier favorable comments from Yokosuka's
firefighters' department.

H2020 EUJ-02-2016

CPaaS.io

Page 8 of 25

Deliverable D2.11

Feedback and Validation Report (V2)The system has proved to be popular with the medical crew as well due to its of use and the less bulkier
hardware devices than before. Thus, the adjacent Miura city has decided to use the system as well. Currently
the Yokosuka Emergency Medical Care system including the video transmission is used by both cities. The
city government is happy about its relatively inexpensive price tag.

3.4 Validation report
ID
YEM-1

Description
Data collection

YEM-2

Consensus forusing medical andpersonal data bycity government
Privacy protectionof patients

YEM-3

YEM-4

Low-effort for
emergency
medical teamstaff members

YEM-5

User feedback
collection
Metadata
management

YEM-6

YEM-11

YEM-14

Efficient query ofpatient, hospital
bed and doctor
data
Data sharing

YEM-17

Communication

H2020 EUJ-02-2016

Fit Criterion
Existence ofautomaticcollection
systemAgreementwith Yokosuka
city
government
Existence ofprivacy dataprotection
mechanism
Existence of
low-effort
mechanismand evaluation
score by theteam staff
members
Number ofuser feedbacks
Method forcollecting
metadata andenriching
sensor data
Query latency
in sec.

ReportVideo transmission function and ambulance
location shared function is working well.

Availability ofdata sharing
mechanisms
Number ofdata points
ingested per
second,processing
latency,storagefootprint

Yes, data collected in the ambulance is shared andhospital bed and doctor availability is shared now.

Yes, an agreement has been signed.

Yes, proper security protection mechanism basedon existing network security practices is
implemented.
Several training sessions and user feedback haveconfirmed that the system is easy to use. It is in
DAILY use in Yokosuka and Miura.

Yes, several user hearing sessions were held
Vital data from the ambulance have beentransmitted to the hospitals. To this extent, this has
been archived.

Yes, it has been achieved and thus the propertransfer of patient to the right hospital is done now.

The application collects medical data of the patient
inside the ambulance and share this via
communication to doctors at various hospitals. The
current collection and transmission of data are
deemed sufficient by doctors of participating
hospitals.

CPaaS.io

Page 9 of 25

Deliverable D2.11

Feedback and Validation Report (V2)4 Event Management – Enhanced User Experience
In this section we describe how we are validating the results of the Enhanced User Experience Application.

4.1 Feedback
In the previous validation report (D2.6) we described the feedback we have collected from event
participants, company internal workshops as well as feedback from city representatives and feedback fromdomain experts. In the second feedback round we have primarily focused on feedback from productmanagers and domain experts.
The intention to obtain feedback from product managers was to receive comments on the commercial
viability. The feedback sessions were conducted with AGT product managers that are working on relatedcommercial offerings, i.e. IoT analytics solutions related to Social IoT and enhanced event experiences. In
a nutshell we received the following feedback:1. Highly relevant analytics have been applied and the application concepts was considered to be ofhigh potential value for commercial exploitation
2. The fact that the application builds on a platform that implements the NGSI standard has been
considered as an important benefit for better integrating with third parties
3. In light of the GDPR and the need to store and process data, the security and privacy enablers usedby the application has been considered as a key technology to secure and control access topersonal data managed by the application
In addition, we consulted with domain experts, i.e., lead data scientists that have developed analytics forenhancing event experiences on larger scales. We wanted to learn whether the analytics developed could
work in a real setting. The result of these consultations was that subject to further data collection andtuning of the metrics and algorithms developed in the application, it should be feasible to realize theapplication to reliably detect relevant moments when visiting events.

4.2 CPaaS.io Integration
Figure 3 shows how the EUE application has been integrated with the CPaaS.io platform. Note that thearchitecture and the use of its components has not changed since the first validation report (D2.6).
However, the implementation of the application has progressed (most notably covering the Sapporo Snow
Festival as an additional event), and more functionality has been pushed down from the application to theplatform (see validation report below). Here we briefly repeat the description of the architecture.
In the resource layer, sensor data from events such as the Color Run Utrecht is collected via AGT’s IoT
analytics platform and inserted into the IoTBroker via the IoTAgent. Via the NGSI to RDF mapper, data
inserted in the IoTBroker is converted to RDF data that is managed by the IoT Knowledge Server. Examples
include aggregated Twitter sentiments. RDF data can be accessed in the application via a SPARQL interface.
To this end, personal data is accessed using the FIWARE components directly. This way we can benefit fromthe security and privacy components that provide access control and encrypted storage of the data in the
CPaaS.io platform. In addition, the application uses the deployment tool that provides information about
how sensor devices are deployed. For instance, it contains information about which wearables have beenworn by which individual. This information is also accessible via a SPARQL endpoint.

H2020 EUJ-02-2016

CPaaS.io

Page 10 of 25

Deliverable D2.11

Feedback and Validation Report (V2)EU FIWARE-based Platform Instance
SMART CITY Services Layer

SPARQL App
Developer

FIWARE App
Developer

Security &Privacy Pillar

IdM API

KeyRock

Knowledge Layer
SPARQLKAT Toolkit

Platform
Management ,Federation andOperation
Pillar

XACML
XACML

Semantic Data &Integration Layer

IoT Knowledge
Server
Deployment
Tool

NGSI to RDF
mapper

STH Comet
NGSI10KnowledgeServer API

Capability
API

NGSI10NGSI9NGSI10

PEP-Proxy

Virtual Entity Layer

FogFlowNGSI9

IoT Discovery

Capability
ManagerContext Broker

NGSI9

Encryption
API
NGSI10NGSI10CP-ABE

IoT Data &Ingestion Layer
NGSI10IoT Broker
NGSI10SPARQL

Existing components
New/enhanced
components

IoT Resource Layer

SPARQL
Agent
Device API

IoT Agent

LoRaWANNGSIIoTA to to
NGSI
bridgeBridgeDevice

LoRa Device

Figure 3: Integration of the Enhanced User Experience Application into the CPaaS.io platform. CPaaS.io components used arehighlighted in red.

4.3 Performance and stability
The MyEvents application is continuously running using the CPaaS.io components as described above
supporting metrics and data for both the Color Run Utrecht 2016 and Sapporo Snow Festival 2018 event.
On average the application is checked once per day and we are not aware of any downtimes.
We also conducted rudimentary performance tests to test the performance of the IoT Broker component.
A script was written which randomly pushes user data to the IoT Broker on a per-user basis. Below we show
a simple example of such randomly generated message:{"contextElement": {"entityId": {"id": "agtres:Jon","type": "foaf:Person","isPattern": false},"attributeDomainName": null,"domainMetadata": [],"attributes": [{"name": "agtres:hasActivity","type": "xsd:string","contextValue": "DANCING","metadata": [{"name": "Timestamp",
H2020 EUJ-02-2016

CPaaS.io

Page 11 of 25

Deliverable D2.11

Feedback and Validation Report (V2)

"type": "agtres:timestamp","value": "2017.09.13 11:38:54:120 +0100"},{"name": "ConfidenceValue","type": "xsd:number","value": 0.93

}

]
}
]
}

Each run was carried out for 5 minutes with a delay of 2 seconds every time a record was inserted to the
IoT Broker. The runs were repeated multiple times. Table 1 shows the averaged results of multiple
conducted runs. In each run 150 entities were injected. We measured the average insertion time dependingon the number of users. We can see that there is some minimal increase in the insertion times. Please notethat this are only rudimentory experiments and larger performance tests need to be conducted for further
evaluation and discussion of theses results.
Table 1: Measurement results

Number of users
1
10
100
150

Avg time to inject one attr/value(ms)
54
61
96
153

4.4 Validation Report
In D2.6 we have already fully validated the requirements and shown how the CPaaS.io delivers thecorresponding application functionalities. In this section we show the architectural evolution of theEnhanced User Experience Application and show an increasing amount of CPaaS.io components are usedby the application.
Table 2: CPaaS.io components used by the EUE application listed by application version.

Prototype
Version
0.1

Used Components# Components

Documentation

None

0

1.0

Deployment Knowledge Base,
Deployment Tool
Deployment Knowledge Base,
Deployment Tool, IoT Broker, KAT
Toolkit (for Sentiment Analysis)
Deployment Knowledge Base,
Deployment Tool, IoT Broker, KAT
Toolkit (for Sentiment Analysis), NGSI
to RDF Mapper, IoT Knowledge
Server/Semantic Data Repository
Deployment Knowledge Base,
Deployment Tool, IoT Broker, KAT
Toolkit NGSI to RDF Mapper, IoT
Knowledge Server, Cost Fed, Security
and Privacy Pillar, ucR Manager

2

D2.1 Requirements Specification(V1)
D2.2 First Prototype Releases

4

D2.4 Second Prototype Releases

6

D2.8 Third Prototype Release9

D2.9 Validation of federation andtransferability, D2.10 Final
Prototype release

2.0

3.0

4.0

H2020 EUJ-02-2016

CPaaS.io

Page 12 of 25

Deliverable D2.11

Feedback and Validation Report (V2)

As can be seen by the table above, overall we developed five versions of the application. The initial version(0.1) was built according to the initial requirements independently from the CPaaS.io platform in order tounderstand the application domain and develop the UI with initial data and analytical capabilities. In eachversion we integrated more of the CPaaS.io components as show in the table. In the final version, theapplication relies on 9 CPaaS.io components.

5 Event Management – Tokyo Management of Service Vehicle
5.1 Feedback
The Tokyo Management of Service Vehicle use case had its field test during April 2018 to June 2018. With
its stability and ease of use, the delivery management service has been adopted widely in the partnercompany.
••••

The developed platform will enable route management of delivery and monitoring.
The use case is truck delivery management in the Tokyo metropolitan area delivery included garbage
packers tracking and bird-hazard prevention.
It supported 3000 delivery/collection transactions per day during the field test. It gathered truck bed
photo for each delivery/collection transaction. In addition, it collects the location data of each truck
for second-scale short intervals. These location data and photo data are high-volume requirementsof a smart city platform.
After the field test, the service is widely used in the partner company. Only a minor portion of delivery
is excluded from the service adoption (due to specific business contexts).

During the field test, it was observed that the engagement of drivers to make sure the proper truck bed
photo was low. From the sample of 4838 instances, only 781 photos were valid truck bed photos. Using
the data from the field test, a supervised learning analytics was performed, aiming at the automatic
identification of truck bed photos.
The result of the test is depicted in Table 3.
Table 3: Results of Tokyo Management of Service Vehicules use case

Precision

Recall

F-Measure0.84079

0.772930.75643

The application is not deployed productively as of yet, but it provides a feasible solution to improve thedriver engagement of taking valid photo.

5.2 CPaaS.io integration
Figure 6 shows the components of the u2-based CPaaS.io architecture that are being used for the
implementation of this use case.

H2020 EUJ-02-2016

CPaaS.io

Page 13 of 25

Deliverable D2.11

Feedback and Validation Report (V2)Figure 4: CPaaS.io integration

5.3 Performance and stability
The performance and stability of Tokyo Management of Service Vehicle was observed during April 2018
and June 2018. The system occupied a major portion of storage usage because it is a very data-intensive
application. The storage monitoring display is depicted in Figure 5. The green part shows the data for theservice on the CPaaS.io platform.

Figure 5: Data storage monitoring

The availability ratio during the field test period is depicted in Table 4. There was a database connectivity
incident in May 2018. Data retransmission was performed at the application level and no data was lost,
therefore it did not impact the quality of service.

H2020 EUJ-02-2016

CPaaS.io

Page 14 of 25

Deliverable D2.11

Feedback and Validation Report (V2)

Table 4: Availability Ratio

Month
April 2018
May 2018
June 2018

Availability Ratio(%)
100.0099.97
100.00The service is very data-intensive application. It is estimated that its cloud base installation enables 10
times more data as long as the adequate storage is added.
It shows the validity of the CPaaS.io platform for data-intensive smart city applications.

5.4 Validation report
ID

TMS-3

Description
Live Data
collection anddistribution

Fit Criterion
Existence ofautomaticcollection systemand distribution
system of live
operation data.
Existence of
cryptographic
communication
mechanism andfault tolerance
mechanism.
Visual data accessfor users

TMS-5

Secure androbustcommunication

TMS-11Visualization ofdata

TMS-12

Data processing

Availability ofanalytical services

TMS-18

Data sharing

Availability of data
sharing mechanism

ReportYes. The use case deployed a data-intensive service
with location data and truck bed photo data. It covers
monthly sub-TB-scale data-intensive transactions.

Yes. The QoS of 99.97% and beyond availability was
obtained.
Fault tolerance with data transmission was verified. It
can accommodate data-intensive application such as
TB per month data collection.
Yes. Realtime monitoring of delivery trucks wereensured. It provides high level usability for truck
drivers and delivery plan managers.
Yes. Deep-learning techniques are applied to thetruck bed photos. It provides a promising 75% Fmeasure from supervised learning data sets.
Yes. Photo and location data were successfully sharedon a cloud.

6 Tokyo Public Transportation
6.1 Feedback
The Tokyo Public Transportation application is part of a set of applications that use the CPaaS.io platform
to manage public transportation information. This sub-application focuses on integrating data fromdifferent public transportation services. In order to promote the open data approach in the public
transportation sector in the greater Tokyo metropolitan area, the Association for Open Data of Public
Transportation (AODPT) had been formed with the participation of public transportation operators, ICT
companies, government offices as observers, etc. in 2015 before CPaaS.io started. The chair of AODPT isProf. Ken Sakamura (Yokosuka Telecom Research Park, Inc. (YRP), the CPaaS.io Japanese coordinator).
With the emerging CPaaS.io infrastructure based on the u2 architecture in Japan, AODPT held a hackathon
called “Open Data Challenge for Public Transportation in Tokyo” with the support of CPaaS.io partners
including YRP, Microsoft Japan Co., Ltd. (MSJ) as the members of AODPT, and UoT as the co-host. The
H2020 EUJ-02-2016

CPaaS.io

Page 15 of 25

Deliverable D2.11

Feedback and Validation Report (V2)contest was held from December 7, 2017 to March 15, 2018.2 This open hackathon provided open datafrom 22 public transportation operators using the CPaaS.io-based application framework. During thecontest period, approximately 800 registrations including ones from outside Japan were recorded. By thedeadline, the hackathon received almost 100 submissions: in other words, 100 applications. The winningapps were announced and CPaaS.io logo was prominently shown in the result page.
With the successful operation and wide media coverage of the first hackathon, “The second Open Data
Challenge for Public Transportation in Tokyo” is already under way with the support of YRP and MSJ as themembers of AODPT and UoT as a co-host. The contest period is from July 17, 2018 to January 15, 2019. 3
A third contest is in the planning.

6.2 CPaaS.io integration

Figure 6: u2-based Architecture for Tokyo Public Transportation: "Open Data Challenge for Public Transportation inTokyo"

2 cf. also https://tokyochallenge.odpt.org/2017/en/index.html
3 cf. https://tokyochallenge.odpt.org/en/index.html

H2020 EUJ-02-2016

CPaaS.io

Page 16 of 25

Deliverable D2.11

Feedback and Validation Report (V2)Figure 7: High-level application architecture of Tokyo Public Transportation (from D2.1)

Based on the correspondence of the high-level application architecture of “Open Data Challenge of Public
Transportation in Tokyo,” we can explain the validation as follows. Data collection infrastructure is in thelower layer of the architecture (Open Data Distribution Platform). Meta-data is handled in Semantic Dataand Integration Layer. In this hackathon, visualization is done by the applications themselves. That is, theapplications by the contest participants add to the Smart City Services Layer.

6.3 Performance and stability
Stability and performance are designed on the u2-based CPaaS.io architecture. Access to the open data
sources via open APIs, documented on the developer site, was available over the course of contest period
and could support more than 800 registrants, i.e., developers, and additionally the users of the released
apps. The system was as robust as the underlying cloud and proved very reliable.

6.4 Validation chart
In the following table, we list the status of validated requirements as described in deliverable D2.5. Please
note that we only report on the requirements that we marked for validation and that do not relate tofederation.

ID
TPT-2

Description
Multi-lingual data

Fit Criterion
Ratio of multilingual
information in all
information.
Existence of automatictranslation system.

TPT-3

Live data collection anddistribution

Existence of automaticcollection system anddistribution system of live
operation data.

H2020 EUJ-02-2016

CPaaS.io

Validation Report
In the “Open Data Challenge for
Public Transportation in Tokyo,”some apps used the automatictranslation features. But wecould not force all the operators
to do so.
The 22 operators offered thelive, up-to-date data and suchdata was made available via thehigh-level applicationarchitecture to developers andapps.
Page 17 of 25

Deliverable D2.11

Feedback and Validation Report (V2)

TPT-4

Data format and API
standardization(Metadata + API)

Existence of standard data
format and standard API
specifications.

Yes, we have the standard data
format and open API
documentation available at thedeveloper site, and the data
sources were preparedaccording to the specification.
The data, not compliant with thedata format, were converted into
the compliant form in u2-based
CPaaS.io instance to someextent.

TPT-5

Secure and robust
communication

Commonly available
cryptographic communication is
used.

TPT-6

Developers' Site

Existence of cryptographic
communication mechanismand fault tolerance
mechanism.
Existence of developers' siteof Tokyo Public
Transportation Open Data.

TPT-7

Data License

Existence of open data
licenseTPT-8

Third party apps

TPT-15

Efficient query oftransportation open data

Number of third party apps
or software.
Query latency in sec.

TPT-18

Data sharing

TPT-19Semantic data integration

Availability of data sharing
mechanisms
Ontologies for data to besharedDeveloper site was created forthe contest specifically aside
from the general developer site(https://developer.odpt.org/). This
was accessed by theapproximately 800 registrants toobtain the specification of data
format and API.
Data license was created for thecontest and the data providers
and contest participants abided
by it.
About 100 apps were submitted
during the 1st contest.
The data from the public
transportation operators were
provided in real-time including
the up-to-date operation status
and the location of trains andbuses, etc.
The data from 22 transportation
operators was shared.
The ontology for transportation
was published and used forformatting the data from 22transportation operators.

7 Smart Parking – Murcia
Besides the envisioned use cases, we have included a new one regarding a common problem that every
big city has to deal with day by day, the problem of traffic congestion. It is also means as an opportunity
to us by developing a new flexible framework able to handle complex processing tasks splitting them intoeasier to handle ones and distributing them into a shared cloud-edge environment to the different layers.
Murcia is a medium-size city in the south east of Spain with a population of 450.000 residents. This number
increases by about 5.000 residents per year, with a noteworthy distribution of people among the city
H2020 EUJ-02-2016

CPaaS.io

Page 18 of 25

Deliverable D2.11

Feedback and Validation Report (V2)

districts. Because of this and the fact that Murcia is the capital of the Murcia Region, a dramatic rise ofaccesses to the city centre has been detected in the last years. Day by day, commuters, tourists and families
traveling by car collapse the core of Murcia, wanting to park at commerce, financial and historical areas.
To improve the situation, the aim of this use case is that of taking advantage of the CPaaS.io platform toprovide a service that tracks the state of the free parking spots to provide the drivers with this information
beforehand, so they can better plan where they will park. This will also lead to a more fluid traffic in thecentre of the city, because there will be less drivers wandering, looking for a parking spot in areas that do
not have any available.
To do so, upon our CPaaS.io platform we have created a new service comprising the tasks presented in thefollowing picture. This service uses the FogFlow framework which provides a mechanism to support
dynamic processing flows over cloud and edges. It composites multiple NGSI-based processing tasks toform high level IoT services, orchestrating and optimizing their deployment within a shared cloud-edge
environment.

Figure 8: Smart Parking Services

Following a bottom-up approach, our Context Broker provides the information about the registered private
parking sites, as well as the daily information about the tickets issued for the regulated parking zones of
the city. Such provision of information is reported with different frequency because of the availability of
the information.
Real-time prediction and estimation tasks process the corresponding information and provide their results
to the Parking Site Recommendation Task. This way, when user of this service (represented as the blackvehicle in Figure 8), establishes the destination and an arrival time estimation, the Parking Site
Recommendation Task will take these elements as inputs, as well as his location and will provide the routeto that specific destination.

7.1 Feedback
This use case will provide specific KPIs like the free parking slots in private parking sites, the estimation offree spots in the regulated parking zones of the city, the amount of time spent by the users of the service
to find a free parking spot and park his car, the most frequent destination points.

H2020 EUJ-02-2016

CPaaS.io

Page 19 of 25

Deliverable D2.11

Feedback and Validation Report (V2)

7.2 CPaaS.io integration
Regarding how Smart Parking is integrated inside the European architecture, Figure 9 shows theaforementioned architecture where the components involved in this use case are highlighted.

Figure 9: Smart Parking components of EU architecture

According to this figure, devices provide information through their corresponding IoT Agent. This
component handles lightweight protocol messages and translates them to an NGSI representation. In casethe information coming from the parking sites and restricted parking zones is integrated at the platform
layer, such integration only involves the IoT Broker component.

Figure 10: Smart Parking flow of information

Following the diagram represented in Figure 10, once the information is distributed along IoTBroker
entities, each FogFlow edge node which have been deployed as virtualized services at edge layer (SP-5) is
H2020 EUJ-02-2016

CPaaS.io

Page 20 of 25

Deliverable D2.11

Feedback and Validation Report (V2)able to process its corresponding parking availability information to do the aggregating process. For acorrect operation, having the correct information is fundamental (SP-1, SP-2, SP-4).
This information arrives to the Recommender which makes an analytical process (SP-3) when a request fora destination is issued by a connected car. It will answer a recommendation of the best area and/or private
parking site according to the information provided in the request.
Security has also been considered in this use case by integrating the Identity Management andAuthorization enablers, specifically KeyRock and XACML, Capability Manager and PEP_Proxy. This way, theuser must be registered into the platform.
Firstly, both the real-time estimation and the prediction module access to the information provided by thesensors and stored into the Broker by presenting the capability token (CT, authorization credential). This
access is captured by the PEP_Proxy which validates the CT in order to grant access to the requested
information. Once they calculate their output, these values are also stored into the Broker by using again
a capability token. Eventually, the PEP_Proxy will be the one validating the token before granting access tostore the information again. Likewise, the access to that information from the Recommender module
requires also the interaction with the PEP_Proxy by providing again the capability token.

7.3 Performance and stability
Our development cycle includes peer reviewing and continuous integration. It allows us to detect potential
bugs and deliver a reliable and performing product. We are also having manual tests in our lab to ensure
end-to-end functionality.
As the implementation of this use case also is relying on the TTN LoRa network, the same comments
regarding the reliability of the network apply that were already mentioned in section 2.3. The network is
reliable enough to handle the number of messages in this use case. Also here, the original monolithic
architecture has been changed toward a distributed architecture based on FogFlow,leading to better
reliability as well as transferability.

7.4 Validation report
After developing the prototype, we have updated the following table including two new requirements (SP6and SP7). These new requirements consider security aspects in both the interaction between components,
as well as the access control to the information stored in the broker.
ID
SP-1

Description
Data collection
infrastructure

SP-2

Edge processing

SP-3

Cloud processing

SP-4

Communication

H2020 EUJ-02-2016

Fit Criterion
Existence of a datacollection
infrastructure
Existence ofdevices able tohandle processing
tasks in the edge.
Cloud services able
to processcomplex tasks
Online availabilityof services anddevices

CPaaS.io

ReportReal time information provided by parking sites
as well as daily logs of the expenders of RPZs
Edge nodes responsible for aggregating
information of parking availability and RPZ use.

The Recommender, located at cloud layer, is
responsible for the analytical part of the process
It is fundamental to have reliable and real-time
information regarding availability in order tomake an informed decision regarding the best
parking area

Page 21 of 25

Deliverable D2.11

Feedback and Validation Report (V2)

ID
SP-5

Description
Virtualized
services

Fit Criterion
Support for
virtualized services

SP-6

Securedinteraction
betweencomponents

SP-7

Secured access tothe information

Existence of
security techniquesand enablers
capable of secure
the exchange of
information
betweencomponents.
Existence ofenablers providing
access control tothe stored
information

ReportEdge nodes must support virtualized services
allowing them to dynamically run the specifictasks distributed by the FogFlow framework
It is fundamental to secure the exchange of
information between components in order toprovide reliable information in time.

The access to the information must be controlled
allowing only the legitimate entities (producers or
consumers) to access to it.

8 Sapporo Visitor Experience
8.1 Feedback
The Sapporo Open Data committee is now on service of data service for foreign tourists. Several open data
hackathons had been held in Sapporo. Some of attendees include citizen and private companies that use
open data and location data of Sapporo to build applications. Almost all feedback from the attendees
wwas positive, and many of developers are continuing to develop applications using the data. Many of theparticipants want to continue to use the API of the Sapporo open data service. Sapporo City and the Open
Data Committee will drive the data platform to include more open data and citizen services, so the Sapporo
Tourism Open Data will be a part of the whole Sapporo City service.

8.2 CPaaS.io integration.

H2020 EUJ-02-2016

CPaaS.io

Page 22 of 25

Deliverable D2.11

Feedback and Validation Report (V2)Figure 11: Components of the u2-based architecture used in Sapporo

The Data collection infrastructure is in the lower layer of the architecture (IoT Resource Layer), using various
sources. Efficient data query is handled by the databases in the Open Data Distribution Platform, while thelinking to virtual entities and metadata is handled in the Semantic Data & Integration Layer. Visualization
finally happens finally in the Service Layer, making also use of the Kokosil component.

8.3

Performance and stability

Stability and performance is designed on CPaaS.io architecture and running on Microsoft Azure as a Public
Cloud platform, basic services are guaranteed by the cloud platform itself, and the CPaaS.io architecture
supports appropriate data providing that depends on beacon location data and device profiles.

8.4 Validation report
In the table below, we list the status of validated requirements as described in deliverable D2.5. Please notethat we only report on requirements that we marked for validation and that do not relate for federation.

ID
SVE-1

Description
Data collection
infrastructure

Fit Criterion
Existence of a datacollection
infrastructure

SVE-2

Metadata
management

SVE-3

Visualizationof personal
data andopen data

Method forcollecting metadata
and enriching
sensor data
Visual data accessfor end users

SVE-7

Efficient queryof sightseeing
information

Query latency insec.

Validation Report

The data collection infrastructure is currently realized as
described in the high-level application architecture (seeD2.1) and marker information is also collected to learn
the whereabout of users in underground passages.
The metadata is managed and accessed by the high-levelapplication architecture (see D2.1).
Open data related to tourism and public transportation isincorporated and used for user interaction. In the current
high-level application architecture (See D2.1). Personal
data is not incorporated yet. (being planned for the
future).
In the current implementation, the query latency basedon the high-level application architecture fully satisfies
the visitor experience application requirements. Thisrequirement is extremely important for providing tourism
information as envisioned by the original application.

9 Outlook”For the next review, consider scaling use-cases up, highlight their strongest assets, and consider cross-borderdemonstrations” was the objective set by the reviewers after the first. In this regard, use cases started tomove they inner service to the CPaaS.io platform as much as possible and to scale up. We have in thisdeliverable also clearly outlined which components of the complete CPaaS.io architecture are used by thedifferent use cases.
H2020 EUJ-02-2016

CPaaS.io

Page 23 of 25

Deliverable D2.11

Feedback and Validation Report (V2)

In each use case, the owners listened to the feedback of the stakeholders, being users or organisation andstarted to design or redesign their solution to match expectation and policies. Currently the focus is put
on enhancing the scalability of the solutions and strengthening the value brought to the stakeholders.
With the current form of CPaaS.io, transferability and scalability is made easier by the addition of FogFlow
and the different mapping services.

H2020 EUJ-02-2016

CPaaS.io

Page 24 of 25

Deliverable D2.11

Feedback and Validation Report (V2)

Appendix 1 - Enhanced User Experience, Color run feedback form
Participant details:
Name:
Age:
Gender:
About yourself
1)
2)
3)
4)

What were your thoughts when you first heard about this event?
Why did you decide to join this event?
How would you rate yourself on a fitness scale from 1 to 10 ?
How often do you run (never, occasionally, regularly (once, twice, etc. per week/month …)? Whatdistances do you typically run?
5) What are your favorite colors? List 2 that you like the most and 2 that you don’t like.
The Run6) How did you feel when you reached the start of the color run?
7) How did you feel when you reached the finish of color run?
8) Please give us your energy levels (high , medium , exhausted) over the period of color run. Make
slots of 5 min from start to end and assign energy level and emotions (Happy, disgusted, afraid,
sad etc.) for the same.
9) Between your friends, who was the most active one in the run?
The whole event
10) What do you remember (interesting scene, own sentiment) about the color run? What are yourstrongest memories about it and why?
11) Will you discuss any of your experiences with friends after the event? If yes, what exactly did you
share with them (e.g., run details, color, and venue)? If not why.
12) Did you take any pictures or videos with your phone during event? If yes, did you take pictures of color
stations, other participants? Or did you take any selfies?
13) Would you like to join this event next time? Why? Why not
Miscellaneous14) Was it comfortable to wear all sensors during the run? What was not comfortable?
15) Did you encounter any other issues with the devices of the set up ?
16) Do you have any further comments, suggestions?

H2020 EUJ-02-2016

CPaaS.io

Page 25 of 25