Deliverable D4.1

H2020-EUJ-02-2016
H2020 Grant Agreement Number 723076
NICT Management Number 18302

Deliverable D4.1

Analysis and Selection of Task
Migration Technologies
Version Draft V1.1
October 3rd, 2018

ABSTRACTThis deliverable presents a pragmatic view of a framework developed thanks tothis project with the purpose of dynamically and automatically distribute specificprocessing tasks over different components in a shared edge/cloudenvironment called FogFlow. This document is tightly related to D4.2 where thisframework is thoroughly detailed.
In this deliverable we provide three different use cases for the application of thisframework providing not only a diagram, but also specific requirements, thedistribution of tasks and even the data model in order to successfully deploy the
FogFlow framework.

This work is licensed under the Creative Commons Attribution 4.0 International License.
To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

H2020 EUJ-02-2016

CPaaS.io

Page 1 of 28

Deliverable D4.1

Disclaimer
This document has been produced in the context of the CPaaS.io project which is jointly funded by theEuropean Commission (grant agreement n° 723076) and NICT from Japan (management number 18302).
All information provided in this document is provided "as is" and no guarantee or warranty is given thatthe information is fit for any particular purpose. The user thereof uses the information at its sole risk andliability. For the avoidance of all doubts, the European Commission and NICT have no liability in respectof this document, which is merely representing the view of the project consortium. This document issubject to change without notice.

Document Information
EditorAuthors

Reviewers
Delivery Type
Dissemination
Level
Contractual
Delivery Date
ActualDelivery Date
Keywords

Juan Antonio Martinez Navarro, Antonio Skarmeta (OdinS)
Juan Antonio Martinez Navarro, Antonio Skarmeta (OdinS)
Bin Cheng, Jihed Khiari, EvertonLuis Berz (NEC)Toshihiko Yamakami (ACC)
Klaus Moessner and Fahim KhanR
Public31/01/2018
31/01/2018;
Edge computing, use cases, smart cities, programming model

H2020 EUJ-02-2016

CPaaS.io

Page 2 of 28

Deliverable D4.1

Revision History
Rev.
v0.1
V0.2

Date
09/12/2017
10/01/2018

Description
Initial structure of the deliverable
Smart Parking contribution

V0.3

09/01/2018

V0.4
V0.5
V0.6

15/01/2018
26/01/2018
26/01/2018

Security Requirements for IoT
BrokerOpportunities for EU/Japan Coll
Sections 4.1 and 4.3
Introduction, BackgroundV0.7

26/01/2018

Final contribution

V1.0
V1.1

31/01/2018
01/10/2018

Submitted
Analysis and selection of task
migration

H2020 EUJ-02-2016

CPaaS.io

Contributors
Bin Cheng (NEC)Juan Antonio Martínez, Agustín Cánovas, Antonio
F. Skarmeta (OdinS)
EvertonLuis Berz (NEC)Toshihiko Yamakami (ACC)
Bin Cheng, Jihed Khiari (NEC)Juan Antonio Martínez, Agustín Cánovas, Antonio
F. Skarmeta (OdinS)
Juan Antonio Martínez, Agustín Cánovas, Antonio
F. Skarmeta (OdinS)
Stephan Haller
Bin Cheng (NEC)Page 3 of 28

Deliverable D4.1

Table of Contents
1

2

3

Introduction............................................................................................................................................................................... 6
1.1

Background ...................................................................................................................................................................... 6

1.2

Objectives ......................................................................................................................................................................... 9

Use Cases ................................................................................................................................................................................... 9
2.1

Anomaly detection in shops ...................................................................................................................................... 9

2.2

Smart parking ................................................................................................................................................................ 12

2.3

Anomaly detection for transportation ................................................................................................................. 18

2.4

Opportunities for EU/Japan Collaboration ......................................................................................................... 21

Requirement Analysis .......................................................................................................................................................... 22
3.1

Security requirements for IoT Broker ................................................................................................................... 23

3.2

Analysis and selection of task Migration ............................................................................................................ 25

4

Conclusions & Next Steps ................................................................................................................................................. 27

5

References................................................................................................................................................................................ 28

List of Figures
Figure 1. Comparison of BigData and FogFlow approaches............................................................................................ 7
Figure 2. illustration of a retail-related use case ................................................................................................................ 10
Figure 3. Power consumption of a coffee machine in one hour .................................................................................. 11
Figure 4. Processing flows for anomaly detection use case ........................................................................................... 12
Figure 5: FogFlow entities interaction .................................................................................................................................... 13
Figure 6: Diagram of the tasks for the use case ................................................................................................................. 17
Figure 7. to illustrate the anomaly detection for transportation use case scenario ............................................. 19
Figure 8. Induction loop traffic sensors ................................................................................................................................. 20
Figure 9. Service topology .......................................................................................................................................................... 21
Figure 10. Child search use case scenario ............................................................................................................................. 22
Figure 11. Query data flow. ........................................................................................................................................................ 24
Figure 12. Security enablers interactions on the Query data flow (Operation-based). ....................................... 24
Figure 13. Update data flow (composite scenario)............................................................................................................ 25
Figure 14. Security mechanisms applied to the Update data flow. ............................................................................. 25

List of Tables
Table 1. Summary of requirements per use case ............................................................................................................... 23
Table 2. Requirements on task migration for each use case ......................................................................................... 26

List of Acronyms
Acronym
NGSI
API
BC

Definition
Next Generation Services Interface
Application Programming Interface
Business Case

H2020 EUJ-02-2016

CPaaS.io

Page 4 of 28

Deliverable D4.1

List of Editing Shortcuts
Shortcut
e.g.
etc
feat.
h/w
i.e.
incl.
p/f
resp.
s/w
tbd
vs.
w.r.t.

Definition
exempli gratia – for example
et ceterafeaturinghardwareid est – that isincludingplatform
respectively
softwareto be discussed/decided
versuswith respect to

H2020 EUJ-02-2016

CPaaS.io

Page 5 of 28

Deliverable D4.1

1 Introduction
This deliverable is tightly related to Deliverable D4.2 [3], since with both documents we provide acomplete view of FogFlow [4]. A framework developed within the scope of this project with the purposeof distributing a dynamically and automatically composite multiple data processing tasks over a sharedcloud-edge environment providing a more scalable approach to fulfil this kind of complex tasks bysplitting them into more straightforward ones that can be easily handled at the edge layer.
While deliverable D4.2 focuses on the technical details, models and how the orchestration is performed,
in D4.1 we highlight the pragmatic application of this framework over different use cases such as:
anomaly detection in shops, smart parking and anomaly detection applied to transportation.
So, after introducing the reader into the needed background related to edge/fog computing and smart
city platforms, the bulk of this deliverable is reached where the use cases already mentioned are detailed.
This section contains specific and concrete details for the specific components involved, the defined data
model and the delegation and orchestration of the different tasks that are distributed over the edge andcloud plane needed to achieve each use case.

1.1 Background
The purpose of this section, as commented above is that of providing the reader with the neededbackground related to the framework proposed in this deliverable. To do so, the following two
subsections are presented.
1.1.1 Edge/fog computing
Nowadays cities are becoming more and more digitalized and connected as numerous sensors havebeen widely deployed for various purposes. For example, deployments in smart cities include CO2
sensors for measuring air pollution, vibration sensors for monitoring bridges, and cameras for watchingout potential crimes. Those connected devices form a large scale of IoT system with geographically
distributed endpoints, which generate a huge volume of data streams over time. Potentially, thegenerated data can help us increase the efficiency of our city management in various domains such astransportation, safety, and environment monitoring (e.g., garbage management). However, to utilize thedata efficiently we need to have a flexible IoT platform that allows developers to easily program various
services on top of a shared and geo-distributed smart city IoT infrastructure.

H2020 EUJ-02-2016

CPaaS.io

Page 6 of 28

Deliverable D4.1

Figure 1. Comparison of BigData and FogFlow approaches

Fog computing perfectly fits the geo-distributed nature of smart city infrastructure where the acquired
information can be processed and aggregated at different levels until it reaches the cloud to perform themost complex tasks. Nevertheless, it is still challenging for smart city IoT platforms to adapt to this newcomputing paradigm. The heterogeneity, openness, and geo-distribution of the new cloud-edge
environment raise much more complexity on the management of data and processing tasks than thecentralized cluster or cloud environments. Therefore, we need a sufficient and flexible programming
model with open interfaces that allow developers to implement various IoT services on top of the cloudedge environment without dealing such complexities.
1.1.2 State of the art
In the past, most of the existing city IoT platforms were designed considering that the servers werelocated on cloud, such as CityPulse1 and City Data and Analytics Platform (CiDAP). However, this is nolonger a sustainable and economical model for the next generation of IoT smart city platforms, giventhat many city services (e.g., car accident detection) require ultra-low latency and fast response time.
Moreover, bandwidth and storage costs can be substantially high if we send all sensor data such as video
frames to the cloud. Recently there is a new trend to offload more computation from the cloud anddevice layer to the middle layer components which are IoT gateways and edge/core networks, callededge/fog computing.
The current state of art on edge computing, such as Foglets [6], has been mainly focused on how to
optimize task deployment over distributed edges in terms of saving bandwidth and reducing latency.
However, there is not much work that has to be done to explore the programming model for fog
computing. The existing studies either just reuse the programming models from existing frameworks(e.g., Apache Storm and Spark) or come up with their own programming models with non-standardized
interfaces. Both solutions are not suitable for smart city IoT platforms to adopt edge computing in termsof openness, interoperability, and programmability. Therefore, it is strategically important to take astandard-based approach to the design and implementation of cloud-edge computing frameworks for1(2017) CityPulse. [Online]. Available: http://www.ict-citypulse.eu

H2020 EUJ-02-2016

CPaaS.io

Page 7 of 28

Deliverable D4.1

smart cities. In this case, it is easy to achieve the consistency of programming interfaces between clouds(public clouds and/or private clouds) and various edges (small data centers, IoT gateways, andconstrained devices) and also across many city domains.
We also investigated the state of the art on task migration in the context of edge computing. The results
are summarized from the following perspectives.
- Task migration cross physical sites: a cloud-edge environment typically consists of a large number ofedge nodes across different locations or different systems, which can be considered as physical sites.
One way to look at different task migration mechanisms is to check how task migration is supported
across physical sites. Most of existing task migration studies have been mainly focused on the migration
between centralized cloud(s) and geo-distributed edge nodes. For example, Azure IoT [7] edge andAmazon Greengrass [8] are the most popular edge computing platforms available in the market. Both
only support task migration between their own clouds and IoT edge gateways that can be deployed byany user at their own places, such as shops, factories, streets, or offices. The edge-to-edge task migration
at the entire system scope is not supported yet. However, in terms of smart city use case, it is highly
important to enable the edge-to-edge task migration, for instances, the smart parking use case we
propose in this project. On the other hand, in terms of task migration across different systems or
management domains, it is also not explored by the state of the art how task migration could be
supported due to the lack of standardized data model and service interface. Since NGSI is a widely
supported standard for both data model and service interface in the smart city area, we think enabling
cloud-edge computing based on NGSI cloud provide us a unique opportunity to explore task migration
across federated systems/domains, especially in the FIWARE community.
- Task migration at various system layers: task migration could be done at different layers. For
example, Cloudlet [9] proposes to instantiate a task within a virtual machine. In this case, in order tomigrate a running task, we have to move instantiated VMs, which is usually much heavier than moving arunning docker container in terms of the required delay to finish the migration action. Recently, more
and more edge computing frameworks are using docker containers to migrate tasks. For example, Azure
IoT edge is now using docker container as the underlying virtualization technology to spawn tasks.
Nowadays, as the serverless computing model [10] is becoming a more light-weight and transparent
mean to deploy tasks in the cloud, tasks can be migrated even at the function level. In this case, rather
than moving docker containers, we can just move the function code, but still being able to share thesame docker container. This can be done as long as the task only relies on some available docker image
basis. As compared to VM-based task migration, function-based task migration is much faster andintroduces only low system overhead. In some cases the edge node has no virtualization environment toexecute dynamic application modules, task migration has to be supported by executing binary code. Thisapproach is very limited, usually for constrained devices like small sensor nodes with limited battery
lifetime.
- Migration of stateful and sateless tasks: a task is provided by application developers. For a generictask, it might have its own internal states. For this type of stateful task, we might have to recover its
internal state during the migration process. For example, for many server applications like vEPC (evolved
packet core), they have some internal state to keep track of user sessions. It is much more challenging tomigrate those stateful tasks than a stateless task, because we have to deal with the recovery of theirinternal states and keep them consistent during the migration process.
1.1.3 High-level challenges addressed
The main challenges raised by smart city platforms are the following ones.
H2020 EUJ-02-2016

CPaaS.io

Page 8 of 28

Deliverable D4.1

On the one hand the heterogeneity and amount of the stored information is a great challenge. A city canprovide information of different kind, such as the state of its parks and gardens, their lighting system,
information about energy consumption of their facilities or administrative information just to name afew.
How to develop services and applications that must deal with that information with ultra-low latency andfast response time is also another challenge which is specifically the one that our FogFlow framework
tackles. At least two different strategies can be applied: big data approach and FogFlow approach.
The first one operates as follows. After storing all the information in a specific repository of information, acomplex processing task must be applied to that information, requiring also high-performancecomputers to obtain results in reasonable time. However, this method is very far from real-time.
Nevertheless, as Figure 1 presents, the organization of the city fits better the second approach where thewhole processing task is split and distributed into a shared edge/cloud environment. This way, thecomplexity of the task is reduced, and a more flexible approach is achieved.

1.2 Objectives
The main objectives of this WP4 can be summarized in the following paragraphs:Standard-based programming model for fog computing: Extend dataflow programming modelwith declarative hints based on the widely used standard NGSI, which leads to two benefits forservice developers: 1) fast and easy development of fog computing applications, this is because
the proposed hints hide lots of task configuration and deployment complexity from service
developers; 2) good openness and interoperability for information sharing and data source
integration, this is because NGSI is a standardized open data model and API and it has been
widely adopted by more than 30 cities all over the world.
Scalable context management: to overcome the limit of centralized context management, we
introduce a distributed context management approach and our measurement results show that
we can achieve much better performance than existing solutions in terms of throughput,
response time, and scalability.

2 Use Cases
2.1 Anomaly detection in shops
This use case aims at demonstrating detection of abnormal energy consumption in real time using edge
computing in the context of retail shops. As illustrated in Figure 2, a retail company has a large numberof shops distributed at different places geographically. For each shop, a Raspberry Pi device is installed inorder to monitor the power consumption from all power panels in the shop. Once an abnormal powerusage is detected on the raspberry Pi, the alarm in the shop will be triggered to inform the shop owner;in addition, the detected event is reported to the Cloud for information aggregation. The aggregatedinformation is presented to the system operator via a dashboard service.

H2020 EUJ-02-2016

CPaaS.io

Page 9 of 28

Deliverable D4.1

Figure 2. illustration of a retail-related use caseRequirements for this specific use case
1. Data source
For this proposed use case, we do not have the electricity consumption data directly collected from a
retail shop. In order to analyse its technical requirements, we take into account the following open data
set ACS-F2 [5], published by WATT ICT2. This data set includes the measurements of the electricalconsumption at low frequency, typically every 10 seconds, for two acquisition sessions of one hour. The
data were collected from 225 home appliances in the following 15 categories:
















Fridges & freezers,
TVs (LCD),
Hi-Fi systems (with CD players),
Laptops,Computer stations (with monitors),Compact fluorescent lamps (CFL),
Microwaves,
Coffee machines,
Mobile phones (via battery charger),Printers,
Fans,
Shavers,
Monitors,
Incandescent Lamps,
Kettles.

The electricity consumption was measured in terms of:




2

real power (W)reactive power (var)
RMS current (A)

http://www.wattict.com/web/index.php/databases/acs-f2

H2020 EUJ-02-2016

CPaaS.io

Page 10 of 28

Deliverable D4.1


frequency (Hz)RMS voltage (V)phase of voltage relative to current (φ)

The following Figure 3 shows the sample data for one hour, which is a time serial data generated by acoffee machine every seconds.

Figure 3. Power consumption of a coffee machine in one hour

2. Required data processing logic
The processing topology is defined to meet the requirement of Use case 1: anomaly detection ofenergy consumption in retail shops (please check the detail in Section 1.2). Two processing
components are defined as follows, as illustrated by the topology example in Figure 4.
Anomaly Detector: the purpose of this processing component is to detect anomaly events based on thecollected data from power panels in a retail shop. It has two types of inputs: (1) detection rules, provided
and updated by the operator. This input streamType is associated with “broadcast”, meaning that therules are needed by all task instances of this processing component; (2) sensor data from power panel.
The input streamType is “PowerPanel”: the attributes for data entity are provided by the developer but
the attributes for profile entity can be provided by the worker during run-time automatically.
Counter: this processing component counts the total number of anomaly events for all shops in eachcity. Therefore, its task granularity is “per city”. Its input streamType is the output streamType of theprocessing component Anomaly Detector.

H2020 EUJ-02-2016

CPaaS.io

Page 11 of 28

Deliverable D4.1

Figure 4. Processing flows for anomaly detection use case

3. Using the generated results
There are two types of result consumers:(1) A dashboard service in the cloud, which will subscribe to the final aggregation results generated
by the counter component for the global scope;(2) The alarm in each shop, which will subscribe to the anomaly event generated by the Anomaly
Detector task on the local edge node for the local shop.
4. Expected benefits
The following benefits are expected various customers in this use case:Reduced bandwidth consumption: since the measurement points are constantly generated by
all devices over time, the edge computing infrastructure provider expects to reduce thebandwidth consumption used between Cloud and edges;
Fast notification time: the shop owner expects any abnormal usage of electricity in the shop to
be detected in real-time and would like to be informed as soon as possible;
Real-time view of all shops: the retailer, who has multiple shops located at different places,
expects to have a real-time view of all shops, including the total electricity consumption and thetotal number of detected anomaly events;

Flexibility: the retailer would like to manage shops with slightly different strategies, for example,
changing the detection algorithms and/or configuring different shops with different detection rules.

2.2 Smart parking
Murcia is a medium-size city in the south east of Spain with a population of 450.000 residents. This
number increases by about 5.000 residents per year, with a noteworthy distribution of people among thecity districts. Because of this and the fact that Murcia is the capital of the Murcia Region, it has been
detected a dramatic rise of accesses to the city centre in the last years. Day by day, commuters, tourists
and families traveling by car collapse the core of Murcia, wanting to park at commerce, financial andhistorical areas.
H2020 EUJ-02-2016

CPaaS.io

Page 12 of 28

Deliverable D4.1

To improve the situation, the aim of this use case is that of taking advantage of the CPaaS.IO platform toprovide a service that tracks the state of the parking spots to provide the drivers with this information
beforehand, so they can better plan where they will park. This will also lead to a more fluid traffic in thecentre of the city, because there will be less drivers wandering, looking for a parking spot in areas that do
not have any available.

Figure 5: FogFlow entities interaction

As shown in Figure 5, by using the FogFlow framework, the FogFlow Orchestrator will decide, according
to the requirements of the application, the different tasks to be done as well as the edge/cloudcomponents responsible for them.
Once such instantiation is done, and according to the diagram, both parking sites and regulated parking
zones provide the information about parking slots (FogFlow-edges). The FogFlow Context Management
System receives this information and make it available to the Parking Site Recommendation (FogFlowcloud). Finally, when a vehicle uses this service to obtain an estimation of the most appropriate location
for finding a free parking slot. Such information will be provided by the FogFlow Context Management
System, thanks to the Parking Site Recommendation Task.

2.2.1 Requirements for this specific use case
2.2.1.1 Available context data
From our current platform and focusing on the application of smart parking, we provide two different
datasets of information: ones corresponding to parking slots of private parking sites in Murcia city, andthe other corresponding to information provided by the expenders machines of the regulated parking
zones in the city.
Parking slots of private parking sites
H2020 EUJ-02-2016

CPaaS.io

Page 13 of 28

Deliverable D4.1

This information is updated each 5 minutes regarding 5 parking sites of the city centre:-

Morales Meseguer-

La Vega-

Alfonso X-

Centrofama-

Libertad

The following code is an example of querying such information using NGSI.
#!/bin/bash(curl http://fiware-dev.inf.um.es:1026/v1/queryContext -s -S --header 'Content-Type: application/json' -header 'Accept: application/json' --header 'fiware-service: aparcamiento' --header 'fiware-servicepath:/murcia/#'\-d @- | python -mjson.tool) <<EOF{"entities": [{"type": "Sensor","isPattern": "true","id": "Aparcamiento:105"
}
]
}
EOF

The result of this query is the following:{"contextResponses": [{"contextElement": {"attributes": [{"name": "forzado","type": "number","value": "0"},{"metadatas": [{"name": "location","type": "string","value": "WGS84"}],"name": "geoposicion","type": "coords","value": "37.9942698,-1.1294058"},{"name": "libres","type": "number","value": "113"},{"name": "nombre","type": "number","value": "Morales Meseguer"},{"name": "plazas%20libres","type": "number","value": "118"},{"name": "plazas%20totales","type": "number","value": "220"},{H2020 EUJ-02-2016

CPaaS.io

Page 14 of 28

Deliverable D4.1

"name": "totales","type": "number","value": "220"}],"id": "Aparcamiento:105","isPattern": "false","type": "Sensor"},"statusCode": {"code": "200","reasonPhrase": "OK"}}]}

From the previous information we take the total amount of slots of the parking site (totals) and the freeparking slots (libres).
Regulated Parking Zones
Since the technology of the expenders provide their information to the system during the last hours ofthe day. Such information is updated day by day, counting always with the information of the day before.
#!/bin/bash(curl http://fiware-dev.inf.um.es:1026/v1/queryContext -s -S --header 'Content-Type: application/json'
header 'Accept: application/json' --header 'fiware-service: ora' --header 'fiware-servicepath:/murcia/#'\-d @- | python -mjson.tool) <<EOF{"entities": [{"type": "Venta","isPattern": "false","id": "Ticket: 443994176"
}
]
}
EOF

--

The result of such request is the following"contextElement": {"attributes": [{"name": "emision","type": "Date","value":},{"name": "entra","type": "Date","value":},{"name": "expendedor","type": "Number","value": null},{"name": "importe","type": "Number","value": null},{"name": "pago","type": "String",

H2020 EUJ-02-2016

CPaaS.io

Page 15 of 28

Deliverable D4.1

"value": null},{"name": "sale","type": "Date","value": null},{"name": "tarifa","type": "String","value": null},{"name": "zona","type": "Number","value": null}],"id": "Ticket:443994176","isPattern": "false","type": "Venta"},"statusCode": {"code": "200","reasonPhrase": "OK"}The representative information about these tickets is the following one:-

Emision: issue date-

Entra: beginning date-

Sale: expiration date-Zona: the regulated parking sector (mora than one expenders are associated to the same parking
sector)Information about expenders is also available, below we provide an example of NGSI query to retrievethem.
#!/bin/bash(curl http://fiware-dev.inf.um.es:1026/v1/queryContext -s -S --header 'Content-Type: application/json'
header 'Accept: application/json' --header 'fiware-service: ora' --header 'fiware-servicepath:/murcia/#'\-d @- | python -mjson.tool) <<EOF{"entities": [{"type": "Punto","isPattern": "true","id": "Parquimetro:*"
}
]
}
EOF

--

The result of such request is the following one:{"contextElement": {"attributes": [{"name": "azules","type": "number","value": "12"},{"metadatas": [{"name": "location",

H2020 EUJ-02-2016

CPaaS.io

Page 16 of 28

Deliverable D4.1

"type": "string","value": "WGS84"}],"name": "geoposicion","type": "coords","value": "37.99128625,-1.128053634"},{"name": "laborales","type": "number","value": "12"},{"name": "nombre","type": "string","value": "PUERTA%20NUEVA"}],"id": "Parquimetro:20","isPattern": "false","type": "Punto"},"statusCode": {"code": "200","reasonPhrase": "OK"}}In this case, for the expender #20 (Parquimetro:20) which is of the type Punto, it is worth highlighting its location,a representative name and the regulated parking sector (“azules”).
2.2.1.2 Required data processing logic
The following diagram, Figure 6, presents the different task comprising this use case, as well as the flowof information among them.

Figure 6: Diagram of the tasks for the use case

Following a bottom-up approach, we can see the following tasks:
Real-time prediction task: This task corresponds to the private parking sites; whose data is available inreal time. The town will be divided in five zones, each having multiple private parking sites. A task
instance will be launched for each zone, listening to occupation changes from its parking sites, but will
H2020 EUJ-02-2016

CPaaS.io

Page 17 of 28

Deliverable D4.1

only publish significant changes, i. e., only when the difference in available spots is above a certain
threshold, because a change in a single parking spot will not change the behavior of the system.
Estimation task: This task corresponds to the regulated parking zones of Murcia city. Their data isreceived by the system once a day, containing all the tickets issued that day, and therefore the data is
not available in real time. Because of that, an intelligent application will predict the occupation based on
the previous data. However, the prediction should be updated daily to include the new data. This task will
read the new data, update the prediction application and then publish it so the other components canuse it.
Parking Site Recommendation task: The results from the two previous tasks will feed this task, which
will be used to generate the response the user will receive.
Arrival Time Estimation task: This task is the endpoint receiving the requests by the users which areparametrized by specifying the location of the vehicle, as well as the desired destination.

2.2.1.3 The way to use the generated results
This application would provide some KPIs regarding the mobility of the city, and more specifically the
success rate of finding a free parking slots, benefiting both traffic scheduling in the city, by adjusting theperiod of traffic lights for instance. These results will also benefit the performance of the application
allowing it to improve the Parking Site Recommendation task.
2.2.1.4 Expected benefits
The expected benefits will come in an expected reduction of traffic, an improvement in the quality of air
thanks to this reduction of traffic, and a better traffic schedule.

2.3 Anomaly detection for transportation
Use case description
The availability and pervasiveness of a variety of sensors allows the collection of huge amounts ofmobility data which help not only study the patterns of human mobility, but also develop automatedincident detection systems (AID). Such systems contribute to the safety of urban areas and by
anticipating incidents, congestion and accidents can be reduced or even avoided. Adopting a data-driven
approach and applying machine learning techniques is a well-known method to tackle this problem. The
approach is summed up in 3 steps:
1. It clusters sensors with a novel multi-criteria distance metric and then it classifies them in normal
or malfunctioning sensors.
2. A spatial cross-correlation framework identifies seasonal and individual abnormal readings insome of those sensors to filter them out
3. A novel ML-based fundamental diagram discovers the critical density of a given road section/spot
on a data-driven fashion that is resistant to both outliers and noise within the input data.
Our focus for this use case is on the first step. The number of sensors being considerably high in sucha network, it is necessary to identify the abnormal i.e. malfunctioning sensors or the ones that arecharacterized by a low information/noise ratio. This helps not only with the efficiency, but also withthe reliability of the results.
H2020 EUJ-02-2016

CPaaS.io

Page 18 of 28

Deliverable D4.1

Figure 7. to illustrate the anomaly detection for transportation use case scenario

In Figure 7, a site represents a neighbourhood of sensors to which an edge device is associated. A
distance metric is calculated for each site by the edge device. Depending on the value, the FogFlow
Context Management System allocates the clustering task to either the FogFlow edge or to the FogFlow
Cloud. This is done periodically depending on the needs of the Transportation Management Systems.
Requirements for this specific use case
1. Available context data
This use case can be applicable to data from either static sensors (such as loop counters) or
dynamic ones (such as smartphones) which are relative to the transportation context. Figure 8
illustrates how induction loop traffic sensors operate.

H2020 EUJ-02-2016

CPaaS.io

Page 19 of 28

Deliverable D4.1

Figure 8. Induction loop traffic sensors

2. Validation data
In particular, we have relied for validation on data collected through a traffic monitoring systemof a major freeway deployed in an Asian country. This system both collects and broadcasts trafficbased measurements in real-time with distinct temporal granularities. Each sensor measurestraffic flow, lane occupancy, rate and instantaneous vehicle speed. The largest time granularity ofthis data collection system (p = 5 minutes) was used to normalize all the collected time series.
This dataset was formed by collecting data from 106 sensors which includes both freeway’s transit
directions. The total length of the analysed sections is roughly 20km while its sensors are
deployed each 500m. This data was collected through 3 non-consecutive weeks.
3. Required data processing logic
Our main task is to cluster the sensors into abnormal and normal ones based on a customized
statistical distance. The clustering is done in a distributed fashion following the fog flowarchitecture.
 First, we define neighbourhoods of sensors based on a geographical distance. The sensors aregrouped by section id of a given road. The roads are in turn grouped into.
 Then, the fog-flow edges perform a custom-made clustering task in each of theseneighbourhoods.
 It is key to define a threshold, based on which we decide if the clustering task is going to be
performed in the edge or in the cloud.
 Based on the transportation operators, we define the periodicity with which the clustering isbeing performed at the edge and the cloud.
Figure 9 details the service topology. We define two operators, A and B which respectively: calculate acustomized distance metric to group together sensors with similar behaviour, and perform a selection
based on principal coordinates analysis (PCoA). These two operators define our 2-stage clustering task
which differentiates between anomalous and functional sensors.

H2020 EUJ-02-2016

CPaaS.io

Page 20 of 28

Deliverable D4.1

Figure 9. Service topology

4. The way to use the generated results
The generated results are labelled sensors and allow us to dismiss data from faulty ones or thosewith very low information/noise ratio. The data from the normal/functional sensors is used towithin a machine learning framework that detects incidents per road section and helps minimize
the effect of congestion or accidents.

5.

Expected benefits
Performing the clustering task according to the fog flow architecture considerably reduces thebroadband requirements for a huge network of sensors such as the one used in transportation
management systems. Furthermore, it can allow for a faster reaction time to incidents since only
reliable sensors are considered as opposed to faulty ones. By relying on the FogFlow architecture
and this customized clustering technique, the system can handle incomplete and inaccurate
readings from the sensors without losing performance.

2.4 Opportunities for EU/Japan Collaboration
Japanese partners use u2-based architecture with IoT-aggregator concept. In IoT-aggregator, devices are
assumed to be resource-tight and inflexible. The IoT-aggregator concept supplements this limitation ofdevice capabilities and flexibilities.
FIWARE may be deployed in the future. However, a possible adoption of FIWARE platform during thecollaboration research period is unlikely. So, in order to accommodate federated research between EU
partners and Japan partners, we explore opportunities for EU/Japan collaboration.
The following scenario represented in Figure 10 is inspired by NEC, an EU partner and should be credited
as such.
H2020 EUJ-02-2016

CPaaS.io

Page 21 of 28

Deliverable D4.1

M issing
ChiildIdentification

Sea rch forEdge Nodes
Sea rchSea rchNetw ork (EU)

Netw ork (Ja pa n)

Public ca m era
edge

Public ca m era
edge

Orion Broker

Figure 10. Child search use case scenario

It is possible that a missing child can be found after a long period of vain search in a totally different
place. As time passes, a possible landing place can be extended to ones in cross-regions. To enable sucha long-term child search, the aim of this use case is that of taking advantage of the CPaaS.IO platform toprovide a service that tracks a missing child with widely-available public monitor cameras in a wider
search area, crossing country borders. This also provides a preventive measure of child kidnapping orother crimes because a missing child can be found over long time and place frameworks.
So, by using the FogFlow framework, the FogFlow Orchestrator will decide, according to therequirements of the application, the different tasks to be done as well as the edge/cloud components
responsible for them.
Once such instantiation is done, and according to the diagram, a federated public camera network
provides the information about possible place of a missing child (FogFlow-edges). The FogFlow Context
Management System receives this information and make it available to the original searchrequest(FogFlow-cloud). Finally, when a parent uses this service to obtain an extended search result of a
missing child to take a possible measure for securing their child. Such information will be provided by the
FogFlow Context Management System, thanks to the Missing Child Identification Task.

3 Requirement Analysis
In this section we summarize the specific requirements for each use case with the purpose of identifying
common aspects for all of them and to provide an overview of the performance specifications that eachservice requires too. The function requirement is done by the provided data processing flows.
To do so, the following table is presented grouping this task per use case and requirements in terms ofprogrammability and performance. This table covers the non-function requirements (e.g. scalability,
security, performance on bandwidth saving and latency, interoperability, privacy), mentioned in theprevious deliverables D2.1 [1] and D3.1 [2].

H2020 EUJ-02-2016

CPaaS.io

Page 22 of 28

Deliverable D4.1

In light of this table we can see that edge tasks can usually be completed without a high-performancedevice. We can also see that the organization of the Smart Cities also fits perfectly to the task distributionof the FogFlow framework where aggregation is always possible, distributing always the lighter task to
the edge and maintaining the heavy ones into the cloud plane.
In addition, we have also identified other requirements that must be considered in this framework which
involve the integrity and privacy of the information transmitted along the network infrastructure. Since
the core of our platform is the broker of information, and in this framework such task is achieved by the
IoT Broker component, we have added a subsection describing its security requirements.
Use Case
Anomaly
Detection in
shops
Anomaly
Detection in
shops
Smart Parking
Smart Parking
Smart Parking
Smart Parking
Anomaly
Detection fortransportation
Anomaly
Detection fortransportation

Task

Cloud/Edge
plane
edge

Programmability
requirements
Low

Performancerequirements
Low

Overload

Counter

cloud

High

High

High

Real-time
prediction
Estimation
Parking Site
Recommendation
Arrival Time
Estimation
Distance MetricCalculation

Edge

Low

Low

Low

Edge
Cloud

MediumHigh

Medium
High

Low
High

Cloud

Medium

High

High

Edge

Low

MediumMedium

Cloud

Low

High

High

Anomalydetection

PCoA Selection

Low

Table 1. Summary of requirements per use case

3.1 Security requirements for IoT Broker
The IoT Broker takes care of the task of communicating with the different IoT Devices and Gateways inorder to retrieve the needed information on behalf of the applications. For this reason, it must guarantee
the following security key aspects: identification, authentication, authorization, confidentiality andintegrity. Let us illustrate such requirements in the most common operations the IoT Broker supports.

3.1.1 Query
Figure 11 shows the Query data flow for IoT Broker. This data flow has the following steps:
1) Data consumer invokes QueryContext on IoT Broker;

H2020 EUJ-02-2016

CPaaS.io

Page 23 of 28

Deliverable D4.1

The query specifies which data the application wants to retrieve. The query can be
specified through wildcards, so it is not possible to know, at this step, which entities,
attributes, metadata or values are going to be presented to the user.
2) IoT Broker invokes DiscoverContextAvailability on IoT Discovery to find data sources that cansatisfy the query in (1);3) IoT Broker invokes QueryContext on the IoT data providers retrieved from IoT Discovery in step(2);4) IoT Broker assembles the query results from (3) into a single query result and returns it to thedata consumer.
data consumer(1) (4)(2)
IoT Discovery

IoT Broker(3)
IoT data providers

Figure 11. Query data flow.

At the operation level, the security components must check if the subject (user or system, for example) inthe data consumer is allowed to perform a QueryContext operation in the specified Broker. Figure 12
presents a suggestion of architecture for this scenario. Three components are added to the flow: (i)Identification/Authentication; (ii) Authorization; and (iii) a Policy Enforcement Proxy.
data consumer

Identification /Authentication(2) Authorization
requestNGSI 10NGSI 10 +Authorization
Token(1) Identity checkand authentication

Authorization(Access Control)(6)Query results(3) Query Context
Proxy

IoT Broker

IoT Discovery
NGSI 9NGSI 10(4) DiscoverContextAvailability* All messages sent over HTTPS(5) QueryContext

IoT data providers

Figure 12. Security enablers interactions on the Query data flow (Operation-based).

3.1.2 Update
Figure 13 shows the Update data flow in the composite scenario, where IoT Broker pushes data to data
consumer and data sink. This data flow has the following steps:
1) IoT data producer invokes UpdateContext on IoT Broker;
This operation is typically used for sending data in push-mode without a previous
subscription.
2) IoT Broker invokes DiscoverContextAvailability on IoT Discovery to find information on how tointerpret the update. This is used for Association purposes.
H2020 EUJ-02-2016

CPaaS.io

Page 24 of 28

Deliverable D4.1

3) IoT Broker invokes an UpdateContext on a data sink and NotifyContext in the event of subscriberinterested in the pushed data.
data sinkdata consumer

(3)

(3)

UpdateContext

NotifyContext

DiscoverContext
Availability(2)

IoT Broker

IoT Discovery(1) UpdateContext
IoT data producer

Figure 13. Update data flow (composite scenario).

In this scenario, the security components must check if the subject in the data producer is allowed toperform an UpdateContext operation in the specified Broker. Figure 14 shows a suggestion ofarchitecture and the respective components.
data consumer(5) Notify
Context

data sink(5) Update
Context

NGSI 10* All messages
sent over HTTPS(4) DiscoverContext
Availability

IoT Broker

IoT Discovery
NGSI 9Identification /Authentication(1) Identity checkand authentication

NGSI 10 +Authorization
Token

Proxy

(3) UpdateContext

IoT data producer

Authorization(Access Control)(2) Authorization
requestFigure 14. Security mechanisms applied to the Update data flow.

3.2 Analysis and selection of task Migration
One of the core research issues in WP4 for the CPaaS.io platform is to explore how dataprocessing/analytics tasks should be dynamically orchestrated and migrated over cloud and edges. Witha service/application- first approach, we first analyse the concrete requirements on task migration fromthe use cases described in the previous section, and then summarize existing approaches in the state of
the art. In the end, we select the high level approach to be explored in CPaaS.io.
3.2.1 Use case requirements to task migration
Table 2 summarizes the requirements of task migration in each use case from the following aspects:-

Needs of task migration: for each task in the service logic of each use case, is the task migrationneeded or not. In some cases, a task is required to stay always either in the cloud or at edge. For
example, if some task is very computation intensive and it must be assigned to the cloud; or if a

H2020 EUJ-02-2016

CPaaS.io

Page 25 of 28

Deliverable D4.1-

task is dealing with some sensitive data, it will be required to stay always at the local edge due tothe data protection reason.
Migration latency: the latency requirement to migrate a task from one place to another place;
Frequency of migration: how frequently the task migration will happen in order to maintain
good performance, such as low response time;
Stateful or stateless: whether the task has some internal state or not;Cloud-to-edge migration: task migration happens only between the cloud and some edge
node;
Edge-to-edge migration: task migration happens between two different edge nodes;

According to the analysis result, the following requirements need to be considered by the CPaaS.io
cloud-edge programming framework. 1) the designed programming models needs to allow developers
to specify whether a task should be migrated or not; 2) task migration could happen not only between
cloud and edges, but also between two different edges; 3) task migration could happen frequently as acommon case and the migration overhead expects to be low; 4) task migration is of important tostateless tasks.
Use
Case
Anomaly
Detection in
shops
Smart
Parking incitiesTask

Need oftask
migration

Migration
latencyFrequencyofmigration

Statefulorstateless

Cloud
to
edge

Edge
to
edge

Anomalydetection

Yes

Low
in seconds

stateless

Yes

Yes

Counter
Real-time
prediction
Estimation
Parking Site
Recommendation

No
No

-

Occasionally,
when edge isoverloaded-

StatelessStateless

-

-

No
Yes

Low
in seconds

Statelessstateless

Yes

Yes

Arrival Time
Estimation

Yes

Low
in seconds

stateless

Yes

Yes

Distance MetricCalculation

Yes

Low
in seconds

stateless

Yes

Yes

PCoA Selection

Yes

Low
in seconds

stateless

Yes

Yes

Anomaly
Detection fortransportationCommon,when switchfrom one area
to anotherCommon,when switchfrom one area
to anotherOccasionally,
when edge isoverloaded
Occasionally,
when edge isoverloaded

Table 2. Requirements on task migration for each use case

3.2.2 Selection of task migration in CPaaS.io
According to our study on the state of the art described in Section 1.1.2 and also the concrete
requirements of the proposed CPaaS.io use cases listed in Table 2, the task migration mechanism andorchestration algorithms in CPaaS.io need to fulfill the following features, which are actually not coveredby the state of the art, such as EdgeX, Azure IoT Edge, Amazon Greengrass.
H2020 EUJ-02-2016

CPaaS.io

Page 26 of 28

Deliverable D4.1



Task migration happens not only between central cloud and heavy edges, but also between
edgesThis design principle can largely simplify the required development effort and management overhead,especially FogFlow can support well distributed applications which could run across all cloud nodes andedge nodes seamlessly without knowing the details of how tasks coordination between cloud and edge
or between different edge nodes should be carried out. However, for most of the other IoT Edge
Computing frameworks, services or applications are designed for each edge and they are not really
distributed services or applications, because those services or applications are able to run either in thecloud or at some edge but they are not able to run over cloud nodes and edge nodes in a distributed
fashion. The FogFlow services and applications are designed against a global view of all cloud nodes andedge nodes, rather than from the perspective of each individual edge node.
To be more light-weight, migration is enabled at the function level for mainly stateless tasks

In CPaaS.io we choose to have a light-weight approach of task migration due to the latency requirementof task migration in many of our proposed use cases. For example, in the case of smart parking, theparking recommendation is expected to be updated in real-time according to the real-time situation ofroad traffic and latest availability of current parking pots. Also, when a connected car is moving from one
area to another area, the associated tasks will be switched from one edge node to another edge node.
Task migration with low overhead and fast migration time will be our top option.


The service orchestration in CPaaS.io (more specially, the FogFlow framework) is globally
driven by context, rather than raw events or topics.

This feature is enabled by the design of introducing a new layer, namely IoT Discovery, which provides
the updated summary of available entity data on all IoT brokers. As compared to event or topic based
orchestration, our context-based orchestration in FogFlow is more flexible and more light-weight. This isbecause the orchestration decisions in FogFlow can be made based on aggregated context, without
reading through all involved data streams.

4 Conclusions & Next Steps
Along this deliverable we have posed an interesting questing regarding how to address application andservices based on the information provided by a city over a smart city platform. The heterogeneity of theinformation, as well as the huge amount of data that must be analysed fit perfectly the use of our
FogFlow framework where different tasks can be deployed dynamically over a shared edge/cloudenvironment and be easily orchestrated.
In this deliverable we also provide three concrete use cases where the deployment of such framework is
detailed at different levels, presenting the different tasks that must be undertaken, the responsibility to
do such tasks and even the data model that should be used.
Thanks to the detailed view of each specific use case, we were able to identify the different requirementsthat each of them needed. Allowing us to properly define in the future the different tasks eachcomponent of this framework will undertake and the features each edge/cloud component must comply.
So, as future work these use cases will be physically deployed with a concrete instantiation of ourframework obtaining this way valuable information and results to measure the performance of our
FogFlow framework.
H2020 EUJ-02-2016

CPaaS.io

Page 27 of 28

Deliverable D4.1

5 References[1] D2.1: CPaaS.io deliverable “Requirement Specification (V1)”, Martin Strohbach ed.
[2] D3.1: CPaaS.io deliverable “Requirement Collection and Analysis”, François Carrez ed.
[3] D4.2: CPaaS.io deliverable “Profiling Specifications for Cloud-Edge Orchestration”, Bin Cheng ed.
[4] B. Cheng, G. Solmaz, F. Cirillo, E. Kovacs, K. Terasawa and A. Kitazawa, "FogFlow: Easy
Programming of IoT Services Over Cloud and Edges for Smart Cities," in IEEE Internet of Things
Journal, vol. PP, no. 99, pp. 1-1.
[5] A. Ridi, C. Gisler and J. Hennebert. ACS-F2 - A New Database of Appliance Consumption Analysis,in Proceedings of the 6th International Conference on Soft Computing and Pattern Recognition(SocPar 2014), pages 145-150.
[6] E. Saurez, K. Hong, D. Lillethun, U. Ramachandran, and B. Ottenwalder, “Incremental deployment
and migration of geo-distributed situation awareness applications in the fog,” in Proceedings ofthe 10th ACM International Conference on Distributed and Event-based Systems. ACM, 2016, pp.
258–269[7] Azure IoT Edge, https://azure.microsoft.com/en-us/services/iot-edge/[8] AWS Greengrass, https://aws.amazon.com/greengrass/[9] Cloudlet, http://elijah.cs.cmu.edu/[10] Baldini, Ioana, et al. "Serverless computing: Current trends and open problems." Research
Advances in Cloud Computing. Springer, Singapore, 2017. 1-20.

H2020 EUJ-02-2016

CPaaS.io

Page 28 of 28