# A Software System for the Discovery of Situations Involving Drivers in Storms

Markus Stocker, Okko Kauhanen, Mikko Hiirsalmi, Janne Saarela, Pekka Rossi, Mauno Rönkkö, Harri Hytönen, Ville Kotovirta, Mikko Kolehmainen

## ► To cite this version:

Markus Stocker, Okko Kauhanen, Mikko Hiirsalmi, Janne Saarela, Pekka Rossi, et al.. A Software System for the Discovery of Situations Involving Drivers in Storms. 11th International Symposium on Environmental Software Systems (ISESS), Mar 2015, Melbourne, Australia. pp.226-234, 10.1007/978-3-319-15994-2_22. hal-01328554

HAL Id: hal-01328554

https://inria.hal.science/hal-01328554

Submitted on 8 Jun 2016

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A Software System for the Discovery of Situations
Involving Drivers in Storms

Markus Stocker¹, Okko Kauhanen¹, Mikko Hiirsalmi², Janne Saarela³, Pekka Rossi⁴,
Mauno Rönkkö¹, Harri Hytönen⁵, Ville Kotovirta², Mikko Kolehmainen¹

¹Department of Environmental Science, University of Eastern Finland, Kuopio, Finland
{markus.stocker,okko.kauhanen,mauno.ronkko,mikko.kolehmainen}@uef.fi

²VTT Technical Research Center of Finland, Espoo, Finland
{mikko.hiirsalmi, ville.kotovirta}@vtt.fi

³Profium Oy, Helsinki, Finland
janne.saarela@profium.com

⁴Finnish Meteorological Institute, Helsinki, Finland
pekka.rossi@fmi.fi

⁵Vaisala Oyj, Vantaa, Finland
harri.hytonen@vaisala.com

**Abstract.** We present an environmental software system that obtains, integrates, and reasons over situational knowledge about natural phenomena and human activity. We focus on storms and driver directions. Radar data for rainfall intensity and Google Directions are used to extract situational knowledge about storms and driver locations along directions, respectively. Situational knowledge about the environment and about human activity is integrated in order to infer situations in which drivers are potentially at higher risk. Awareness of such situations is of obvious interest. We present a prototype user interface that supports adding scheduled driver directions and the visualization of situations in space-time, in particular also those in which drivers are potentially at higher risk. We think that the system supports the claim that the concept of situation is useful for the modelling of information about the environment, including human activity, obtained in environmental monitoring systems. Furthermore, the presented work shows that situational knowledge, represented by heterogeneous systems that share the concept of situation, is relatively straightforward to integrate.

**Keywords:** Environmental knowledge systems, situation theory, ontology, knowledge representation, MMEA, Wavellite

## 1 Introduction

Computational models [1], in particular empirical and physically-based models, can be used on data to obtain information. In some environmental software systems, data is obtained in environmental monitoring and information is about the environment. Data is, specifically, the result of measurement, defined as "the process of empirical,
objective, assignment of numbers to properties of objects or events of the real world
in such a way as to describe them" [2]. To automatically sample physical properties at
high spatiotemporal resolution, measurement is often implemented by environmental
sensor networks.

An environment monitored in space-time can be abstracted as structured parts of
reality, in other words situations [3]. The Situation Theory, developed in the 1980s by
Barwise and Perry [4] and Devlin [3], proposes a mathematical ontology for infor-
mation about situations. A situation is said to support infons, whereby and infon is a
tuple consisting of a relation, a set of objects, and the polarity to state whether or not
the objects stand in the relation. Environmental phenomena, including humans in
activity, are objects. Such objects have attributes and stand in relation to each other.

Models can be used on measurement data to obtain information about situations.
The term *situation* can serve as the most abstract concept in a framework that models
information about structured parts of reality monitored in space-time. We present the
application of such a framework, called Wavellite, to the modelling of situations with
storms and driver locations as the objects. The resulting system reasons over situa-
tional knowledge to detect those in which storms and drivers overlap in space-time.
Such situations are of interest because people may be at higher risk of accident.

The contribution of this work is two-fold. First, together with our related work [5, 6], we underscore the suitability of the notion of situation for the modelling, i.e. explicit representation, of situational knowledge obtained using models from processed data collected and managed by environmental monitoring systems, possibly building on environmental sensor networks. Second, we present the integration of situations with information about the environment (storms) and situations with information about human activity (driver locations). The application suggests that the notion of situation is suitable not only to model situational knowledge about the natural environment but also situational knowledge about human activity occurring in the built environment. The application arguably supports the claim that to share the concept of situation among heterogeneous environmental monitoring systems facilitates situational knowledge processing, including integration and reasoning.

## 2 Materials and Methods

The presented system builds on the Measurement Monitoring and Environmental
Assessment (MMEA) Platform,<fnref n="1" /> the Profium Sense<fnref n="2" /> semantic technology-based soft-
ware platform, and Wavellite.<fnref n="3" /> The MMEA Platform is a software infrastructure for
the collection, storage, processing, and distribution of (processed) observation and
forecast data resulting in environmental monitoring. The platform implements an
Enterprise Service Bus (ESB) based architecture and was developed within the Finn-
ish MMEA CLEEN/SHOK research program.

<footnote n="1">¹ http://mmea.fi/</footnote>

<footnote n="2">² http://www.profium.com/en/profium-sense</footnote>

<footnote n="3">³ http://www.uef.fi/en/envi/projects/wavellite</footnote>
In this work, the MMEA Platform handles the collection and processing of data and initiates situational knowledge extraction. The platform also powers the user interface. Profium Sense serves as RDF database for the persistence and retrieval of situational knowledge. Wavellite is a software framework for situation awareness in environmental monitoring. It supports the collection and processing of data, specifically data of environmental sensor networks, and the representation of situational knowledge acquired from data using models, in particular empirical (data-driven) and physically-based environmental models. In this work, Wavellite builds on top of the MMEA Platform, reuses platform functionality and extends it with situational knowledge extraction and representation.

We use radar data for the reflectivity of rainfall intensity in Finland. Radar data is obtained by the Finnish Meteorological Institute (FMI) and is made available by FMI as GeoTIFF encoded images via its Open Data INSPIRE-compliant interface.<fnref n="4" /> FMI also provided a MATLAB script that implements an algorithm for the extraction of storm polygon data from radar data. Because the MMEA Platform uses Octave,<fnref n="5" /> we implemented the MATLAB program logic as Octave script. Given a scheduled driver direction (consisting of departure time, origin, and destination) we use Google Directions<fnref n="6" /> to obtain (alternative) routes, driving distance and estimated driving time.

The MMEA Platform continuously (configurable time interval) retrieves radar data and processes it to extract polygon coordinate data. Polygon data is Well-Known Text (WKT) encoded and wrapped in MMEA observation messages, which are published to the ESB for further processing. We implemented a Wavellite derivation reader, i.e. the component of the Wavellite architecture that reads Wavellite dataset observations, as ESB listener for MMEA observations messages containing storm polygon data. Upon incoming message, the derivation reader converts the MMEA observation message to a Wavellite dataset observation with two component properties, one for time and one for the WKT polygon data. Wavellite dataset observations are then processed by a Wavellite situation engine, i.e. the component of the Wavellite architecture that represents situations with knowledge extracted from dataset observations. For storms, situations $s_{sa}$ support the infon

$$ \ll \text{storm} - at, \hat{s}, \hat{t}, \hat{l}, 1 \gg $$

where $\hat{s}$, $\hat{t}$, and $\hat{l}$ are parameters for a storm, a temporal location, and a spatial location, respectively. Given a dataset observation with component properties for spatiotemporal locations it is thus straightforward to map dataset observations to situations by anchoring a new individual (representing the storm) to $\hat{s}$ and the temporal and spatial locations of the dataset observation to $\hat{t}$ and $\hat{l}$, respectively. The infon states that at time anchored to $\hat{t}$ the storm with identifier anchored to $\hat{s}$ covers the spatial region delineated by the polygon anchored to $\hat{l}$.

<footnote n="4">⁴ https://en.ilmatieteenlaitos.fi/open-data</footnote>

<footnote n="5">⁵ https://www.gnu.org/software/octave/</footnote>

<footnote n="6">⁶ https://developers.google.com/maps/documentation/directions/</footnote>
Upon user registration of a scheduled driver direction – consisting of origin, A, destination, B, and departure time – a component of the MMEA platform uses Google Directions to obtain a route, r, between A and B. Given r, the component computes route segments and driver location for each segment. The component uses locations for which there exist situations $s_{sa}$ to segment the route. It is thus temporal locations at which radar images are obtained, or at which storm situations are projected, that determine time steps. For driver locations, situations $s_{da}$ support the infon

$$ \ll \text{driver} - \text{at}, \dot{d}, \dot{t}, \dot{l}, 1 \gg $$

where $\dot{d}$ is a parameter for a driver and $\dot{t}, \dot{l}$ are parameters for temporal and spatial locations, respectively. The infon states that at time $\dot{t}$ the driver $\dot{d}$ is located at $\dot{l}$. Users can register driver directions using the prototype Web user interface.

A reasoning component of the MMEA Platform uses situations $s_{sa}$ and $s_{da}$ to discover situations $s_{hr}$ that support the infon

$$ \ll \text{higher} - \text{risk}, \dot{d}, \dot{t}, \dot{l}, 1 \gg $$

which states that the driver $\dot{d}$ is in a higher risk situation at temporal and spatial locations $\dot{t}$ and $\dot{l}$, respectively. The parameter $\dot{l}$ for spatial location is optional in this infon, as it can be obtained from the situation with driver-at-relation infon with same objects anchored to parameters $\dot{d}$ and $\dot{t}$. Situations $s_{hr}$ are of interest to drivers.

The discovery of such situations is implemented as SPARQL [7] queries that leverage on Profium Sense support for quantitative spatial reasoning and qualitative spatial relations in SPARQL queries. Specifically, the query is for situations with storm-at-relation infon and situations with driver-at-relation infon with same temporal locations and driver spatial location inside storm spatial location.

Of particular interest in this system is situation projection. The projection of situations $s_{da}$ is trivial, as it amounts to the registration of a point of departure at some future temporal location. The projection of situations $s_{sa}$ is, however, more complex. Indeed, the task amounts to forecasting the future spatial location of storms. As a proof of concept, we have implemented Octave program logic to support forecasting storms. The method uses temporal clustering [8] and recursively searches for matching storms in consecutive radar images to form storm tracks over time. At each time step, a storm has no predecessors if it just formed; one predecessor if it has a defined path from the previous time step; more than one predecessor if several storms have merged together since the previous time step; or it disappeared. A storm can progress in one direction or multiple directions, if it has split into different storms [9]. For each storm track, the method then fits a suitable model to observed storm centroids and projects the storm using the model. The weighted linear regression model assigns smaller weights to older observations in order to make projection more sensitive to newer observations. The direction vector of the storm centroid is used to displace all storm polygon vertices.
The Web graphical user interface is a JavaScript client that makes use of OpenLayers,<fnref n="7" /> jQuery,⁸ Google Direction Service,⁹ and a Web service for communication with the MMEA Platform (retrieval of situations and addition of directions). In Wavellite, situations are individuals, instances of the class Situation. The class is defined in the Situation Theory Ontology (STO) [10] and reflects Situation Theory semantics. Thus, Wavellite builds on Semantic Web technologies, in particular the Resource Description Framework (RDF) [11] and the Web Ontology Language (OWL 2) [12]. Wavellite interacts with Profium Sense to store and retrieve RDF.

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Overview of the Web graphical user interface. Situations are shown animated on a map according to simulation parameters (start and end time). On the right hand side, new directions can be added. Visible are situations at 17:50, including a large storm and the location of a driver (circle) on a direction from Turku to Kuopio. The direction is highlighted.</figcaption></figure>

<footnote n="7">⁷ http://openlayers.org/
⁸ http://jQuery.com/
⁹ https://developers.google.com/maps/documentation/javascript/directions</footnote>
<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Examples for situational knowledge visualization for storm and driver locations at several points in time on October 10, 2014, between 13:00 and 18:00. At 14:15, Figure (a), a driver travelling from Turku to Kuopio is within the storm. A second driver travelling from Kuopio to Tampere is not. The storm has left Turku and is over Helsinki. At 15:55, Figure (b), a third driver travels from Tampere to Helsinki and is not affected by the storm. The storm is approaching Kuopio. At 17:00, Figure (c), the driver travelling to Helsinki had arrived. The driver travelling from Kuopio to Tampere exited the storm. The storm is now over Kuopio.</figcaption></figure>

## 3 Discussion

The system operates in near real-time and populates the RDF database with situation- al knowledge, which can be visualized. Figures 1 and 2 provide examples.

Upon user registration of an origin (e.g. Kuopio) and a destination (e.g. Tampere) for a scheduled driver direction starting at temporal location $t_l$, the system segments the route and computes situations $s_{da}$. Given situations $s_{da}$ and $s_{sa}$, we can create visualizations such as the map shown in Figure 1 where the driver location (circle) is shown along the direction between Kuopio and Tampere and the spatial location of the storm is shown as polygon. The system reasons over such situational knowledge to discover higher-risk situations in which the driver location and storm location over- lap in space-time. Figure 2 visualizes several higher-risk situations. Drivers may want to be aware of such situations.

Figure 2 shows situations at three time steps on October 10, 2014: at 14:15, Figure 2(a); at 15:55, Figure 2(b); and at 17:00, Figure 2(c). The relatively large storm is moving south-west to north-east from Turku to Kuopio. At the same time, several drivers are located along directions, Turku-Kuopio, Kuopio-Tampere, and Tampere- Helsinki. At various times, drivers overlap the storm. Only the driver directed from Tampere to Helsinki is never in a higher-risk situation.

The presented software system is a proof of concept. The aim was to demonstrate the integration of situational knowledge obtained from heterogeneous data sources using a set of technologies, including the MMEA Platform, in an environmental monitoring context. However, with further development the system could mature so that it can be deployed for active use.

Drivers are perhaps the obvious potential interest group for such a system. Directions scheduled in a calendar could be automatically 'safety checked' hours prior to departure. The system could inform drivers accordingly. For end consumers, such as
drivers, mobile applications may be of particular interest. The presented Web graph-
ical user interface is a lightweight JavaScript application. It communicates via HTTP
requests with map, direction, and situation services, which implement the bulk of the
program logic. Thus, developing a graphical user interface for mobile platforms
should be fairly trivial.

The current system can also be extended to integrate more situational knowledge, and thus improve situation awareness [13]. For instance, during winter road conditions are critical. In addition to snow storms, of particular interest is situational knowledge for road sections that may be covered with ice, of poor visibility, or congested. Such information is likely to result from data processed by different systems. However, because of the shared ontology, the integration of situational knowledge would be as straightforward as in the case presented here.

Google Directions suggests alternative directions. This feature can be exploited when higher-risk situations are discovered. For such situations, the system can notify drivers and suggest (1) alternative directions with same departure time, (2) same direction with earlier or later departure time, or (3) to postpone travelling altogether if no direction can be scheduled for which the driver will not find herself involved in a higher-risk situation.

Critical for future work, in particular work that aims at a mature system deploya-
ble for active use, is the evaluation of situation projection. This can be achieved
through analysis of how well forecasts agree with observations, which requires the
construction of a dataset that appropriately aligns storms in space-time. Of particular
interest is the comparison of storm polygon centroids and storm spatial extent. Uncer-
tainties related to storm development could be modelled explicitly. Instead of project-
ing storms along a line, or path, a system that takes uncertainties into account could
project storms along regions that widen the further the projection is in space-time.
Rather than to search for higher-risk situations along a line, the system could thus
search for such situations in the region in which a storm will likely be. Naturally, such
a search is computationally more demanding. Further issues need to be addressed in
work that aims at a system deployed for active use, including liability in case of sys-
tem failure. Such concerns are however beyond the scope of this paper.

Route planning is an old problem and algorithms have been developed that are
constraint aware. For instance, authors have proposed estimating the cost of travelling
through particular grid cells, whereby factors such as weather conditions can influ-
ence the cost value [14]. In [15], the authors present a personalized mobile traffic
information system capable of handling weather information, including rainfall.

There exist online services for road trip planning that include weather infor-
mation. For instance, AccuWeather<fnref n="10" /> supports this feature. Together with the direc-
tion, the service shows (forecast) weather at various locations along the direction. The
Weather Channel<fnref n="11" /> offers a similar feature. Aside the obvious difference between
weather and storms as well as the nature of the data involved, the main novelty here is
arguably in technologies that support the explicit representation of situational

<footnote n="10">¹⁰ http://www.accuweather.com/en/driving-directions-weather</footnote>

<footnote n="11">¹¹ http://www.weather.com/travel/trip-planner</footnote>
knowledge with machine readable and interpretable semantics specified in ontologies
that can be shared among heterogeneous environmental monitoring systems.

4 Conclusion

We presented an environmental software system that obtains, integrates, and reasons
over situational knowledge about storms and driver directions. As such, the system
demonstrates the integration of situational knowledge about natural phenomena and
situational knowledge about human activity. All knowledge is represented explicitly.
As a consequence, the system can manipulate situational knowledge. Spatio-temporal
situation reasoning is an example for such manipulation. As we demonstrated, infer-
ring situations in which drivers are at higher risk can be achieved by means of a sim-
ple SPARQL query. The work underscores the suitability of the concept of situation,
as defined in Situation Theory, for the modelling of situational knowledge obtained
using computational models from processed data collected and managed by environ-
mental monitoring systems.

A thorough discussion of the advantages and disadvantages of the presented sys-
tem could be valuable. However, such an evaluation requires at least one comparable
system that uses a different set of technologies. We currently do not have such a com-
parable system. Aside from the discussed advantages, a disadvantage of the presented
system may be the potentially large number of generated RDF statements. However,
such concerns need to be addressed primarily by the RDF database.

Naturally, the described system could be extended to include situations of various
other types that are of interest to driver Situation Awareness [13, 16]. For instance,
situations involving road sections possibly covered with ice may be of interest. It is
trivial to see how such additional situational knowledge can be integrated in the pre-
sent system. Other directions for future work include the development of mobile ap-
plications and the integration with driver assistance systems.

Acknowledgements

We thank the Finnish Funding Agency for Technology and Innovation (TEKES) and
the Academy of Finland for funding this research. This work has been carried out in
TEKES funded CLEEN SHOK programme “Measurement, Monitoring and environ-
mental Assessment” (MMEA, decision number 427/10) and in the Pathway project
funded by the Academy of Finland.

References

1. Mulligan, M., Wainwright, J.: Environmental Modelling: Finding Simplicity in Complexity, chap. Modelling and Model Building, pp. 7-73. John Wiley & Sons, Ltd (2004)
2. Finkelstein, L.: Theory and Philosophy of Measurement. In: Sydenham, P.H. (ed.) Handbook of Measurement Science, Volume 1, Theoretical Fundamentals, pp. 1-30. John Wiley & Sons (1982)

3. Devlin, K.: Logic and Information. Cambridge University Press (1991)

4. Barwise, J., Perry, J.: Situations and Attitudes. The Journal of Philosophy 78(11), 668-691 (November 1981), http://www.jstor.org/stable/2026578

5. Stocker, M., Baranizadeh, E., Portin, H., Komppula, M., Rönkkö, M., Hamed, A., Vиртанен, А., Lehtinen, K., Laaksonen, A., Kolehmainen, M.: Representing situational knowledge acquired from sensor data for atmospheric phenomena. Environmental Modeling & Software 58(0), 27-47 (2014), http://www.sciencedirect.com/science/article/pii/S1364815214001108

6. Stocker, M., Rönkkö, M., Kolehmainen, M.: Situational knowledge representation for traf fic observed by a pavement vibration sensor network. IEEE Transactions on Intelligent Transportation Systems 15(4), 1441-1450 (August 2014)

7. Prud'hommeaux, E., Seaborne, A.: SPARQL Query Language for RDF. Recommendation, W3C (January 2008), http://www.w3.org/TR/2008/REC-rdf-sparql-query-20080115/

8. Ester, M., Kriegel, H.P., Sander, J., Xu, X.: A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise. In: Proceedings of 2nd International Con-ference on Knowledge Discovery and Data Mining (KDD-96). pp. 226-231. AAAI Press (1996)

9. Dixon, M., Wiener, G.: TITAN: Thunderstorm Identification, Tracking, Analysis, and Nowcasting–A Radar-based Methodology. Journal of Atmospheric and Oceanic Technology 10(6), 785-797 (1993)

10. Kokar, M.M., Matheus, C.J., Baclawski, K.: Ontology-based situation awareness. Inf. Fusion 10(1), 83-98 (Jan 2009)

11. Manola, F., Miller, E., McBride, B.: RDF Primer. W3C Recommendation, W3C (February 2004)

12. W3C OWL Working Group: OWL 2 Web Ontology Language Document Overview (Second Edition). Recommendation, W3C (December 2012), http://www.w3.org/TR/2012/REC-owl2-overview-20121211/

13. Endsley, M.R.: Toward a theory of situation awareness in dynamic systems. Human Factors: The Journal of the Human Factors and Ergonomics Society 37(1), 32-64 (1995)

14. Szczerba, R., Galkowski, P., Glicktein, I., Ternullo, N.: Robust algorithm for real-time route planning. Aerospace and Electronic Systems, IEEE Transactions on 36(3), 869-878 (Jul 2000)

15. Balke, W.T., Kiessling, W., Unbehend, C.: A situation-aware mobile traffic information system. In: System Sciences, 2003. Proceedings of the 36th Annual Hawaii International Conference on. (Jan 2003)

16. Stanton, N.A., Stewart, R., Harris, D., Houghton, R.J., Baber, C., McMaster, R., Salmon, P., Hoyle, G., Walker, G., Young, M.S., Linsell, M., Dymott, R., Green, D.: Distributed situation awareness in dynamic systems: theoretical development and application of an ergonomics methodology. Ergonomics 49(12-13), 1288-1311 (2006), http://dx.doi.org/10.1080/00140130600612762, pMID: 17008257