* More than 5 billion pieces of content (web links, news stories, blog posts, notes, photo albums, etc.) shared each week

* More than 3.5 million events created each month

* An average user has 130 friends on Facebook, spends more than 55 minutes per day on the site, writes 25 comments each month, is invited to 3 events per month, and is a member of 13 groups.

* Facebook Platform currently holds 500,000 active applications, more than 250 of them have more than one million monthly active users.

* There are more than 100 million active users currently accessing Facebook through their mobile devices, and they are twice more active on Facebook than non-mobile users.

Some selected statistics about Twitter (Berry, 2010; Weil, 2010):

* In February 2010, Twitter was handling 50 million tweets a day

* In average, it represents 600 tweets per second

* Twitter.com is visited by 180 million visitors every month

* 75% of Twitter's traffic comes from outside their own site (e.g. mobile clients)

More general statistics on usage of social networking sites: News (2010)

* The average social-networking user around the world spent more than 5 and a half hours on their social networking sites, in December 2009.

* In the USA, 6 hours.

* In France, 4 hours.

* This duration increased by 82% from December 2008.
### 2.1.12 Conclusion

In this section, we have seen that SNS are powerful social communication platforms with several different communication modalities relying on the viral spreading of social content, interactions and updates. Nevertheless, the big enthusiasm generated by SNS has led to a productivity issue which made many companies restrict access to SNS. The paradox in that matter is that several initiatives are translating the SNS paradigm to enterprises in order to improve corporate knowledge management and sharing through participation of their employees. With intranet-based SNS, enterprise workers can benefit from immediate expertise and news rather than relying on classical Knowledge Management practices, which are costly in terms of redaction, and less personalized than talking directly with an expert. SNS can thus support decision making, by incorporating business intelligence from large amounts of individual contributions, assuming that overall participation is high.

SNS are a potential killer application for casual awareness, opportunistic communication (e.g. between businesses and customers), and for improving knowledge sharing, collaborations, productivity and fulfillment of professionals, but they are much too time consuming yet, since every piece of information to spread has to be entered manually. In the next section, we will further analyze how productivity can suffer from the use of social networking tools, and identify opportunities to improve them in this regard.

## 2.2 Usage and impact of real-time microblogging

In April 2010, we conducted a survey towards 256 users of real-time microblogging platforms, mostly Twitter users, in order to analyze the usage and impact of those platforms. In this section, we present the results of this survey, and discuss their explanation and possible usage improvements towards reducing information overload and interruptions.

### 2.2.1 Results of the survey

As seen on the distribution chart 2.7, most respondents (31%) follow between 100 and 250 people feeds. The second category of respondents (28%) follow more than 250 feeds, which is much higher than the Dunbar number (i.e. the maximum number of people one can naturally maintain a stable relationship with).
<figure><img src="image_33.png" /><figcaption>Figure 2.7: Distribution of respondents among number of followed feeds -</figcaption></figure>

<figure><img src="image_34.png" /><figcaption>Figure 2.8: Uses of microblogging -</figcaption></figure>
Figure 2.8 reveals that 75% of respondents use microblogging platforms for their professional activity, 69% use those for keeping up with news, and 64% leverage the ‘six degrees of separation’ effect for networking (i.e. making announcements, or ask for advice from their contacts). Personal and fun usage of microblogs is lowly represented (23% to 38%), at least by the respondents of this survey.

<figure><img src="image_35.png" /><figcaption>**Figure 2.9: Usefulness of microblog updates** - for current activities (left), and for longer-term goals (right)</figcaption></figure>

On a 6-rank scale between 0 and 5, most respondents (between 35.9% and 38.7%) gave a medium-high rate of 3 to evaluate the usefulness they perceive over all the updates they read, as depicted on Figure 2.9. However, we observe that usefulness is rated slightly higher for longer-term goals (4 being the second most given rate, with 30.5%) than for current activities (2 being the second most given rate, with 24.6%). This can support the hypothesis that, in most cases, new real-time updates should rather not interrupt users in their current activities, as many updates show little usefulness for this activity.

According to Figure 2.10, most respondents (35%) keep the last updates from their microblogging platform always visible on their screen. About as many respondents (33%) activated pop-up (also called ‘toaster’) notifications on their screen, in order to keep up with the last updates in real-time. Fewer respondents chose to monitor the number of unread updates (20%), in order to decide when to read those messages, or to receive notifications on their mobile phone (e.g. SMS) (14%).

As seen on Figure 2.11, the amount of attention given to real-time update notifications varies among respondents who receive such notifications, whereas 18% of them do not receive such notifications. Most of them (62%) read most notified updates, depend-
<figure><img src="image_36.png" /><figcaption>Figure 2.10: Notification modalities for real-time updates -</figcaption></figure>

<figure><img src="image_37.png" /><figcaption>Figure 2.11: Attention given to notified updates -</figcaption></figure>
ing on their ongoing activity and availability. Others either read all notified updates (7%), or simply ignore those (13%).

<figure><img src="image_38.png" /><figcaption>**Figure 2.12: Probability of consulting content attached to updates - (left), and of responding to them (right)**</figcaption></figure>

From Figure 2.12, we can observe that microbloggers consume more content than they produce. Indeed, 66% of them consult the attached content of 50% to 80% updates they receive, whereas the same proportion of respondents actually respond by replying or relaying 10% to 50% updates.

In response to the large amount of social updates they receive, Figure 2.13 depicts that 78% of respondents don't use filtering mechanisms. The most number of others who do (8%) developed their own filtering mechanism, instead of leveraging existing services offering filters (5%) or trends (4%). 5% respondents, beyond not using filtering mechanisms, would rather receive more updates. This last category represents a close portion to respondents who follow less than 10 feeds, as depicted on Figure 2.7, which would explain their eagerness for more updates.

Although we have observed that most respondents don't use filtering mechanisms, Figure 2.14 reveals a need for filtering: 59% of them would like to filter by type of updates (e.g. news, professional, personal, fun, etc...) and 51% by subject. This need for filtering mechanisms is confirmed by other studies (Bernstein et al., 2010). Relevance filtering is also expected by respondents, either according to their current interests (37%) or current activity (36%). Fewer of them would like to benefit from other filtering criteria on updates, such as the type of content (20%), geographical proximity (18%), or relevance with future activities and plans (16%).
<figure><img src="image_39.png" /><figcaption>Figure 2.13: Use of filtering -</figcaption></figure>

<figure><img src="image_40.png" /><figcaption>Figure 2.14: Filtering criteria most expected from respondents -</figcaption></figure>
### 2.2.2 Filtering mechanisms: challenges, solutions and opportunities

On Figure 2.14, we have seen that the two most expected filtering mechanisms are by type of update, and by subject. This can be surprising, as it is already granted for microblogging users to choose whose feeds they follow, and to follow subjects of interest represented by corresponding terms and hashtags. However, these native features provided by current microblogging platforms are a partial solution to that problem: people do not always post updates of the same type (or at least, they do not have to), and it can be tedious for users to select the right terms and hashtags for following updates on a specific subject. Query enrichment and semantic search mechanisms could support them for this second need. Concerning the type of updates, a solution could be that people post updates through different feeds, depending on their type. They would thus have to expose and maintain separate professional, personal, and other specific feeds, and their followers would have to follow the feeds they are interested in, depending on the type of updates posted to those. Another solution would be to ask users to provide some metadata to describe the type of update they intend to post, as long as microblogging platforms would allow type-filtered subscriptions to their author's feed, instead of simply following his whole feed.

The two following most expected filtering mechanisms are more dynamic: they rely on relevance of updates with the consumer's current interests and activities. As a person's current interests can be profiled, updates can be ranked against this profile, in order to deliver the most relevant ones. Such filtering mechanism is proposed by the 'my6sense' application <fnref n="1" />, in which users' interests are profiled according to descriptive features (e.g. metadata, tags, or frequent words from the content) that are attached to the content they decide to read. Concerning the relevance with current activities, such profiling becomes more complex, as users can achieve their activities out of the scope of the application that delivers updates, e.g. on various computer applications, or even without the computer. In that case, some knowledge on the current activity of a user must be gathered by analyzing contextual cues acquired from several sensors: physical (e.g. user's location from a GPS device), virtual (e.g. name of the document being edited with word-processing software on the computer) and social (e.g. messages and

<footnote n="1">¹my6sense: http://www.my6sense.com/</footnote>
updates from other users, giving clues about the user's current activity and context). That challenge is addressed in the following chapters of this thesis.

### 2.2.3 Discussion

This survey reveals that users of real-time microblogging platforms (such as Twitter):

* usually consume many updates (with or without being notified in real-time), mostly for professional use, news and networking;

* mostly receive updates that are interesting in regard to their longer-term goals, more than to their current activities;

* consult the content attached to many updates, but do not respond to them often;

* and that they would benefit from filtering mechanisms.

In response to those observations, we proposed some opportunities for improvement:

* In order to allow users to filter updates by type more easily, we recommend that each user maintains separate accounts (with different followers) for each type of updates (e.g. personal messages, news on specific topics, etc...)

* New metadata could also be added to microblogging systems, so that authors could precise the type of each update they post.

* In response to the need for filtering by relevance with users' current interests and activities, we proposed to develop a system that can maintain a dynamic profile of users, based on physical, virtual and social sensors.

In the next section, we study the results of this survey and identify opportunities for improvement, from a cognitive point of view.

### 2.3 Social notifications: cognitive impact and opportunities for improvement

In the previous section we observed the usage and habits of microblogging users, and proposed technical solutions for user-identified needs (e.g. filtering). Here, we discuss this usage and its impact from a cognitive point of view, and identify additional opportunities for improvements.
On Figure 2.10, we observed that many microblogging users activated screen-based
notifications (feed always visible, or pop-up notifications), in order to be aware of new
updates in real-time. Those notifications can interrupt and distract users involved in a
primary task (e.g. writing a report), as they can happen any time, and possibly at a
frequent pace.

In some cases, interruptions are useful for this primary task, and can even facilitate task performance (Speier et al., 2003). However, in (Roda et al., 2006), it has been reported that interruptions can also ‘increase the load on attention and memory (Gillie & Broadbent, 1989), may generate stress (Bailey et al., 2001; Zijlstra et al., 1999), and compromise the performance of the primary task (Franke et al., 2002; McFarlane & Latorella, 2002; Nagata, 2003; Speier et al., 2003), especially when the user is working on handheld devices in mobile environments (Nagata, 2003).’ According to Miyata & Norman (1986), consuming unexpected messages (in our case: social updates) can be both beneficial and disruptive. They recommended that users can control the trade-off, to be able to accept diverted or not, depending on the context.

In their study on usage of email notifications thrown by MS Outlook, Iqbal & Horvitz (2010) observed that, even through notifications can imply costly focus interruptions of a primary on-going task, the lack of awareness induced by the disablement of those is also costly. Indeed, many users who were forced to disable email notifications, opened MS Outlook more frequently to check for incoming messages, in order to prevent having to process long chunks of emails later.

From this study and the results of our survey, we can infer that some users are
addicted to being notified of new information in real-time. We certainly believe that
emails are still the preferred way to send formal and decisive messages that can have
an impact on users' on-going tasks, especially in a professional context. This potential
impact can explain the users' need for being notified of new emails in real-time, and for
having to process long chunks of unread emails, whereas a microblogging user would not
mind skipping hundreds of updates. However, we support that social updates can be
important or useful for people's long-term decisions (e.g. career, goals; as seen on Figure
2.9), personal activities, and even synchronous group arrangements and announcements
while in mobility (e.g. canceled flight, selection of a new time and place to meet; as
usually done by exchanging SMS between only two people).
### 2.4 Conclusion

In this chapter, we analyzed the specificities and usage of social networking sites (SNS) and microblogging systems as communication systems, Facebook, Twitter and Myspace have been cited several times as examples. Then we studied recent usage trends concerning the attention given by microblogging users to real-time notifications, based on the results of a survey we conducted. We identified emerging needs for computer-supported filtering of social updates, either expressed by users through the survey, and by literature on cognitive psychology (Roda et al., 2006). In particular, context has been identified as a promising criteria for reducing the frequency of social updates to notify Iqbal & Horvitz (2010).
# Chapter 3

## From Awareness Systems to
Information management
techniques

In the previous chapter, we studied the usage of popular social networking systems, allowing people to keep in touch, for maintaining professional, personal, affective links in higher scales than it is possible naturally or with previous communication tools.

In this chapter, we survey research efforts also aiming at (or enabling to) creating and maintaining social ties, but also to enhance more specific kinds of interactions, such as collaborative work. In particular, we focus on:

* 'awareness systems', systems that allow people to be aware of the status of other people;

* and 'social matching systems', systems that allow to discover relevant people, and their activities.

After presenting a selection of awareness and social matching systems, we will study several techniques that can enable such systems, including information filtering and extraction techniques, and underlying similarity measures that can be leveraged. Then, we will identify existing techniques to improve the performance and scalability of awareness systems.
### 3.1 Awareness and social matching systems

#### 3.1.1 Definitions

Originally used by neurologists and psychologists (Sieb, 1990), the term of ‘awareness’ has been adopted by computer scientists as ‘an understanding of the activities of others, which provides a context for your own activity’ (Dourish & Bellotti, 1992), with the aim of supporting human awareness, communication and interaction with computer systems. More specifically, the Ambient Awareness project<fnref n="1" /> defines awareness systems as ‘Awareness Systems are computer mediated communication systems that help people maintain a peripheral awareness of others.’ According to Liechti (2000), Awareness Systems can be divided in four categories: group awareness, workspace awareness, contextual awareness and peripheral awareness. Those categories represent different goals, research disciplines and human interaction models.

Social matching systems, in the other hand, are defined by Terveen & McDonald (2005) as ‘a type of recommender system that bring people together rather than recommend items to people. They offer great potential to increase social interaction and foster collaboration among users within organizational intranets and on the Internet as a whole.’

In our case, we intend to enable awareness by filtering social information streams, assuming that users will accept to be notified only about updates that are the most relevant to their current activity and context. Notifications can be delivered to users through various modalities, including: dedicated computer screen areas, ‘toaster’/pop-up notifications on computer screens, mobile phones (e.g. vibration and screen), peripheral displays, and/or sounds. In this thesis, we focus on computer-mediated notifications, and intend to leverage context-based social matching techniques for identifying the most relevant updates to deliver.

#### 3.1.2 Foundations and some applications

In 1992, Dourish & Bellotti (1992) proposed the ‘shared feedback’ approach, consisting of tracking users’ actions on shared documents, and allowing them to comment those, in order provide useful context information and avoid work duplication.

<footnote n="1">¹Ambient Awareness project description: http://www.awareness.id.tue.nl/main.php</footnote>
Based on the Strauss’ Theory of Action, Fitzpatrick (1998) proposed the ‘Locales’
framework for supporting design of collaborative and communication applications. This
framework relies on three assumptions: (1) user’s actions constantly evolve based on the
actions of others, (2) interaction occurs within social worlds, and (3) the interactional
needs of users are important.

Following Fitzpatrick's principles and recommendations on civic structures, indi-
vidual views, interaction trajectory, mutuality, and embodied interaction from Dourish
(2001) (defined as *the creation, manipulation, and sharing of meaning through engaged
interaction with artifacts*'), Amelung (2005) developed the *Context-Aware Notification
System* ('CANS'). Extending the *Sakai* collaborative environment <fnref n="1" />, CANS acts as
a publish/subscribe system for delivering notifications on actions of other users of the
environment. Notifications are selected adaptively to the user's previous and current
activities, projects and preferences on the collaborative environment. The social con-
text used for generating and dispatching notifications relies on knowledge and actions
handled within the Sakai environment. The author recognizes that the Internet and
the physical world are additional social contexts that are also relevant to users. For
example, users might be interested to be notified when their colleague (or any other
person of interest) has arrived or left the office, as this information would impact the
communication and collaboration modalities with him.

Erickson & Kellogg (2000) experimented the social proxy as an abstract represen-
tation of activity in an enterprise communication space: ‘babble’. In these semiprivate
asynchronous discussion spaces, users are aware of the level of participation of their
colleagues on several ongoing entitled discussions, and can decide opportunistically to
join one of those. Additionally, common context between people has been identified
as an important factor to trigger new interactions (Vogiazou et al., 2003) (e.g. sign of
respect, conversation, or exchange).

As extensive usage of awareness systems can lead to ‘information overload’, Damian et al. (2007) claims that updates and notifications should be filtered by relevance to users’ tasks, in order to prevent users from ignoring too frequent updates potentially containing crucial information for the success of a project. Machine-interpretable contextual information can support awareness systems to identify to unplanned interactions (e.g. such as changes in the team members involved, which are very frequent in

<footnote n="1">¹Sakai: http://sakaiproject.org/</footnote>
collaborative development projects), and thus dynamically adapt the dispatching and presentation (e.g. notification modalities) of awareness information to users according to their current context (Liechti, 2000). Several awareness experiments have been carried out for collaborative bug tracking (Ellis et al., 2007) and configuration management of collaborative software development projects (Biehl et al., 2007; Sarma et al., 2008), in order to promote communication, reduce overlapping work, and improve ability to detect and resolve conflicts. In terms of visual summarization of awareness, Lin et al. (2008) propose an interactive and dynamic visualization framework for summarizing social networking interactions, and applied it with the Flickr social photo sharing site.

Delalonde (2007) developed a social matching application based on the social foundations of Actor-Network Theory. By formulating a query, the user is proposed a list of relevant people, based on their activity traces. User profiles are built by aggregating weighted keywords from the content they contributed, using TF-IDF (Term Frequency - Inverse Document Frequency, (Salton & McGill, 1986)). The recommended people are ranked using a 'ContactRank' measure which evaluation relies on the knowledge, relay, reputation and participation of users on keyword-based subjects. It is possible for users to collaborate through a wiki-based interface in which best solutions to a question are discussed by participants, similarly to Q&A systems (e.g. Yahoo!'s). The evaluation of this system was based on a vocabulary of 250 keywords, and revealed that the participation rate among users was too low to observe enough emerging collaborations. Two shortcomings were criticized by users: the keyword vocabulary was strict, preventing semantically close profiles to be leveraged, and user feedback mechanisms would allow users to improve the relevance of recommendations.

Furthermore, this study revealed that users will not necessarily be motivated to use a new tool. One explanation can be that information retention gives some power to employees, and they have no clear reason of being altruistic. Indeed, this behavior can hardly be expected between colleagues who have no common history, culture, nor context, especially in a high-scale enterprise environment. Providing more interaction history and transparency to users might ease their involvement with the system.

With 'Panorama', Vyas et al. (2007) proved that informal ties between co-workers are also essential to improve awareness, and thus better collaboration. In that case, computational support is not given to work-related content, but to casual content, so
that people decide to meet more naturally and eventually make each other aware of
professional activities.

## 3.2 Recommender systems and information filtering techniques

Our problem is to rank the relevance for a user U of Social Updates from every other user X, according to the similarity of contextual information between U and X. This problem can find some solutions in the domain of Recommender Systems (Adomavicius & Tuzhilin, 2005; Terveen & McDonald, 2005), which splits into two categories: content-based and collaborative recommendation system. We now study some specificities and relevant applications for each category, as possible ways to model and solve our problem.

### 3.2.1 Content-based filtering

Content-based filtering and recommender systems rely on users' preferences, provided explicitly by users or learned from a history of actions, to rank the potential interest/relevancy of new items, according to those preferences (user's profile).

Up to our knowledge, the closest existing solution to our problem is a web-based service and mobile application called My6sense<fnref n="1" />. This software can filter entries from RSS feeds and social streams (thus including social updates from Social Networking Systems) according to the user's preferences. This content-based filtering solution relies on a profile which contains the user's favorite (or at least usual) subjects of interest, and which is continuously evolving by tracking entries that are viewed or annotated by the user. However, despite the evolving design of user profiles, the filtering is not adaptive to the user's current context.

Under the assumption that user profiles and context can be inferred from the content of documents that are currently browsed by the user in a computer-centered environment, it is possible to run term frequency algorithms such as TF-IDF (Salton & McGill, 1986) on currently opened documents in order to identify keywords that represent the subject of this current browsing/editing activity. Following this approach, the Fab recommender system (Balabanovic, 1997; Balabanovic & Shoham, 1997) recommends web pages which most representing keywords match the user's profile, using a cosine

<footnote n="1">¹My6sense: http://www.my6sense.com/</footnote>
similarity measure on 100-keyword lemmatized and normalized vectors. However, this
method does not scale up when the number of users increase, because it is necessary
to compute similarities between every user.

With WordSieve, Bauer & Leake (2001) outperform TF-IDF in terms of precision,
but requires training data to tune several parameters. Dragunov et al. (2005) extended
the extraction of contextual information to support additional desktop document ma-
nipulation applications such as the Microsoft Office suite. With their PRUNE frame-
work, Kleek et al. (2009) model contextual information as entities (e.g. person, place
or resource) and events, allowing extraction for heterogeneous sources like RSS/ATOM
feeds, REST/JSON APIs, or even manual user entry. In the ‘Notes that Float’ appli-
cation, they employ this framework to attach contextual information to notes entered
by the user, so that they can be ranked to increase their visibility adaptively to the
relevance with the user’s current content, which relies on its similarity with the context
at the time these notes were added.

3.2.2 Social matching and collaborative filtering

Social matching systems consist of finding similar user profiles, in order to create com-
munication opportunities. Collaborative filtering and recommender systems rely on
social matching for finding similar users, and recommend items that are highly rated
by one to a similar user who has not yet rated those items.

Budzik *et al.* (2000) provide collaboration opportunities by recommending people that are browsing similar documents, based on a TF-IDF content analysis, users’ context being represented by weighted term vectors. These recommendations also include some information about people’s current activities, as identified by a software module that tracks users’ actions on their computer (e.g. chat sessions, running applications etc...).

Although its usage is more explicit and manual, social bookmarking is also a good
practice for discovering new content and people. On the popular del.icio.us <fnref n="1" /> social
bookmarking website, and its enterprise clone DogEar (Millen *et al.*, 2006), users book-
mark web pages that they like, can provide some ‘*tags*’ to annotate them and find them
easily, and share those annotated bookmarks on the web. Tags are vocabulary-free key-
words that can be selected, entered or even created (e.g. acronyms, personal keywords)

<footnote n="1">¹del.icio.us social bookmarking: http://del.icio.us/</footnote>
by users, relating to the page's subject, nature, ownership, category refinement, char-
acteristics, or for self reference, task organization Golder & Huberman (2005). The
association of tags from several users to every web site implies the emergence of a
crowd-sourced description of those web sites, and a classification based on tags (called
folksonomy). Each user can thus be profiled by the set of tags that he/she has given
to the most web pages, and those hyperlinked tags can be used as pivots to navigate
from page to users, and inversely. The tag clouds emerging on web pages and users give
promising collaborative recommendation applications (Hung et al., 2008; Niwa et al.,
2006). Furthermore, semantics can also emerge from statistical analysis of those tag
clouds (Specia & Motta, 2007; Tesconi et al., 2008; Wetzker et al., 2009).

Also relying on bookmarks, Agosto (2005) developed ‘*SoMeONe*’, a collaboration
tool that can recommend contacts (and their selection of web resources) to users by
identifying communities of interest around topics, using collaborative filtering. Topics
are extracted from the titles of bookmark categories stored by each user on SoMeONe.
Their sociological studies led to the ‘*SocialRank*’ algorithm which integrates social
factors such as reputation, relevance, originality, redundancy and reactivity of contacts.
However, the topics on which SoMeONe relies on, are statically defined by the Open
Directory Project <fnref n="1" />, which reduces the spectrum and granularity of topics that can
possibly be represented. Moreover, the authors identified that more ‘*awareness*’ was
expected from the users of their system: by notifying new recommendations, providing
explanations on why they were recommended, and allowing users to give more explicit
feedback.

Bielenberg & Zacher (2005), in their ‘groop.us’ prototype, have applied the same
approach while relying on tags (folksonomies) attributed to web pages by users (on
the delicious.com social bookmarking website) instead of hierarchical categories. After
clustering contacts by similar tags into groups, they identified the necessity of studying
the combination of tags from multiple social platforms (e.g. blog entries) instead of
relying too much on delicious.

3.2.3 Discussion

Similarly to Budzik *et al.* (2000)'s approach, we believe that opportunistic communica-
tion is key for awareness, and that the similarity of current working contexts (i.e. based

<footnote n="1">¹http://www.dmoz.org/</footnote>
on common concepts of documents being currently manipulated) is a great opportunity to propose such communication.

In response to our problem, collaborative filtering techniques seem to be more appropriate, as it consists of matching people with similar interests. However, in our case, interests are contextual, therefore they do not rely on items but on a dynamic set of features that can be extracted from an open range of information, including documents. Therefore, feature extraction techniques from content-based filtering are also to be leveraged, turning our system into a hybrid filtering system. This requirement fits into the category of the ‘collaborative via content’ filtering paradigm (Pazzani, 1999).

### 3.3 Content models and feature extraction techniques

In the previous section, we observed that filtering and recommendation systems rely on finding similarities between user profiles and/or content. Finding similarities in such data structures is not a trivial task (Quesada, 2008), and can be achieved at different levels:

* at the keyword (or tag) level, weights can be compared using various similarity functions, including the classic cosine similarity, or the contrast model (Tversky, 1977);

* or by measuring the similarity of semantic concepts (e.g. defined in ontologies, see Appendix A), either identified from plain text, emerging from rich tag sets (e.g. co-occurrence analysis) (Cantador, 2008; Specia & Motta, 2007; Yi, 2008), or provided by users (Damme et al., 2007; Tesconi et al., 2008).

#### 3.3.1 Keyword/phrase and tag-based models

Like in several projects surveyed in the previous section, CALVIN (Leake et al., 2000) relies on TF-IDF to extract the context of a user's task in the form of weighted vectors, by indexing the most frequent terms appearing in opened documents.

Higher abstraction terms can also emerge from textual content, e.g. concepts and/or topics possibly defined in specific taxonomies (e.g. thesaurus or vocabulary). KEA <fnref n="1" /> is a Java-based open-source library that can extract such keywords (or keyphrases) after a

<footnote n="1">¹Keyphrase Extraction Algorithm: http://www.nzdl.org/Kea/</footnote>
training phase in which documents are associated to the expected keyphrases. However, KEA can also be used without training data, implying a reduced output expressivity, but providing the benefits of a more advanced keyword extraction than just counting keywords (e.g. stemming, stop-word removal, 4-feature-based extraction).

On crowd-sourced information spaces (e.g. web-based social bookmarking systems), despite some issues related to semantic ambiguity and variant spelling of terms combined in folksonomies (Marlow et al., 2006; Mathes, 2004; Sinha, 2005), the increasing amount of tags given by Internet users to annotate digital resources (e.g. web pages tagged on delicious) naturally turned folksonomies into a good metadata representation and content classification scheme from which humans are able to recognize meaningful descriptions (Lamantia, 2008). Poor in quality (i.e. possibly ambiguous, structure-less, no common vocabulary) but rich in quantity and variety (e.g. openness to words from any language), tags have become good indexing features for filtering and recommender systems (Hotho et al., 2006; Niwa et al., 2006). Furthermore, with the growing use of twitter and geotagging applications in mobility, tags and natural words are now emerging from places, events and other real-world entities (Naaman & Nair, 2008), enabling those entities to be indirectly leveraged as features.

### 3.3.2 Semantic extraction

Relying on ontologies and subject-relation-object triples, the semantic representation of data prevents ambiguity of terms in natural language and enables powerful reasoning capabilities that can improve the performance of similarity measures in filtering and recommendation systems.

With the rise of the ‘Semantic Web’, its enabling technologies (e.g. RDF, microforms, see Appendix A) and the semantic federation of data (e.g. LinkedData<fnref n="1" />, which includes DBpedia², the semantic translation of Wikipedia³ (Auer & Lehmann, 2007)), several web services have been developed to extract semantics from plain text content, including Semantic Proxy ⁴, tagthe.net ⁵ and AlchemyAPI ⁶. Those services can be

<footnote n="1">¹LinkedData: http://linkeddata.org/
²DBpedia: http://dbpedia.org/
³Wikipedia: http://www.wikipedia.org/
⁴Semantic Proxy: http://semanticproxy.com/
⁵tagthe.net: http://tagthe.net/
⁶AlchemyAPI: http://www.alchemyapi.com/</footnote>
invoked by software developers to return a list of identified semantic instances from a
given text, e.g. people, places, events, technologies.

Semantics can also be inferred from statistical analysis of crowd-sourced features
(e.g. tags given by users on social bookmarking systems). Pera et al. (2009) built a word
similarity matrix by identifying correlations in library keywords, in order to broaden
the results of a search engine. By evaluating several term-based similarity measures
against the Wordnet lexical database <fnref n="1" />, Cattuto et al. (2008) observed that synonyms
and spelling variants could be discovered among large tag sets (which are heterogeneous
by nature in vocabularies and languages) using cosine similarity. Additionally, they
observed that FolkRank (Hotho et al., 2006) and co-occurence relatedness analysis
performed better to identify hierarchies of concepts (hyponym and hypernym relations
between tags).

At last, words (including tags) can be mapped to ontological concepts (Damme
et al., 2007; Tesconi et al., 2008). However, this method requires human support,
either from Internet users or from experts. In the first case, users are solicited for
providing the semantics of the tags they want to use to annotate some content. With
fact-tags (Amazon, 2008), users have to select the nature of their tags from a predefined
list of categories and relations: subjective review of an item, characteristic of an item,
comparison with another item. In Machine-tags (Cope, 2007), tags are to be prefixed by
a predicate and a namespace referring to an ontology or a vocabulary, similarly to the
RDF markup. In the second case, a mapping between words in natural language to the
corresponding semantic representation are to be maintained by experts. Databases like
Wikipedia or WordNet can be leveraged to generate a preliminary mapping (Szomszor
et al., 2008).

3.3.3 Discussion

In this section, we presented several models and techniques for finding similarities
between user profiles and items, either semantic or term-based. We conclude that tags
are good primary features to leverage, while keeping in mind that semantics can later
emerge from their quantity without soliciting users.

<footnote n="1">¹Wordnet: http://wordnet.princeton.edu/</footnote>
### 3.4 Improving performance and scalability

In the previous sections, we have seen that the performance of vector-based matching systems can suffer from their high dimensionality: similarities are complicated to find, and their computation hardly scale when the number of users increase. In this section, we identify several techniques that could potentially be applied to improve the performance and the scalability of our awareness system, in the future.

#### 3.4.1 Dimensionality reduction of term vector space

The performance of vector-based matching systems can be improved by reducing the dimensionality of the term vector space. In this regard, we have identified the possible use of incremental histogram-based statistics as an evolution of TF-IDF (Ciesielski et al., 2007), and of leveraging semantic relatedness of terms as in the ‘*Generalized Vector Space Model*’ (Tsatsaronis & Panagiotopoulou, 2009).

Latent Semantic Analysis can also reduce the dimensionality of term vectors, by removing noise and combining some of the dimensions towards a more conceptual classification (Hofmann, 1999). This technique (and its derivatives, including Latent Semantic Mapping (Bellegarda, 2007)) have been shown to improve the accuracy of a tag-based recommendation system (Wetzker et al., 2009). Nevertheless, we note that this technique requires to learn from training data.

#### 3.4.2 Tag clustering techniques

Hierarchical tag cloud clustering is proposed by Brooks & Montanez (2006) to cluster a set of tagged documents into a common/shared hierarchy of tags (based on tag co-occurrence). Emerging root nodes are general tags, whereas leafs are more specific tags. A similar approach was applied in the discerner and extender methods to transform concept maps into topic maps (Leake et al., 2003).

In our case, co-occurrent leaf tags between two users' tag clouds would imply a more precise relevance measure and should thus be highly ranked than co-occurrent root tags. Additionally, users' contexts (partly extracted from a set of documents) are dynamically evolving over time, making this algorithm too costly to run every time a new contextual tag cloud is processed. Therefore, an on-line clustering algorithm should be privileged
to enable the scalability of the system. Growing Neural Gas (Fritzke, 1995) is a good candidate according to these requirements (Stan *et al.*, 2008).

### 3.5 Conclusion

In this chapter, we have presented several awareness and social matching systems, surveyed content-based and collaborative filtering techniques which we decided to combine into a hybrid filtering system, chose to focus on a keyword/tag-based model instead of a semantic model, and identified potential optimization techniques that we could adopt in the future for improving the performance of our system.

In the next chapter, we study how users' context can be modeled and transformed into features that can be leveraged in our context-based filtering system.
# Chapter 4

## Extracting and modeling
contextual information

In this thesis, we support the hypothesis that, by sharing clues about their current context, one can implicitly solicit relevant information and communication opportunities, without having to make a phone call or to submit a query on the Internet.

In the previous chapters, we analyzed common user practices with social networking systems and studied the scientific techniques that can be leveraged for identifying relevant people and social updates, given a user profile.

In the aim of supporting the user in being notified of the most relevant signals, the challenges are to:

* identify what is the user's current task,

* determine which contextual entities are relevant to the execution of this task,

* and evaluate the relevance of social updates for a user, knowing his/her current task and context, and other users'.

In this chapter we will address the two first challenges listed above in order to build user profiles, by defining what we call ‘*context*’, studying the state-of-the-art of context-aware applications, context models and context management systems, and identifying context sensors of three kinds: physical, virtual, and social.
<figure><img src="image_41.png" /><figcaption>Figure 4.1: Contextual information: from raw sensor data to ambient intelligence - ©Bell Labs</figcaption></figure>
### 4.1 What is context-awareness?

Context-Awareness is an area of Computer Science which deals with the adaptation of computing systems to the user's current context. Introduced by (Schilit et al., 1994), the most recognized definition for the term 'context' was proposed by (Dey, 2000): 'Context is any information that can be used to characterize the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and applications themselves.' By being 'aware' of the user's context, a computing system can make deductions about the current situation of the user, from various hardware sensors such as GPS receiver (i.e. for positioning), surrounding Bluetooth equipments (and thus, surrounding people), and even software sources (called virtual sensors) like the user's agenda and the list of currently opened documents. As shown on Figure 4.1, by combining (i.e. aggregation of context data from several sensors) and inferring on this sensed information (i.e. interpretation, abstraction or enhancement of context), a meaningful knowledge can be derived, for instance position or activity like 'in a meeting' or 'watching TV' can be deduced.

Most research projects leverage the context of users of a system in order to adapt the interaction between the user and the system (Christopoulou, 2008), or for the system to predict users' goals and activities (Oliver et al., 2002; Snoeck, 2007; Tapia et al., 2004), and pro-actively make decisions intended to support the user in his future contexts (Nurmi et al., 2005; Wang et al., 2004a). Such applications include Intelligent Meeting Rooms (Chen et al., 2004a; Leong et al., 2005), Smart Homes (Gu et al., 2004b), Personalized mobile advertisement (Zhdanova et al., 2006) and electronic healthcare (Broens et al., 2007). With the emergent popularity of augmented reality application on mobile devices, contextual information can also be displayed directly to users. Baldauf & Simon (2010) proposed to represent context in the form of 'ambient tag clouds' which evolves while the user moves from a place to another, based on the geographical distance between his/her device and geo-referenced content.

In our case, we consider context as sampled information about people's environment and actions in time, and we propose to leverage this context to give relevant opportunities for users to communicate with each other. The aim is not to adapt human-system interaction, or to envision a pro-active system for user support, but to create relevant
links between people. Several context-aware social applications have been developed (Gaonkar et al., 2008; Koolwaaij et al., 2006), but the design of these applications was based on context-awareness (bottom-up approach), whereas we decided to rely on existing (and successful) SNS and their specific communication paradigm (top-down approach) in order to improve user acceptance.

## 4.2 Context management systems

### 4.2.1 From context-aware applications to context management systems

The first applications relying on context appeared in the 1990's, with the ‘*Active Badge*’ location system (Want *et al.*, 1992) and the ‘*ParcTAB*’ mobile computing system (Schilit *et al.*, 1993). At that time, context solely consisted of location information, applications were driven by a set of rules based on location changes, and relying on a specific sensor infrastructure.

After Dey *et al.* (2001) promoted Context Toolkit as a generic context management toolkit that could be leveraged by various context-aware applications as an intermediate re-usable software layer, researchers explored several ways to structure contextual information: key-value models, markup scheme models, graphical models, object-oriented models, logic-based models and ontology-based (semantic) models. Gu *et al.* (2004b) expressed the following requirements for context management systems: ‘*An appropriate infrastructure for context-aware systems should provide support for most of the tasks involved in dealing with contexts - acquiring context from various sources such as physical sensors, databases and agents; performing context interpretation; carrying out dissemination of context to interested parties in a distributed and timely fashion; and providing programming models for constructing of context-aware services.*’

Strang & Linnhoff-Popien (2004) concluded that ontology-based models (as introduced in Appendix A) were best suited for expressing context, according to six requirements towards the enablement of the ubiquitous computing vision: distributed composition, partial validation, richness and quality of information, incompleteness and ambiguity, level of formality and applicability to existing environments. At that point, most research on context-awareness was based on semantic context management systems (Baldauf *et al.*, 2007; Lassila & Khushraj, 2005).
### 4.2.2 Semantic context management systems

The use of ontologies (see Appendix A) to store and manipulate context have an impact on other aspects of the underlying system: context knowledge exchange, learning, user interactions, security (especially for privacy control) and applications. In this subsection we will review several semantic-based approaches for context management systems and identify the most critical lacks to overcome towards our goal.

One of the first semantic context modeling approaches was the Aspect-Scale-Context (ASC) model proposed by Strang *et al.* (2003). Compared to non-semantic models, ASC enabled contextual interoperability during service discovery and execution in a distributed system. Indeed, this model consists of three concepts:

* Aspects are measurable properties of an entity (e.g. the current temperature of a room)

* Scales are metrics used to express the measure of these properties (e.g. Celsius temperature)

* Context qualifies the measure itself by describing the sensor, the timestamp and quality data

Contexts can be converted from a scale to another using Operations, also described semantically, and can be mapped to an implemented service. This model has been implemented as the CoOL Context Ontology Language. The CoOL core ontology can be formulated in OWL-DL (Dean & Schreiber, 2004) and F-Logic (object-oriented). The CoOL integration is an extension of the core to inter-operate with web services. OntoBroker (Decker *et al.*, 1999) was chosen for semantic inference and reasoning, supporting F-Logic as knowledge representation and query language.

With EasyMeeting, Chen *et al.* (2004a) proposed a pragmatic application to demonstrate the benefits of their semantic context-aware system called CoBrA, for Context Broker Architecture. This application assists a speaker and its audience in a meeting situation by welcoming them in the room, dimming the lights, and displaying the presentation slides, either by vocal commands or automatically. The underlying prototype that they developed is a multi-agent system based on JADE <fnref n="1" /> in which a broker maintains a shared context model for all computing entities by acquiring context knowledge

<footnote n="1">¹Java Agent Development Framework: http://sharon.cselt.it/projects/jade/</footnote>
<figure><img src="image_42.png" /><figcaption>Figure 4.2: Overview of CoBrA - (Chen *et al.*, 2004a)</figcaption></figure>

from various sensors and by reasoning on this knowledge to make decisions, as depicted
on Figure 4.2. In the EasyMeeting application, this broker can deduce the list of ex-
pected participants and their role in the meeting by accessing their schedule, and can
sense their actual presence when the bluetooth-enabled mobile phone declared in their
profile is detected in the room. That way, the system can notify the speaker about their
presence, decide to dim the lights and turn off the music when he arrives. These deci-
sions are made possible by reasoning on the context knowledge using rules defined by
the EasyMeeting application. The context knowledge is represented as RDF triples re-
lying on the COBRA-ONT OWL ontology that includes vocabularies from the SOUPA
ontology (Chen *et al.*, 2004b) covering time, space, policy, social networks, actions,
location context, documents, and events, as depicted on Figure 4.3. Inferencing on the
OWL ontology is handled by Jena's API<fnref n="1" /> whereas the Jess rule-based engine<fnref n="2" /> is used
for domain-specific reasoning. The execution of rules (when results cannot be inferred
from ontology axioms alone) uses the forward-chaining inference procedure of Jess to
reason about contextual information. Note that, in this case, essential supporting facts
must be extracted from RDF to JESS representation and the eventual results have to
be injected in RDF to the knowledge base, which implies additional overhead in the
process.

<footnote n="1">¹Jena Semantic Web Framework: http://jena.sourceforge.net</footnote>

<footnote n="2">²Jess rule engine: http://herzberg.ca.sandia.gov/</footnote>
CoBrA's broker enforces privacy policies to define rules of behavior and restrict context communication. The enforcement of user-defined policies relies on the Rei role-based policy-reasoning engine (Kagal *et al.*, 2003) which does description logic inference over OWL. CoBrA also implements a meta-policy reasoning mechanism so that users can override some aspects of a global policy to define specific constraints at their desired level of granularity. However, they do not provide a tool for the user to express his/her privacy policy.

<figure><img src="image_43.png" /><figcaption>**Figure 4.3: The SOUPA ontology** - (Chen *et al.*, 2004b)</figcaption></figure>

The SOUPA ontology proposed by Chen *et al.* (2004b) and used in CoBrA was a collaborative effort to build a generic context ontology for ubiquitous systems. Since 2003 it has been maintained by the 'Semantic Web in Ubiquitous Comp Special Interest Group'. The design of this ontology is driven by use cases and relies on FOAF, DAML-Time, OpenCyc (symbolic) + OpenGIS (geospatial) spatial ontology, COBRA-ONT, MoGATU BDI (human beliefs, desires and intentions) and Rei policy ontology (rights, prohibitions, obligations, dispensations). SOUPA defines its own vocabulary, but most classes and properties are mapped to foreign ontology terms using the standard OWL ontology mapping constructs (*equivalentClass* and *equivalentProperty*), which allows interoperability. In the core ontology in which both computational entities and human users can be modeled as agents, the following extensions are added: meeting & schedule,
document & digital document, image capture and location (sensed location context of things).

Like CoBrA, MOGATU (Perich et al., 2005) is a context-aware system based on the SOUPA ontology. However, this decentralized peer-to-peer multi-agent system implements several use cases covering automatic and adaptive itinerary computation based on real-time traffic knowledge, and commercial recommendation. In this approach, each device is a semi autonomous entity driven by the user's profile and context, relying on a contract-based transaction model. This entity is called InforMa and acts as a personal broker that handles exchanges with other peers. The user profile semantically defines his beliefs, desires and intentions, following the BDI model that is part of the SOUPA ontology. Beliefs are weighted facts depicting user's knowledge and preferences such as his schedule and food preferences, whereas desires express the user's goals. Intentions are defined as a set of intended tasks that can be inferred from desires or explicitly provided. However no clues are given by the authors about how these beliefs and intentions are defined by the user or the system, which let us assume that this is still a manual process yet to be enriched with profiling mechanisms and a graphical user interface to edit the profile. Moreover, this work being apparently focused on trusted peer-to-peer exchange of information according to the BDI user profile, details on the actual reasoning process on context knowledge are not given. InforMa is able to process queries that can possibly involve other peers and advertise information to these peers in vicinity, relying on graph search and caching techniques. However no details were given on how pro-activity is made possible. Another lack identified in the underlying BDI model is that the representation of pre-conditions and effects of intentions are left to the applications, but we have found no clues on how applications fill this issue. Facing an important cost of network transmissions in the exchange process, it seems that this research group is focusing on peer-to-peer networking optimization and trusted exchanges more than on the actual context management. However, they suggested that preparing purpose-driven queries in advance and caching intermediate query results could improve the performance of their system, which is an interesting approach that should be considered in distributed context-aware systems.
### 4.2.3 Uncertainty and quality of context

The CORBA-based GAIA platform proposed by Ranganathan et al. (2004) focuses on hybrid reasoning about uncertain context, relying on probabilistic logic, fuzzy logic and Bayesian networks. In their approach, context knowledge is expressed using predicates which classes and properties are defined in a DAML+OIL ontology (Horrocks, 2002). Predicates can be plugged directly into rules and other reasoning and learning mechanisms for handling uncertainty. This choice reduces the overhead of the CoBrA system relying on RDF triples. Rules are processed by the XSB engine <fnref n="1" />, which is described as a kind of optimized Prolog that also supports HiLog, allowing unification on the predicate symbols themselves as well as on their arguments. HiLog's sound and complete proof procedure in first-order logic is needed to write rules about the probabilities of context.

GAIA's authentication mechanism demonstrates the usefulness of fuzzy/uncertain context reasoning. It allows users to authenticate with various means such as passwords, fingerprint sensor or bluetooth phone proximity. Each of these means have different levels of confidence, and some user roles may require that the user authenticates himself on two of them to cumulate their confidence level up to the required level.

Although GAIA proposes a common reasoning framework, application developers have to define the expected context inputs and specify the reasoning mechanism to be used by providing Prolog/HiLog rules (for probabilistic/fuzzy logic) or Bayesian networks. A graphical user interface is provided to help developers construct rules, whereas MSBN (Microsoft's Belief Network) can be used to create Bayesian nets. Although Bayesian networks are a powerful way to perform probabilistic sensor fusion and higher-level context derivation, they need to be trained. Moreover, inference with large networks (more than 50 nodes) becomes very costly in terms of processing and can result in scalability problems.

Based on previous works, Gu et al. (2004b) propose SOCAM (Service-Oriented Context-Aware Middleware), another OWL-based context-aware framework with the aim to address more general use cases by adding more qualitative information on acquired context. The classifiedAs property allows the categorization of context facts as

<footnote n="1">¹XSB: http://xsb.sourceforge.net/</footnote>
<figure><img src="image_44.png" /><figcaption>Figure 4.4: Partial definition of the CONON ontology extended with the home domain - (Wang *et al.*, 2004b)</figcaption></figure>

Sensed, Defined, Aggregated or Deduced. The dependsOn property allows the justification of a deduced context based on other context facts. Another contribution is the possibility to qualify context information with parameters such as accuracy, resolution, certainty and freshness. The SOCAM framework was proven (Gu *et al.*, 2004a) to reason successfully on uncertain contexts using Bayesian Networks, but no performance results were given. The same group of authors have also carried out a performance experiment of the CONON ontology (Wang *et al.*, 2004b) depicted on Figure 4.4, which is the name that was given to SOCAM's context ontology. Their results show that the duration of the reasoning process exponentially increases with the number of RDF triples stored in the context knowledge base, which reveals that this approach is not scalable for a widespread context-aware system. Therefore two leads were proposed to increase performance:

* to perform static, complex reasoning tasks (e.g., description logic reasoning for checking inconsistencies) in an off-line manner.

* to separate context processing from context usage, so that context reasoning can be performed by resource-rich devices (such as a server) while the terminals can acquire high-level context from a centralized service, instead of performing excessive computation themselves.
Later works of that team were focused on the peer-to-peer architecture for context information systems.

Basing on the CONON ontology, Truong *et al.* (2005) proposed the PROWL language ('Probabilistic annotated OWL') to generalize fuzzy/probabilistic reasoning from applications to domains by mapping Bayesian Networks to ontology classes and properties. This approach must be experimented with various context-aware applications to prove its feasibility.

The FP6 IST project SPICE (Service Platform for Innovative Communication Environment) brought a fresh approach to ubiquitous system, considering them in a wider scope centered on semantic knowledge management for improved ubiquitous end-user services (SPICE, 2006, 2007). On its Knowledge Management Layer, SPICE proposes two different implementations of the context provisioning subsystem: the IMS Context Enabler (ICE) (Strohbach *et al.*, 2007) and the Knowledge Management Framework (KMF). In ICE, the SIP protocol (Session Initiation Protocol) is leveraged to control the parameters of the exchange sessions (e.g. data sets to communicate, update trigger, update frequency) and to flexibly adjust the communication path based on the changes in network structure and available context information. Both KMF and ICE rely on a shared ontology called the Mobile Ontology which is freely downloadable on the Internet <fnref n="1" />, the most important difference being the interfaces: ICE uses SIP whereas KMF uses OWL over SOAP Web Services for exchanging context information. However, gateways are also provided so that context data can be converted from a format to the other. Therefore we will abstract these implementations and focus on the common knowledge model. Embracing the recommendations of the W3C, SPICE Mobile Ontology is defined in OWL and the context data is expressed in RDF. Inspired from the Dutch project Freeband Awareness, SPICE's Physical Space ontology has a finer granularity than any previous context ontology: it notably defines properties for connections between rooms and floors. Following the approach of the 'Doppelgänger User Modeling System' (Orwant, 1995), SPICE's User Profile ontology supports domain-specific and conditional (situation-specific) submodels. In this approach, the profile contains subsets which are considered on certain conditions expressed with the form: Context Type, Operator, Value. This allows variations of the profile, depending on the user's context and/or the targeted application/service.

<footnote n="1">¹SPICE Mobile Ontology: http://ontology.ist-spice.org/</footnote>
The Knowledge Management Layer also contains a Knowledge Storage module, a Profile Manager, a Service and Knowledge Push and Notification module and three kinds of Reasoners: a Predictor, a Learner and a Recommender. The reasoners can request past knowledge directly from context sources or from an external knowledge storage source. Both feedback-based and observation-based learning are supported, generating *LearntRule* and *LearntRuleSet* instances in OWL. The results can be leveraged to propose Recommendations to the user. Experimental results on the use of different learning techniques are to be published. Another interesting contribution of SPICE in the context-awareness domain is the use of a *KnowledgeParameter* class that is used to qualify context information with values defining their probability, confidence, timestamp, temporal validity and accuracy. However we have not found any mechanism that is similar to the *dependsOn* property supported by SOCAM to justify high-level context with lower-level facts from which it was inferred.

<figure><img src="image_45.png" /><figcaption>Figure 4.5: Creating a rule-based service using SPICE’s End User Studio -
(SPICE, 2007)</figcaption></figure>

Another part of the SPICE project called the Distributed Communication Sphere (Kernchen et al., 2007) allows dynamic discovery of users’ surrounding devices, networks and services. This part includes components that leverage context knowledge to enable multimodal interaction, content delivery, data synchronization and dynamic widgets on terminals, requiring a lightweight rule engine to be deployed on every terminal. SPICE also provides the End User Studio, an Eclipse-based GUI (Graphical User Interface) shown on Figure 4.5 that allows end users to create custom trigger-action rules visually.
### 4.2.4 Popularization of context-aware applications

Since the middle of 2008, new open technologies have emerged on the Internet, providing location information to web pages (e.g. Google's Geolocation API <fnref n="1" />, Yahoo! Fire Eagle <fnref n="2" />) and mobile applications running on smartphones (e.g. Apple iPhone <fnref n="3" />, Android <fnref n="4" />) for free. Developing location-based application have thus become easy for developers, even beyond the industry and scientific communities. Contrarily to scientific efforts on context-awareness systems, numerous location-aware applications were brought to end-users (e.g. cell-based and GPS-based positioning in Google Maps mobile <fnref n="5" />, Qype Radar <fnref n="6" />, Layar <fnref n="7" />...) without relying on complex context models nor context management platforms. These applications have been very usage-specific, but also very functional, effective, and easy to use for a broad population.

Also in 2008, the Pachube website <fnref n="8" /> proposed Internet users to share sensor data (e.g. temperature, humidity...) in real-time on their web platform, so that it could be leveraged by other users and applications. Today, Pachube can act as a simplified context management framework, allowing users to define rules that can trigger several kinds of actions depending on specified contextual conditions: notify people through their phone, notify applications or even activate devices and appliances. This platform can be seen as a first open and functional realization of the ‘Internet of Things’ vision. Nevertheless, this platform does not rely on a semantic context model. Sensor data is handled as a neutral data stream, that are described with a name, a unit and tags for the sole sake of visual representation and classification of streams on the Pachube website.

Following an extensive review of context models proposed with context management systems from the scientific community, Bolchini et al. (2007) observes that the intended generality and expressivity of semantic models for any possible application actually reduce their usability. The growth of numerous specific web-based and mobile

<footnote n="1">¹Google's Geolocation API: http://code.google.com/apis/gears/api_geolocation.html</footnote>

<footnote n="2">²Yahoo! Fire Eagle: http://fireeagle.yahoo.net/</footnote>

<footnote n="3">³iPhone: http://developer.apple.com/iphone/library/navigation/index.html#section=Frameworks</footnote>

<footnote n="4">⁴Android Developers - Location and Maps: http://developer.android.com/guide/topics/location/</footnote>

<footnote n="5">⁵Google Maps for mobile: http://www.google.com/mobile/maps/</footnote>

<footnote n="6">⁶Qype Radar: http://www.qype.co.uk/go-mobile</footnote>

<footnote n="7">⁷Augmented Reality Browser: Layar: http://www.layar.com/</footnote>

<footnote n="8">⁸Pachube: http://www.pachube.com/</footnote>
applications that can leverage location information without relying on context man-
agement platforms and models, support this conclusion.

4.2.5 Privacy concerns

As context management systems gather information about their users (e.g. their loca-
tion), secure privacy enforcement mechanisms are to be implemented to avoid leaks of
private information to unexpected parties (Lederer et al., 2003).

The Context Toolkit from Dey (2000) features custom owner permissions to protect private sensor data by specifying access rights to authorized parties. Applications have authenticate to sensors, in order to access their data, through a public-key infrastruc- ture. Most of its successors, such as CoBrA (Chen et al., 2004a), Gaia (Ranganathan et al., 2004) and SPICE (SPICE, 2006, 2007) rely on rule-based security policies for enforcing access rights to context information about users. In all cases, specific privacy- critical contexts have to be predicted to generate rules, which would be a tedious task for end users to execute.

According to the classification by Dourish (2004), the context-awareness frameworks we are reviewing here fall in the category of ‘Architectures for Adaptation’ in which information structures are designed on ‘predefined patterns’, without possibility for users to visualize nor manipulate their own context. Therefore, we can argue that the applications enabled by these systems require users to fit those predefined patterns, and to trust the system concerning the use of their private information, hoping without warranty that their privacy will not be violated.

The necessity for such rule-based policies, however, is implied by the seamlessness
(i.e. no visibility of context, nor interaction expected with the context-aware applica-
tion in this regard) that context-aware application intend to guarantee to their users.
Dourish (2004) criticizes the lost of control (and thus, trust) from users, caused by this
seamlessness: ‘*The flipside of the "seamless access" to which most mobile systems aspire
is the systematic hiding of network heterogeneity; but it is precisely that heterogeneity
that determines users' assessment of security.*’

Therefore we conclude that, in order for users to trust a context-based system, some
control must be given back to them, even if this reduces seamlessness. We agree that
context information should be structured, represented and integrated in a way that
allows visualization, manipulation and appropriation by the users.
4.2.6 A problem of scalability

<figure><img src="image_46.png" /><figcaption>Figure 4.6: HCoM: Hybrid Context Management and reasoning system - (Ejigu *et al.*, 2007)</figcaption></figure>

In previously reviewed semantic context-aware systems, we identified that the com-
plexity of semantic inferencing and reasoning mechanisms imply a scalability problem.
In response to this issue, Tan *et al.* (2005) propose to move from on-demand context
reasoning to event-driven context interpretation so that reasoning on context data is
processed as soon as it is received by the context management framework. Ejigu *et al.*
(2007) proposed an hybrid context management and reasoning system (HCoM) which
relies on a heuristic-based context selector to filter the context data to be stored in
the semantic context base for reasoning, the rest being stored in a relational database,
as depicted on Figure 4.6. They report that this approach is more scalable than pure
semantic context-awareness systems when the number of static context instances in-
creases. Lin *et al.* (2005) propose a similar approach but they filter context data
according to their relevance to running applications instead of usage heuristics, in or-
der to boost the reasoning performance. However, in their distributed system, the
performance is reduced because of increased communication overheads. Moreover, it
does not support uncertainty yet.
## 4.2.7 Discussion

In this section, we have reviewed several approaches addressing modeling, distribution and reasoning on contextual knowledge. Although semantic technologies have been shown as powerful enablers for empowering interoperable and generic context-awareness systems that can support heterogeneous sensors and applications, they also imply scalability, privacy and usability problems.

Concerning scalability, the complexity of semantic inference and reasoning mechanisms imply that required processing time grows exponentially with the amount of knowledge, which is a major issue towards the realization of context-aware applications. Hybrid context management approaches can reduce the amount of semantic computations, but as long as semantic reasoners with non-linear complexity are being used it is not possible to assert that these systems are scalable.

Concerning privacy protection, several security mechanisms have been implemented on context management systems for enforcing it. However, in the sake of seamlessness intended for context-awareness applications, these mechanisms rely on rule-based policies. As policies can differ for each user, they are expected to customize them, which is a complex task requiring them to anticipate privacy-sensitive situations before they occur. A trade-off is to be discussed between seamlessness and user intervention in privacy control mechanisms.

Concerning usability, it seems that, by intending to maximize genericity and extensibility, context-awareness researchers neglected important human implications: these systems and their applications could not satisfy end-users because they would bring too less effectiveness comparing to the efforts expected from these users (e.g. trust to a system sharing private information, difficulty of use). On the other hand, it has been observed that several context-awareness solutions (e.g. Location APIs, Pachube...) have been opened to the public on the Internet, giving birth to numerous pragmatic and usable end-user applications, without requiring a complex context model, nor a context management system. Several of these applications have met a very broad public, even if some people do not use them for privacy protection reasons. The trend is that each application (and service) has one specific purpose, and explicitly requires (or provides) specific context information. Users are free to combine several applications and services
(or create combined applications, ‘mash-ups’) if they need to leverage more contextual
information about them.

4.3 Physical sensors: Extracting and modeling geograph-
ical contexts

Geographical positioning has been one of the most considered category of contextual
information. Over the years, technological advances enabled new ways to determine
one’s geographical location, each with specific quality and constraints.

Locations can be defined in several ways:

* Absolute geographic location: absolute coordinates of an object on Earth (latitude and longitude)

* beacon-based location: estimated position of an object based on its distance with at least one positioned beacon (e.g. a radio antenna including GSM, WIFI, Bluetooth or even RFID/NFC access points)

* movement-based relative location: estimated position of an object based on movement information, assuming that the position of the starting point is known, and that the actual position of the object is checked regularly

* semantic/structural/political location: this category does not address how to determine a location but what it means to users of the system. A location can have a political meaning (e.g. country names), a structural meaning (e.g. level in a building) or any other semantic (e.g. my grandfather's farm).

In this section, we present several methods to sense, model and aggregate contextual
clues in order to leverage the resulting location information in context-aware software.

4.3.1 Absolute geographic location

The absolute geographic location of an object consists in a latitude, a longitude and
possibly an altitude, which are represented by floating-point decimal values.

Such coordinates can be provided by GPS <fnref n="1" />. This system consists of a set of 24
satellites that orbit around the Earth, and that receivers can rely on to determine

<footnote n="1">¹Global Positioning System</footnote>
their position in real-time, when in range. The usual precision of most GPS devices is 10-20 meters, and 1 meter for D-GPS (Differential GPS). The main constraint for this technology is the quality of reception from the satellites. A GPS receiver requires a clear line of sight to the sky, and must be able to intercept signals from a minimum of 3 or 4 satellites simultaneously, in order to determine the current geographical coordinates in 2D or 3D, respectively. These constraints make it difficult to resolve indoor locations, because walls highly reduce the strength of signal from satellites. Additionally, GPS connectivity is costly in terms of energy, and can be slow to provide a first position when turned on. Nevertheless, this technology has become very usual, from driving navigation devices to most recent smartphones.

Geographical coordinates can be formatted using the Basic Geo Vocabulary ontology<fnref n="1" /> proposed by the W3C Semantic Web Interest Group. This ontology defines a *point* class (extending the *SpatialThing* class) with the following properties:

* *lat* to store the latitude

* *long* to store the longitude

* *alt* to store the altitude

Note that GPS receivers also provide the following information: *gpsPositionDilution*, *gpsHorizontalDilution*, *gpsVerticalDilution*, *gpsSpeed*, *gpsUtctime*, *gpsHeading*,

### 4.3.2 Beacon-based location

Whereas a GPS receiver can provide an absolute geographic position directly to applications, indirect alternatives also exist to determine a location based on other sensors. Because they are relying on fixed beacons, wireless networks can help. By beacon, we can mean: GSM cells (including Femtocell), fixed WIFI access points, fixed Bluetooth devices and fixed RFID/NFC readers. Mobile wireless devices can also be leveraged to determine a location, as long as their current absolute location is known by the system in real-time or near-real-time.

Note that, infrastructure-based location mechanisms, also relying on wireless network access points, can provide one's approximate position directly to location-aware

<footnote n="1">¹Basic Geo Vocabulary ontology: http://www.w3.org/2003/01/geo/</footnote>
services, like a GPS would, but this approach is thus equivalent to the previous *abso-lute geographic location*. In this part, we will focus on client-based location mechanisms relying on beacons.

<figure><img src="image_47.png" /><figcaption>Figure 4.7: Beacon-based positioning -</figcaption></figure>

In opposition to absolute geographic locations, and as depicted on Figure 4.7, the beacon-based location of an entity would be defined by one or more *isNearBy* associations between *SpatialThing* instances (including the entity which position is evaluate, and beacons in range). The geographical distance of those instances can be represented in a *proximity* attribute, or a *distance* attribute.

The *proximity* attribute can represent the signal strength between the entity's wireless network interface and each beacon. Such data can then be interpreted (e.g. by triangulation or fingerprinting algorithms) in order to compute the absolute position of the entity, assuming that the coordinates of the beacons are known.

In the case of a NFC/RFID, swiping a chip on a reader is equivalent to being in range of a wireless beacon, with a near-zero distance.

Cell-based location

In opposition to GPS positioning, cell-based (e.g. GSM) positioning is also possible indoors, as long as beacons are in range. Although this technology is less precise than GPS positioning, it is natively usable with cell phones without additional hardware and it's less power consuming.

Cell-based localization is the use of multi-lateration to determine the location of cell phones, usually with the intent to locate the user. Multi-lateration, also known as hyperbolic positioning, is the process of locating an object by accurately computing the time difference of arrival (TDOA) of a signal emitted from the object to three or more receivers. It also refers to the case of locating a receiver by measuring the time
## 4. EXTRACTING AND MODELING CONTEXTUAL INFORMATION

difference of arrival (TDOA) of a signal transmitted from three or more synchronized transmitters. Multi-lateration should not be confused with tri-lateration, which uses absolute measurements of time-of-arrival from three or more sites.

Cellular phones can provide the identifier of the cell (antenna) on which the phone is connected. This information is useful to provide a rough but reliable location in most areas, as long as they are covered by the cell infrastructure.

Chen et al. (2006) experimented the use of a proposed client-based GSM positioning system in residential (26 cells/Km²) and downtown (66 cells/Km²) areas of Seattle. They utilized three different methods: the centroid algorithm, fingerprinting and Monte Carlo (with Gaussian processes signal models) to estimate their position and evaluate the error.

<table><caption>**Figure 4.8: Comparison of three GSM-based positioning methods** - (Chen et al., 2006)</caption><thead><tr><th></th><th>Storage Required (for Seattle, compressed)</th><th>CPU Usage</th><th>Accuracy (Dense Towers)</th><th>Accuracy (Sparse Towers)</th><th>Required Density of Training Data</th><th>Requires Same-Device Training Set</th><th>Benefits from Cross-Provider Scanning</th><th>Tolerant of Phones Exposing Single Cell</th></tr></thead><tbody><tr><td>Centroid</td><td>Low (44KB)</td><td>Low</td><td>232m</td><td>760m</td><td>Low</td><td>No</td><td>Yes</td><td>Yes</td></tr><tr><td>Fingerprinting</td><td>High (188MB)</td><td>Med.</td><td>94m</td><td>277m</td><td>High</td><td>Yes</td><td>No</td><td>No</td></tr><tr><td>Gaussian Processes</td><td>Med. (80MB)</td><td>High</td><td>126m</td><td>196m</td><td>Med.</td><td>No</td><td>Yes</td><td>Yes</td></tr></tbody></table>

As shown on Figure 4.8, they realized that the fingerprinting method was the most accurate in downtown areas (dense towers), whereas Gaussian Processes were the most accurate in residential areas (sparse towers). Their experimental results show that existing GSM devices can achieve a positioning accuracy with a median error of 94 to 196 meters using cells from a single provider. Furthermore, they have identified an opportunity to significantly improve accuracy by scanning cells across all available providers, for a medium error of 65-134 meters, which is a factor of 3-4x of the published accuracy for WiFi beacon-based positioning projects.

### WiFi-based location

Nowadays, technologies for wireless LAN are going through an implementation boom. Numerous wireless network providers are installing their systems in hotels, cafes, airports and other kind of buildings in which a facility of high speed Internet access is considered profitable.
In the area of tourism and cultural heritage, WiFi has been deployed experimentally in museums, archaeological excavations, hotels, fun fairs; obviously with the main aim of providing interesting services related to the user localization at any moment (e.g. providing more information about surrounding artefacts).

Wireless networks are widespread 75 meters inside buildings, and 300 meters outdoor, potentially attaining some kilometers by using specific antennas.

The WIFI connectivity of a device, similarly to cell infrastructures, can be used to locate it relatively to hot spots and other WIFI transmitters. This localization is made possible by triangulation or fingerprinting of signal strength with the surrounding WIFI devices:

* In the case of infrastructure-based coordinates triangulation, access points (APs) and their controllers detect visiting devices. The devices are seen with a certain signal strength by each AP; the APs also know their precise coordinates. Using the signal strength and coordinates of each AP, it is possible to determine the coordinates of the device. The advantage of this method is to map to any coordinates and locate any device. The drawback is to require complete control over the WiFi infrastructure. (Thornycroft, 2009)

* With device-based WiFi fingerprinting, the device itself learns to recognize sets of signal strengths from its surrounding APs, and to deduce positions where it has already been. The advantage of this method is to locate anywhere without controlling the WiFi environment. The drawback is to require intelligence on the device, and also a learning phase.

### Bluetooth-based location

Bluetooth is an industrial specification for wireless personal area networks (PANs). Bluetooth provides a way to connect and exchange information between devices such as mobile phones, laptops, PCs, PDAs, printers, digital cameras, and video game consoles over a secure, globally unlicensed short-range radio frequency. The Bluetooth specifications are developed and licensed by the Bluetooth Special Interest Group<fnref n="1" />.

<footnote n="1">¹Bluetooth Special Interest Group: https://wwwbluetooth.org/</footnote>
The main use of Bluetooth is for short range (about 10 meters) connections between
devices (e.g. PDA's, mobile phones and laptops). This technology is suitable for rela-
tively static contexts, in which it is difficult to set a wireless network in a conventional
way (e.g. inside cars).

Thanks to their unique MAC address, fixed Bluetooth devices can be used to ‘tag’
(or check in) a place so that a visiting user can be located in that place.

Hybrid / Collaborative positioning

The Navizon <fnref n="1" /> mobile service proposes a hybrid approach for positioning: it uses location knowledge of surrounding peers to locate devices when unable to use GPS localization. This knowledge can be extracted either by GPS, cell or WiFi positioning, depending of the capabilities of each participating device. This technology empowers a commercial product called ‘*Virtual GPS*’ which returns approximate GPS coordinates using the surrounding Navizon peers, as if the host device was GPS-enabled. Note that Navizon also proposes location-based applications like GeoTags and local services. This technology is probably patented and no public research references have been found on this technology, but this hybrid approach is inspiring and demonstrates that peer-to-peer networks and collaborative platforms can be useful to enhance context data.

In the ContextWatcher software (Koolwaaij et al., 2006), the Location Provider of
IST project MobiLife can infer the location of a mobile user by GPS and/or GSM cell
positioning information, and can use previously discovered mappings between GSM
cells and GPS coordinates.

Since 2008, many web-based interfaces are openly provided to support applica-
tion developers for leveraging location information inferred from several sensors (e.g.
GSM/cell-based, GPS, IP routes), as introduced in the previous section: Yahoo! FireEa-
gle, Google Location API, Android and iPhone APIs. Most of these APIs can adaptively
rely on various positioning mechanisms, and they can represent the resulting location
using various structures: approximate GPS coordinates, street-based address, or place
name(s). In order to switch between those representations, geocoding (and reverse
geocoding) services are also exposed on the web: the Google Geocoding API <fnref n="2" />, the

<footnote n="1">¹Navizon: http://www.navizon.com/</footnote>

<footnote n="2">²Google Geocoding API: http://code.google.com/apis/maps/documentation/geocoding/</footnote>
Geocoding API from Yahoo! Maps Web Services <fnref n="1" />.

### 4.3.3 Movement-based relative coordinates

In some cases, the position of an entity can be computed relatively from movement data. This data can be provided by inertial sensors / odometers or vision-based algorithms (Gay-Bellile *et al.*, 2010). This data alone can provide the relative position of an entity over time. A mono-dimensional sensor (e.g. providing speed/acceleration information) can be sufficient if the entity is moving on a known path (e.g. on a rail). If the relative position of an entity can be mapped at some points to its absolute position (e.g. provided by a GPS receiver), this entity can be located with absolute coordinates with a decreasing confidence (i.e. the relative error increases).

For systems that cannot directly provide absolute coordinates, virtual/relative coordinates are to be gathered from sensors, and the context-aware system can map the virtual coordinates to absolute coordinates whether an absolute location is given by the same entity from time to time. Similarly to the previously-defined *point* class, we could define a *relativePoint* class, with relative coordinates in three dimensions. Every instance of this class must be associated with an instance of a *checkPoint* class which defines the function to convert virtual coordinates into absolute coordinates, for a given entity.

### 4.3.4 Political regions and logical spaces

A location may be represented as a reference to a geographical object in an ontology. Many geographical ontologies exist <fnref n="2" /> <fnref n="3" />. Additionally to GPS coordinates, they often include notions of country, region, city, places, and other points of interest.

It may be necessary to combine with ontologies defining finer concepts such as meeting rooms and offices, train stations, business plants, warehouses etc. This is notably a unifying concept introduced by the SPICE project <fnref n="4" /> which has the advantage of covering all sorts of locations in an object-oriented way: the complexity of choosing

<footnote n="1">¹Yahoo! Geocoding API: http://developer.yahoo.com/maps/rest/V1/geocode.html</footnote>

<footnote n="2">²WSMO ontology: http://www.wsmo.org/2004/d3/d3.2/b2c/20041004/resources/loc.wsml.html</footnote>

<footnote n="3">³Space ontology: http://pervasive semanti web.org/ont/2004/06/space</footnote>

<footnote n="4">⁴SPICE core ontology: http://ontology.ist-spice.org/mobile-ontology/0/10/core/0/core.owl</footnote>
the location description is removed to a *location* class, that may have any number of
representations.

Flanagan (2006) shows that clustering methods can be used to recognize such lo-
cations from available positioning sources. Additionally, ‘geo-tags’ can be attached to
real-world places and objects, allowing people to find digital information (e.g. local his-
tory, art, or communities) on the Internet about them. These geo-tags can be printed
in the form of QR-codes, e.g. as proposed by Sotokolan <fnref n="1" />.

4.4 Virtual sensors: context from computer-based activi-
ties

Computer software and network communications can also provide valuable contextual
information about users’ computer-based activities (Dragunov et al., 2005), such as:

* the subject of web-based documents (including web pages) currently being browsed can be either analyzed using APIs exposed by web browsers, or by capturing network transmissions, e.g. through a proxy. Some particular events (e.g. posting a message on a forum) can be intercepted by subscribing to corresponding RSS feeds. Several web services can be leveraged to extract additional or crowd-sourced descriptions about web-based documents, e.g. tags from Delicious social bookmarking service;

* the subject of a local document (or email) currently being manipulated can be analyzed from Microsoft Word (Budzik & Hammond, 2000; Budzik *et al.*, 2000), or similar applications which expose a sufficient API;

* the current status of a software development task can be captured from IDEs (Integrated Development Environments) such as Eclipse or Microsoft Visual Studio, e.g. frequent unsuccessful compilations can reveal a technical difficulty (Carter & Dewan, 2009);

* additionally, the level of availability of the user can be inferred by analyzing mouse movements, keyboard activity, title of active window, screen-saver mode or even

<footnote n="1">¹Sotokolan: http://www.sotokolan.com/</footnote>
log on/off events. OS-specific daemon(s) must be developed in order to capture
such events.

## 4.5 Social sensors: context from personal streams and crowd-sourced data

The Internet has become a digital space in which rich information has been collabo-
ratively contributed by millions of people, through the use of dedicated web sites and
services. At the so-called ‘Web 1.0’ era, the content of web sites was mostly institu-
tional or personal. But with the rise of ‘Web 2.0’, many web sites opened their content
to their audience, allowing people to contribute their knowledge, point of views, ad-
vices, and even personal status information and activities. Despite the unequal quality
of those contributions, useful knowledge can emerge from their enormous and ever-
growing quantity.

In this section, we firstly present some of those streams and crowd-sourced web sites,
then we identify how contextual information about people can be extracted from them.

4.5.1 Human messages as sensor data

Contextual information about people can be extracted from their recent social updates, as they are shared through the social networking systems they use. Several microblogging systems, including Twitter <fnref n="1" /> already provide the possibility for their users to share the geographical location of their status updates.

By knowing the location of social updates, contextual information about the au-
thor's locality can emerge, like if the author was a sensor. For example, a social update
containing a message like ‘*The power has been cut for 30 minutes now, I'm going for
a walk outside!*’ (which can seem useless even though many messages of this kind are
observed on personal microblogging systems), provides the information that there is a
power outage at the address of its author (which GPS coordinates were attached to the
social update). If other people also post similar social updates from the same neigh-
borhood, it can give a clue about the actual scale of the power outage. The only other
sensors that could be queried for being informed of this problem are part of the infras-
tructure of the electricity provider. By sharing such observations as geolocalized social

<footnote n="1">¹Twitter: http://www.twitter.com/</footnote>
updates, people can quickly know that others are experiencing the same problem, and
"power outage" can be identified as contextual information about the corresponding
location and about the people that are currently located there.

Alternatively, some other contextual clues might also be provided as meta-data
of social updates (similarly to ‘*hashtags*’), or could be queried from semantic web
databases available on the Internet (e.g. LinkedData) (Stankovic *et al*., 2009).

4.5.2 Web repositories of user-generated content

Launched in 2001, Wikipedia <fnref n="1" /> was one of first public crowd-sourced web site. This free encyclopedia has been allowing anyone to edit the content of any article. Whereas this openness has implied many disputes on pages related to controversial subjects (e.g. facts about presidential candidates just before election, about historical events, companies, etc...), it has grown to become a major and useful reference, covering many languages. This encyclopedia has been translated to a semantic database called DBpedia <fnref n="2" /> since 2007, enabling its user-generated content to be machine-readable, so that computer programs (and mashups) can leverage knowledge facts by formulating precise queries.

Even though wikipedia has been opened to any voluntary contributions, contrib-
utors are still few, compared to the number of readers. Participating in social book-
marking sites, like Delicious <fnref n="3" />, have become more popular, as the contribution process
was quicker, simpler and more personal. After creating a (free) account on the site,
users can immediately bookmark web pages that they want to keep, because they en-
joy them, they want to be able to easily find them later, and they (often) want to
share them with other people. In order to make bookmarked web pages more easy to
find later, users are invited to annotate them with ‘tags’, unconstrained words (in any
language, without even spell checking) that subjectively reflect the apparent nature,
function, category and context of those web pages (Ertzscheid, 2008; Golder & Huber-
man, 2005). Web pages bookmarked (and tagged) by several people are thus described
by a ‘tag cloud’, a displayed set of tags which size depend on the number of people who
used each tag to describe this page. As any URL-located resource can be bookmarked
on social bookmarking sites, these descriptions can apply on various types on entities

<footnote n="1">¹Wikipedia: http://www.wikipedia.org/</footnote>

<footnote n="2">²DBpedia: http://dbpedia.org/</footnote>

<footnote n="3">³Delicious: http://www.delicious.com/</footnote>
represented by those resources. For example, tags given to a page that presents a car,
are most probably associated to the car, than to the page/site itself. Now that web
pages exist for almost anything on earth (e.g. people, objects, places, events, etc...),
social bookmarking is a promising paradigm for gathering crowd-sourced descriptions
and classifications of virtual and real entities.

More specific repositories also exist to represent and describe real world entities, and discover their involvement with people's activities. Concerning music, Musicbrainz <fnref n="1" /> and Shazam <fnref n="2" /> can identify the name and interpreter of a song from a sampled audio (e.g. recorded with a microphone), and tags given by people to songs and artists are gathered on web sites like Last.fm <fnref n="3" />, which also maintains a history of the last songs that users listened to. Image sharing web sites like Flickr <fnref n="4" /> can be considered as social bookmarking applied to photographs, as it is possible to tag one's own and other people's photographs, including the time and geographical location where the picture was taken.

Additionally, real-world places are described, reviewed by people and geographically located on various web sites (and their mobile applications) such as Yelp <fnref n="5" />, Qtype <fnref n="6" /> and Google Maps <fnref n="7" />. And some events are announced on sites like Myspace <fnref n="8" /> (mainly for major events, festivals, concerts, and other artistic performances), Facebook <fnref n="9" /> (for commercial and private events) and Taweet <fnref n="10" /> (more generally). Public events can also be advertised through social calendars such as Google Calendar <fnref n="11" />. Future personal plans (e.g. upcoming trips) can be advertised by people on sites like Plancast <fnref n="12" />, Tripit <fnref n="13" /> and Dopplr <fnref n="14" />. Rattenbury et al. (2007) have proven that names of places and events

<footnote n="1">¹Musicbrainz: http://musicbrainz.org/</footnote>

<footnote n="2">²Shazam: http://www.shazam.com/</footnote>

<footnote n="3">³Last.fm: http://www.last.fm/</footnote>

<footnote n="4">⁴Flickr: http://www.flickr.com/</footnote>

<footnote n="5">⁵Yelp: http://www.yelp.com/</footnote>

<footnote n="6">⁶Qtype: http://www.qtype.com/</footnote>

<footnote n="7">⁷Google Maps: http://maps.google.com/</footnote>

<footnote n="8">⁸Myspace: http://www.myspace.com/</footnote>

<footnote n="9">⁹Facebook: http://www.facebook.com/</footnote>

<footnote n="10">¹⁰Taweet: http://taweet.com/</footnote>

<footnote n="11">¹¹Google Calendar: http://www.google.com/calendar/</footnote>

<footnote n="12">¹²Plancast: http://plancast.com/</footnote>

<footnote n="13">¹³Tripit: http://www.tripit.com/</footnote>

<footnote n="14">¹⁴Dopplr: http://www.dopplr.com/</footnote>
can also emerge by analyzing the frequency and temporal distribution of tags associated
to geolocated pictures.

Most web sites cited above expose public feeds that one can subscribe for being
aware of last updates, and/or APIs that allow computer programs to query information,
given specific criteria (e.g. information about a place, a topic, at a given time range).
Thousands of other APIs are referenced on sites like Programmable Web <fnref n="1" />. Also note
that tags are not directly available on all the web sites cited above, but keywords can
be identified from the user-generated content they feature. It is also possible that pages
from those sites are tagged on Delicious.

4.5.3 Extracting context by mashing up web streams and services

We have presented several web-based repositories that can be leveraged to gather infor-
mation about real-world entities and activities. However, they do not directly provide
ways for extracting context information about a person. In this subsection, we pro-
pose an example use case illustrating how they can be combined and processed in this
regard.

Scenario: Alice is walking around in her neighborhood. She can hear some loud
music a few hundred meters away from her, so she decides to walk in that direction.
A few minutes later, she arrives in front of a stage where a rock band is currently
performing. She enjoys their music but does not now the name of that band, although
she would like to. She grabs her mobile phone to check her contextual cloud, it contains
the name of the band: ‘Nemesis’! She is so glad of discovering that band that she
decides to capture this contextual cloud, and shares it to her social network. A few
seconds later, her mobile phone vibrates to attract her attention back to the screen.
Bob, a friend of hers, posted a social update about that concert a few hours ago: ‘mad
about not being able to see Nemesis’ concert tonight :’-(’. She decides to capture a
short video of the concert, with a personal message for Bob, and shares it to the social
network, knowing that Bob will receive the video. The metadata of the video are
automatically populated from her contextual cloud: time, place, name of the band.
Additionally, she realizes that even the name of the song is included. A few seconds
later, Bob sends her a SMS: ‘Hey, I didn’t know you were going to Nemesis’ concert

<footnote n="1">¹Programmable Web: http://www.programmableweb.com/</footnote>
tonight, I'm jealous! You should come visit me when I'm finished, you'll have to tell
me more about their performance!'

Awareness aspects: Thanks to her contextual cloud, Alice discovered the name of the band, without asking for it. By sharing this contextual cloud, she was notified that a friend of hers would have liked to come to that concert. This notification triggered an opportunistic communication between Alice and Bob, which motivated them for meeting each other.

Technical solution:

* When Alice arrived in front of the stage, her contextual cloud aggregated tags from social updates that have been posted by surrounding users, and from the city's federated concert agenda. Because 'Nemesis' was appearing on most of those social updates, and on the event planned at her location, the corresponding tag was highly weighted.

* By sharing her contextual cloud, a social update was generated and shared to her friends, through their social network, making Alice's context visible to Bob. The matching system identified a social update which Bob posted about that band, the 'Nemesis' tag being also part of his context when he posted it.

* When she captured a video, a music recognition service was invoked to identify the name of the song, which was added to Alice's contextual cloud.

* The tags of her contextual cloud were transfered as metadata to the video hosting system, so that other people can easily find it.

## 4.6 Conclusion

In this chapter, we have presented a state of the art of context management systems, from which identified lacks and opportunities for improvement that motivated the adoption of some requirements towards the development of our own context management system. In the following sections, we described three types of sensors that we intend to leverage in our application: (i) physical sensors and their associated techniques which can provide the location of users, (ii) virtual sensors that can extract tags and events from computer programs being used, and (iii) social sensors to extract additional tags
from social streams and crowd-sourced repositories hosted on the Internet. As an ex-
ample of how those repositories could be leveraged, we proposed a use case and its
enabling techniques.

In the next chapter, we will define a context management framework leveraging
those three types of sensors, towards the development of social awareness applications.
# Chapter 5

## A Context Management Framework for Social Awareness

In the previous chapters, we have reviewed the state-of-the-art of social networking systems, awareness systems and information filtering techniques, and context management frameworks. We identified a need for filtering social updates in order to reduce the frequency of interruptions, enabling techniques that can be leveraged, and relevant context extraction and modeling methods.

In this chapter, we introduce and develop our approach for modeling contextual information and filtering social updates against it.

### 5.1 Aims

In this thesis, we intend to enhance digital social networking practices by supporting users with the sharing and filtering of information in high-scale communities, including professional/collaborative environments.

Current social networking systems rely on subscription to specific people's social feeds. This characteristic implies that reading all the social updates from subscribed social feeds is costly, in terms of attention, as their number increase. Furthermore, some users like social updates from their subscribed feeds to be displayed in real time on their screens, but it has been proven that frequent interruptions (implied by the notification of new social updates) can reduce the user's focus and performance on his/her main task.
Users confirmed that some support needs to be provided for them to cope with the quantity of social updates they receive. Novel information filtering techniques are to be studied to address this issue.

By supporting users with adapted ranking and filtering mechanisms, it is possible to reduce the quantity of notified social updates by selecting the most relevant ones.

According to Vogiazou et al. (2003), it is essential to understand, analyze and convey the ‘context’ of a collaborative or communicative act, in order for this communication to make sense. From this analysis, we identify context as a promising criteria for filtering and ranking social updates. In order to rank and filter social updates according to contextual properties, we need to gather contextual information about users.

Therefore, the proposed contributions of this thesis are:

* to model representative contextual information about users and their activities,

* to develop a filtering model that leverages contextual information for measuring the relevance of users’ social updates,

* and to evaluate this context-based filtering model in a social awareness application.

## 5.2 Problem definition and scope

### 5.2.1 Hypothesis

The hypothesis that motivates our research is proposed as follows:

‘a social update shared by person U on a social networking system is relevant to another person X if the current context of X is similar to U’s context at the time of sharing.’

This hypothesis raises the following questions:

* How to model contextual information about users and their activities, towards efficient relevance-based filtering of social updates?

* Where to gather contextual information from?

* On which kind of social updates does this contextual filtering perform best, and why?
* How to ensure that the user can control the transmission of potentially private information about his/her context and activity?

In this section, we address those questions, by specifying constraints and choices, proposing our approach and a methodology to evaluate it.

### 5.2.2 Constraints and choices

#### Quality emerges from quantity

In chapter 4, we have surveyed several modeling and management schemes for contextual information. It is clear that the current technological ecosystem allows gathering rich (however uncertain and heterogeneous) contextual information that could be leveraged for filtering social updates. In addition to data generated by physical sensors, we identified that additional contextual information can be gathered from software-based sensors ('virtual sensors'), and from humans ('social sensors'), thanks to social streams and crowd-sourced data which emerges from web-based services. However, we also identified that a semantic model would reduce the spectrum of contextual information that can be leveraged, because such models are framed by domain ontologies defined by experts, and thus can not cover concepts and entities that have not been defined in yet existing ontologies.

In order to gather rich contextual information, we prefer to leverage the quantity (in spite of semantic expressivity) of human-entered information, over the quality (but rareness) of semantically-represented information. This choice implies more noise and ambiguity than semantic approaches, but we support that quality naturally emerges from quantity of imperfect data.

#### Tangibility for user-centric privacy control

Information exchanged on social networks is personal by nature, but also explicit: users write social updates from their own decision, and with content they intend to share. On the other hand, as contextual information about people is to be gathered automatically, it might include information that they are not willing to share. A social application which relies on such contextual information thus implies some potential privacy issues. Even if this context is not directly visible to other people, the idea of sending and storing it on a server, somewhere on the Internet, is somewhat scary. Indeed, it might
be possible for complete strangers to access this critical information, either by manip-
ulating the servers legally or illegally (e.g. hacking). A technical anomaly could also
cause a leak of information to unauthorized audiences.

Additionally, in their definition of ‘*Systems that Display Their Context*’, Dourish (2004) recommend that a context-based system should ‘*allow users to explore the technical context in which their applications operate*’, so that users can ‘*make continual determinations of the potential consequences of their actions and their opportunities to reconfigure or realign the technologies through which they are conducting their actions*’.

Whereas it would be tempting to automatically (and thus, transparently) gather
as much contextual information from users as possible in order to optimize relevance
rankings, we support that users must be able to control whatever contextual information
going to be sent to the Internet. In order to be humanly understandable and editable,
the amount of contextual information must be reduced, so that each person can easily
and quickly control (and manipulate, if necessary) contextual information that is going
to be sent to the Internet.

5.2.3 Approach: a tag-based context model

Following these two constraints, and similarly to the Library Mirror application (Koch, 2005), we decide to model and represent contextual information as ‘tag clouds’: a limited set of words, with various sizes depicting their individual representativity (e.g. importance and/or confidence of each word for representing the user’s context).

<figure><img src="image_48.png" /><figcaption>Figure 5.1: A sample contextual tag cloud -</figcaption></figure>

With such a representation (as depicted on Figure 5.1), it is quick and easy for
users to see what information has been gathered, modify some parts if necessary (e.g.
remove some tags), and chose to share it, or not.
Additionally, the simplicity of this representation makes it possible to leverage nu-
merous existing information sources, whatever the form (e.g. either semantic, struc-
tured or plain content), and for the user to add contextual information manually if
desired or required.

5.2.4 Methodology

In order to validate this approach, we design the contextual and ranking models, then
we apply this framework to address a specific case study in a closed contextual scope:
a computer-based enterprise environment. Theoretical and software developments are
thus twofold: (i) the common framework, and (ii) specific developments for the case
study. The context model and ranking algorithms are evaluated by volunteers in the
frame of an experiment. Limitations and future research issues are identified to improve
the performance of the system.

Relying on a common framework, two prototypes are developed in parallel: (i) an ‘on-line’ system that implements a interaction model responding to an ‘enterprise awareness’ case study, in order to measure human acceptance and usability, and (ii) an ‘off-line’ toolkit for evaluating the quality of relevance ranking algorithms, by generating personalized surveys from actual user data.

In this chapter, we present the common framework. The ‘enterprise’ case study
and application is presented and evaluated in the following chapter.

5.3 Interaction model

In this section, we specify the framework from the user's point of view.

5.3.1 General interaction flow

As depicted on Figure 5.2, the interaction flow is a loop which iterates from the
*contextchange* event, which can happen at any point of this flow. Such change im-
ply a *contextsynthesis* process which produces a *contextualtagcloud* that is displayed
to the user. At that point, the user can decide to *edit* the contextual cloud before
*sharing* it with the community. The *socialranking* process matches users with similar
contexts, and returns a list of relevant *socialrecommendations* to the user, which he
can *browse* or *contact*.
<figure><img src="image_49.png" /><figcaption>Figure 5.2: Interaction flow -</figcaption></figure>

This flow complies with our previously specified requirements:

* The contextual information can be manipulated by the user, allowing him to add, adjust or remove contextual tags to better represent his context to the community without compromising privacy.

* No contextual information is ever shared before the user explicitly submits it. This also protects users from losing privacy control.

### 5.3.2 Computer-based interaction

On a computer, the user benefits from a potentially large screen and a mouse as navigation interface. It is thus acceptable to allocate a reasonable dedicated portion of the screen for the graphical interface of this social recommendation system (e.g. displayed in a side-bar). In that case, the current contextual tag cloud and social recommendations can both be permanently displayed and dynamically updated, allowing the user to observe near-real-time evolution of the context as he interacts with his computer.

This set up allows a rich user experience. A dynamic display of the contextual tag cloud can make users want to share it frequently, and thus guaranteeing more numerous and recent social recommendations.

### 5.3.3 Nomadic interaction

As mobility in open environments imply frequent transitions through rich contexts, nomadic interaction is to be designed. In this case, one can assume that users are
carrying hand-held devices with compact screens and limited (and/or less reliable)
interaction interfaces: a few buttons, a compact pointing solution, a touch-screen,
motion recognition, and possibly speech recognition. Nomadic situations imply that
users can not always benefit from a comfortable interaction experience: he can be
disturbed by visual, acoustic, or mechanic interferences at any time.

With such constraints, and complying to modern smartphone operating systems,
we propose that users can request the contextual tag cloud to be displayed by:

* pressing a specified button,

* touching a specific graphical icon,

* shaking the device with a specific motion,

* etc...

On some devices, it is also possible to allocate a permanent screen space that is visible
as soon as the device is activated (e.g. a gadget in Android OS). Similarly to the
‘Ambient Tag Cloud’ concept (Baldauf et al., 2009), the contextual tag cloud will evolve
dynamically as the user’s context (including his location and surrounding people and
other entities) evolve.

While using a hand-held device, we assume that the physical constraints are too
restricted to manipulate contextual tags as easily as the computer-based version of
the graphical interface proposed above. Thus we propose to allow removal of tags
only. By sharing the contextual tag cloud, the screen would display relevant social
recommendations in place of the contextual tag cloud. At that point, the user can
browse those recommendations or switch back to the contextual tag cloud.

5.4 Contextual model

5.4.1 Contextual dimensions and information sources

Based on the sensors described in chapter 4, we consider several dimensions of contex-
tual information that can be leveraged for evaluating the relevance of social updates.
### User's activity

Information about on-going activities is essential for determining the potential relevance of a person and his/her social updates for another person. We propose some examples of tags, and how they can be extracted:

* *browsing, reading*: Such activities can be extracted using virtual and social sensors. Subjects and topics describing the content being read by the user can be provided by his/her web browser and/or other reading software and devices. Descriptions can be enriched by leveraging web-based repositories associating the content with tags, such as social bookmarking sites (e.g. delicious).

* *watching, listening*: The multimedia player(s) of the user can provide information about the content being played (e.g. a video, a song). If this content is being played from a web site, the description of this content might be extracted similarly to the previous example. In some cases, music logging services such as last.fm can also be queried.

* *meeting, attending, talking*: A planned meeting or event and its description can possibly be identified from the user's calendars (e.g. MS Outlook, Google Calendar), including events planned in social networks (e.g. Facebook, Twitter). In both cases, a speech analysis sensor can also infer the oral communication status of the user.

* *traveling, driving*: Such activities can be extracted from travel planning services (e.g. dopplr, tripit) in which the user has explicitly shared his/her travel plan. They can also be inferred by measuring the proximity of the user and a wireless beacon located the transportation means (e.g. the user's mobile phone is connected to a bluetooth equipment installed in a car), and a stable geographical movement (the transportation means can be inferred by the speed of movement and/or the correlation of its path with a roadmap). Clues about the destination and the reason of this trip can also be extracted, depending on the sensor.

All those examples can also be extracted from social updates relating to the user (e.g. '*@adrien, according to tomorrow's traffic provisions, you should take the national road instead of the freeway*' sent by a contact).
## Surrounding environment, people and events

Additionally to the user's activity, his/her surrounding context (environment, people, unplanned events) can provide interesting attributes for finding relevant contacts and social updates. We propose a list of attributes and example tags:

* Current location, place: information about the user's current location (e.g. country, city, street address, venue) can easily be extracted by several physical sensors, supported with geocoding services (in order to transform geographical coordinates into a named place/address), and web pages describing the place (e.g. qype, yelp). Location-based tags can also be identified from geolocalized annotations (e.g. local photos from flickr)

* On-going events: assuming that unplanned events can be announced in real-time by user-generated content (e.g. status messages, pictures and videos shared with annotations), they can emerge from geolocalized social feeds.

* Environmental attributes (e.g. *crowded*, *noisy*) can be inferred from physical sensors such as the microphone of the user's mobile phone, but also from external sensors (e.g. surrounding microphones).

* Surrounding people (e.g. *Mike*, *JacquesChirac*) can be inferred from a recent mediated activity (e.g. a status update) of users that are in a close geographical range, but also from the short-range wireless connections (e.g. bluetooth) between the users' mobile phones.

* Broadcasted music (e.g. *RedHotChiliPeppers*) can be identified by music recognition services (e.g. shazam).

## Future plans and intentions

We have identified that social update consumers enjoy discovering information related to longer-term plans and intentions. Figuring out users' goals can be complex, but some future plans and intentions can be extracted from users' calendars, as explained earlier.

Identifying such plans is important for our filtering model, as social networking systems are a good way for getting recommendations and advices from trusted contacts.
Those advices can possibly be relevant with the user's plans. For example, a friend's recommended venue is relevant if the user intends to go there sometime, especially if this venue is in a remote place which the user is not familiar with (e.g. another country).

Such plans can possibly be extracted from the user's calendar, or trips he/she advertised on trip planning services (e.g. tripit, dopplr).

### Discussion

One can argue that the context should be structured (e.g. as an ontology) according to those dimensions and the identified attributes. We reply that those dimensions and attributes are not exhaustive, and that relying on a predefined structure would thus reduce the support for unexpected situations. We insist on the fact that, in our filtering approach, the semantics of contextual tags do not matter, as the relevance between people and their social updates relies on named entities, whatever their relation to the user. This intended lack of structure enables a wide combination of use cases, including:

* matching a user that is attending a concert of a band, with an other user reading a web page about that same band,

* recommending photos from a given place to someone who intends to travel in that region,

* notify users about to enter a metro station, that one of the metro lines is temporarily stuck (e.g. as posted by someone else on that line),

* etc...

Modeling rules for every possible use case is not possible, therefore we believe that a structured context model is not needed.

### 5.4.2 Formalization: a tag-based contextual algebra

In order to represent and to manipulate tag-based contexts, we chose to rely on the well-known vector space model, as proposed by Salton & McGill (1986), and to define specific functions to attribute weights to every term. The weighting functions are
equivalent to *context interpreters* (Dey, 2000): they transform raw data (information from sensors) into higher-level information (tags), by abstraction.

Traditionally, a set of terms representing a document *d* is formalized as a vector of weights *v*(*d*) = [*w*(*t*₁, *d*), *w*(*t*₂, *d*), ..., *w*(*t*<sub>*N*</sub>, *d*)] attributed for each term *t* = [*t*₁, *t*₂, ..., *t*<sub>*N*</sub> ∈ ℝ]. In our model, terms do not necessarily represent a document, they are human-readable attributes that can be extracted from physical, virtual and social sensors to represent some information about one's context. For each contextual sensor to leverage, a corresponding weighting function is to be implemented (or re-used), depending on the specific requirements of each application.

In the case of our computer-based awareness application developed in the next chapter, terms can be extracted from documents being currently viewed or manipulated, as part of the context of the user, but they can also be tags given by people to describe these documents.

In order to manipulate such weighted term vectors (also called contextual tag clouds), we propose the following algebra:

* The *addition operator* (+) yields a vector in which each term is weighted by the sum of the weights of that term in the provided vectors.

* The normalized form ||*v*|| of a vector *v* conforms to Σ<sub>*t*=*t*<sub>1</sub></sub><sup>*N*</sup> ||*v*<sub>*t*</sub>|| = 1, with weight values *v*<sub>*t*</sub> ∈ ℝ in the range [0, 1].

* The *aggregation operator* is the addition of given weight vectors, after their individual normalization. Thus, the aggregation operator applied to a set of vectors *V* = [*v*₁, *v*₂, .., *v*<sub>*M*</sub>] acts as the following function:

$$aggr(V) = \left\| \sum_{t=1}^{M} \|v_t\| \right\|$$

* The *similarity function* between normalized vectors, like in traditional vector-based models, relies on cosine similarity. Thus, the relevance of a vector *R* with another vector *S* is computed by:

$$sim(R, S) = \frac{R \cdot S}{||R|| \cdot ||S||}$$

which returns a similarity score *r* ∈ ℝ in the range [0, 1], 1 being the maximum (i.e. contextual equality).
In the following section, we develop a social matching scheme and the corresponding relevance functions relying on this algebra.

## 5.5 Social matching scheme

### 5.5.1 The temporal dimension of social matching

In this section, we study the temporal dimension of social recommendations. According to the trends of usage of Social Networking Systems presented in Chapter 2, and Awareness systems presented in Chapter 3, we can imply the following statements:

* Most existing recommender (and filtering) systems leverage a history-based profile of recipients (i.e. people who consume content) to propose relevant content, possibly recent.

* Some systems recommend content that is relevant with the recipients' current activity and context.

* Current microblogging systems provide new social updates in real-time, without profiling nor filtering. Consumers can decide which feeds they want to follow: people and/or subjects (represented by hashtags, keywords, or queries).

* Microblog consumers do not necessarily read social updates in real-time, but many like to keep up with new updates (e.g. by keeping the feed, or the number of unread items visible on the screen).

* Microblog consumers mostly find social updates interesting and relevant more for longer-term objectives (planned activities, expected achievements, goals), than for their current activity.

* Filtering support is expected by microblog consumers.

From these statements, we identify the need for four social matching modalities, as depicted on Figure 5.3:

* **Profile-based filtering of new social updates.** Like in traditional filtering systems, history logs can be leveraged for building user interest profiles, in order to filter new social updates according to their similarity to those profiles. In our
<figure><img src="image_50.png" /><figcaption>**Figure 5.3: Different matching modalities** - between a sender $U_S$ and a potential recipient $U_R$, depending on their contextual time lines: profile-based filtering of new content (top-left), context-based recommendation of people (top-right), context-based recommendation of content (bottom-left), plan-based recommendation of content (bottom-right)</figcaption></figure>

case, profiles can be a large set of weighted tags, containing many contextual clouds, over a wide period of time.

* **Context-based recommendation of people.** Additionally to reading social updates from selected people and subjects, new people can be recommended, accordingly to the user's current activity and context. This case is especially useful for getting punctual (and synchronous) support from relevant people on a specific issue, or for creating and enriching communities (of interest and practice), and it does not require users to produce social updates.

* **Context-based recommendation of social updates.** Social updates, even after some time, are a catchy way to promote, discuss or share opinions about a subject that is important to the person that sends them, providing a communication opportunity with this person. Previous social updates can be recommended to users whose current context is similar to the sender's at the time of posting, so that they can communicate asynchronously.
* Plan-based recommendation of content. Most microblog consumers think that social updates are usually more interesting for longer-term activities than for current ones. If users' goals, plans, and other future activities could be identified automatically, it would be possible to rank the relevancy of social updates according to those.

In this thesis, we are investigating context-based recommendation of social updates and people, and integrating profile-based information in this process.

### 5.5.2 General relevance function

Based on the algebra defined in the previous section, we propose a general relevance function that can be used to implement three of the four social matching modalities introduced above.

$$rel(S, R, P, \alpha) = sim(\|S\|, \|\alpha \cdot R + (1 - \alpha) \cdot P\|)$$

This function relies on:

* S is the current contextual cloud of the sender,

* R is the current contextual cloud of the recipient,

* P is the interests profile of the recipient (an aggregation of all user's contextual clouds),

* $\alpha \in \mathbb{R}[0, 1]$ is the contextual factor, to adjust the importance of the current context against the user's profile as criteria for evaluating the relevance,

* $sim(S, R)$ is the function that computes the cosine similarity between vectors S and R, as defined in the previous section.

In the following paragraphs, we explain how to use this general function to achieve three matching modalities. A major distinction between those modalities is the age of S, the contextual cloud of the sender.

In profile-based and context-based recommendation of social updates, relevance is to be estimated with the context of the sender S at the time of posting. The recipient's filtering criteria can be adjusted between his/her profile and current context,
by adjusting the value of the parameter α. With α = 1, recommendations will be based
on the recipient's current context. With α = 0, they will be based on the recipient's
profile (i.e. accumulation of previous contexts).

In order to achieve context-based recommendation of people, *S* must contain the
current (or last) context of the sender, and α must be set to 1.

**5.6 Design and implementation of the framework**

In this section, we transform the conceptual framework defined above into a software
framework, according to the object-oriented design and programming methodology.
Similarly to the Context Toolkit proposed by Dey et al. (2001), we apply the ‘separation
of concerns’ paradigm by implementing some building blocks that can be leveraged as
interfaces for developing context production modules on one side, and context-aware
applications on the other side. That way, application developers can focus on the core
functionality of their context-aware applications, while re-using context production
modules possibly developed by other parties.

5.6.1 Data flow

<figure><img src="image_51.png" /><figcaption>Figure 5.4: Overview of the contextual recommendation loop -</figcaption></figure>

As depicted on Figure 5.4, contextual information is extracted virtual sensors (e.g.
from user-manipulated content) and physical sensors by ‘context sniffing’ modules. A
‘context aggregation’ module gathers contextual information from all these sniffers, and enriches it using social sensors in order to combine them in a ‘contextual cloud’ that represents the user’s current context. When submitting a ‘social update’ (e.g. a ‘tweet’), this contextual cloud is submitted to the ‘filtering system’ (i.e. a server running in the infrastructure) as a meta-description of the social update, so that the filtering system can match it with other users’, using a context-based relevance function. This social update will then be ‘notified’ to relevant people (e.g. users whose last contextual cloud is similar to the one submitted with the social update). That way, every user gets a dynamic (near real-time) list of recent social updates, sorted by decreasing relevance rank as their context evolves. Like with regular Social Networking Systems (e.g. Twitter), these short social updates can be quickly read by users to remain aware of relevant activities going on in their communities. They can also decide to reply to social updates or to call their sender.

### 5.6.2 Main structures

We are now presenting the classes that we have designed in Java, around the three following concepts: contextual clouds (data), social updates (events), and contextual matches (data+event).

#### Classes of contextual clouds

As explained earlier in this chapter, contextual clouds are formalized as term vectors weighted by real numbers, which can be normalized. Besides adjusting the weights, normalization of contextual clouds implies additional processing steps, such as: removal of stop-words and other excluded terms, and selection of most weighted tags. Normalized contextual clouds are thus finished products that are not meant to be modified, but instead to be exchanged between context producers (sniffers, aggregators, repositories) and consumers (applications, aggregators, repositories). Before this normalization, an intermediate structure is to be defined for context sniffers and aggregators to progressively synthesize and manipulate contextual clouds.

For those reasons, we have designed two classes *CtxCloudFloat* and *CtxCloudNorm*, respectively encapsulating an intermediate structure and a finished product. The common structure and functionality of those classes is encapsulated in *CtxCloudFloatBase*, defining a term-to-weight hash map to store the weighted tag vector and exposing
<figure><img src="image_52.png" /><figcaption>Figure 5.5: Hierarchy of classes encapsulating contextual clouds -</figcaption></figure>
## 5. A CONTEXT MANAGEMENT FRAMEWORK FOR SOCIAL AWARENESS

tag access functions returning either objects or output rendered as HTML or CSV. `_CtxCloudFloat` extends `_CtxCloudFloatBase` with tag-manipulation methods, including: weight retrieval (`get`), addition (`add`) and override (`set`). It also implements addition of contextual clouds. Calling the `normalize()` method normalizes and returns the contextual cloud as an instance of `_CtxCloudNorm`, that also extends `_CtxCloudFloatBase`. In that form, weighted tags are cleaned, filtered, sorted and projected as a vector of limited size, and additional import and export methods are exposed.

### From events towards contextual indexing of social updates

In our framework, we have seen that a user's current context is represented as a contextual cloud. But the dynamicity and unpredictability of contexts imply that the framework must be able to receive updates at any time, in an asynchronous manner (e.g. from a context sniffer, or a context aggregator). In order to handle such updates/events, we defined a `_CtxEventBase` class.

The `_CtxEventBase` class encapsulates a contextual *cloud* (an instance of `_CtxCloudFloat`), a *timestamp* and a description of the author (or *source*) of that cloud, and provides the corresponding getters and setters. Additional methods are also provided to import and export such events from/to hash map structures, HTML code and JSON objects.

Social updates are events thrown by social networking systems. In order to support contextual recommendation of social updates, the framework has to integrate those social updates, and also the context of their author at the time they were sent. For that reason, we extended `_CtxEventBase` by two classes that combine a contextual event and a social update. Especially designed for external social updates, `_CtxSocialUpdate` contains a *content* and a *link* attribute, respectively associated to the content of the social update to encapsulate, and its URL on the social networking system. We also added a *TweetEvent* class that only contains a *message* attribute, in order to support internal social updates, not relying on any third-party system. This internal social networking system will be further explained in the next chapter.

The aim of these combined structures is to ease the contextual indexing of social updates, so that previous social updates can easily be retrieved with their corresponding context.
<figure><img src="image_53.png" /><figcaption>Figure 5.6: Hierarchy of classes encapsulating contextualized events - (including social updates)</figcaption></figure>
<figure><img src="image_54.png" /><figcaption>Figure 5.7: Hierarchy of classes encapsulating recommendations -</figcaption></figure>

### Recommending contextually relevant updates and people

When some social updates are indexed by context, and that the system receives a new context from one of the users, a set of relevant social updates are to be returned to this user, according to the similarity between the user's current context and the context of each social update.

In order to maintain a list of relevant social updates, a *Recommendation* class was defined, associating a contextualized *event* (e.g. a *_CtxSocialUpdate* object) to the similarity *score* computed between the context of this event and the context of the user. Most relevant social updates can then be sorted by decreasing *score*, and notified to the user, in response to his/her new context.

An additional class, *RecomWithContext* extends *Recommendation* to also refer to the user whose this recommendation concerned, his/her contextual cloud, and its position in his/her contextual index. This class is useful for generating comprehensive recommendation reports from various users, and their previous contexts at once (e.g. for a batch recommendation process).

### 5.6.3 Using the framework for real-time matching

In this subsection, we present the classes and methods to use for creating a real-time awareness system, as depicted on Figure 5.4, relying on our framework.
Submitting contexts and social updates

As depicted on Figure 5.4, the filtering system developed upon our framework, is to be fed by user contexts and social updates. We recommend generating these contexts in two separate steps: context sniffing, and context aggregation. These processes are preferably implemented in dedicated modules, but they can possibly be implemented in a single module, as long as it can produce normalized contextual clouds and submit them to the system (as instances of the *CtxCloudNorm* class). Nevertheless, we will explain this process in two steps.

A context sniffing module is responsible for observing contextual clues, and notifying a context aggregator of valuable events and information. An event is valuable when it can have an impact on a user's context. The implementation of sniffing modules, and their communication with context aggregators is independent of the framework.

A context aggregator is a module that synthesizes normalized contextual clouds from valuable events concerning a user, and submits substantial changes to the framework, after user confirmation. It can possibly receive such events asynchronously from various context sniffers. In that case, it is responsible for abstracting the meaning of those events into tags, and aggregate those tags progressively into a *CtxCloudFloat* instance, using the *setTagWeight()* and *addTagWeight()* methods. When this contextual cloud contains comprehensive (or sufficient) description of the user's current context, it must be normalized into a *CtxCloudNorm* instance, returned by the *normalize()* method.

According to the requirements presented earlier in this chapter, contextual clouds must not be sent to a remote server (e.g. the filtering system) before having been shown and confirmed by the user. It is possible to implement a contextual cloud manipulation interface allowing the user to edit this cloud before sharing it, using the methods provided in the previous paragraph. Additionally, the user interface may allow the submission of a social update to be attached to this context. When the user confirms the submission of a contextual cloud and/or a social update, they must be sent as a *CtxEventBase* instance (or derivative) to the filtering system. The interface of this filtering system is not specified in the framework, however the event must be passed in its normalized form to the *onEvent()* method of a class which implements the *IEventListener* interface.
## 5. A CONTEXT MANAGEMENT FRAMEWORK FOR SOCIAL AWARENESS

In our implementation, we have developed two filtering servers exposing a RESTful interface. Events are to be submitted by making a HTTP POST request to the *event* entry point (url: 'http://filtering_server_host/event/'). The *'CtxEventBase* instance is expected to be serialized as a JSON string, in the body of the POST request, using the *toJsonString()* method. The *JsonSender* class implements this submission process, and is convenient to use with our servers.

For handling submitted events, our two servers rely on a common reference implementation of the *IEventListener* interface: the *ContextManager* class. This class holds the last events for each user (submitted by their respective context aggregator), evaluate the similarities between the corresponding contexts (using the 'Algo.getCosineDistance*' function, and dispatch events, context updates and resulting matches to subscribers.

### Subscribing to matches

In our reference implementation of the *ContextManager* class, three kinds of subscription are provided:

* The *subscribeToEvents()* method registers objects that implement the *IEventListener* interface. When a new event is received, it is transmitted to subscribed objects by calling the following method on them: *onEvent(CtxEventBase).*

* The *subscribeToMatch()* method registers objects that implement the 'IContextMatchHandler' interface. When a similarity has been evaluated between two contexts, the following method is called on every subscribed object: 'handleContextMatch(String user1, String user2, CtxCloudFloat overlap, float similarity)'.

    *user1* and *user2* identify the users whose contexts have been compared, *overlap* is a contextual cloud that contain the tags that appear in both contexts, and *similarity* is the relevance score in the range [0; 1].

* The *subscribeToContext()* method registers objects that implement the *IContextListener* interface. When a new context is received from a user, the following method is called for each *tag* of this context and of the previous one from this user, on every subscribed object: 'onTagChange(String tag, Float weight)'. *weight* is the weight of *tag*, in the new context. Tags that were appearing in the previous cloud only
are associated a *weight* value of 0. Notice that, in the current implementation, it is not possible to filter the subscription for a specific user only.

### 5.6.4 Using the framework to generate batch matching reports from logs

In order to evaluate the quality of contextual clouds and contextual relevance of social updates, an off-line process with intermediate output is required. In this process, users' actions and social updates are logged for being processed later, in a batch sequence.

Most tools we developed are independent from the type of contexts to process, except the users' activity logging tool and the processing of social sensors, which were required to evaluate the framework, as defined in the next chapter. In this evaluation, users' activities are web browsing events logged from a web browser. Those logs are timestamped lists of URLs which were opened, closed or focused. The social sensors that we developed extract tags describing the content located at those URLs.

<figure><img src="image_55.png" /><figcaption>Figure 5.8: Batch sequence, from logs to reports -</figcaption></figure>

The batch sequence, as depicted on Figure 5.8, consists of:

* extracting the list of URLs to submit to social sensors, from users' activity log (in CSV format);

* the social sensors process the list of URLs and store the resulting tags into a common tag cache (for all users, in JSON format);

* from the users' activity log and the populated tag cache, users' contextual clouds are generated and indexed on a common timeline to a JSON file for each user;

* the users' RSS feed containing their social updates are converted to a JSON file;
* social updates are indexing on the common timeline, and stored in another JSON file;

* finally, several reports can be generated about tags, contextual similarities, and contextual recommendations of social updates.

In this subsection, we present the tools, classes and methods of our framework, to log user activity, and to generate batch reports (and intermediate steps) from those logs.

### Logging user activity

In order to gather contextual information about a user, some events related to this user are to be observed, including user's actions. In a real-time setup, those events are captured by context sniffers in order to synthesize a contextual cloud on the fly. In a sequential/batch process, those events must be logged for later processing.

For evaluating our application, as described in the next chapter, we adapted a context sniffer into a context logger. This context logger works as an extension of the Firefox web browser. Participating users have to install this extension, so that their browsing events (e.g. opening, closing, switching pages) are transparently tracked and logged into a local database. Another tool exports this log into a CSV file called ‘user log’ or ‘actions log’ that can be edited by users (for privacy reasons) and parsed from scripts provided with the framework, as explained in the following subsections.

### Gathering and caching tags

Some context sensors rely on web services to turn user actions into meaningful tags. This particularly applies to what we call social sensors, because the corresponding information is extracted from web-based messages, but it also applies to some virtual and even physical sensors (e.g. geo-coding: transforming geographical coordinates from a GPS sensor into the name of the corresponding place). In a real-time application, these web services can possibly be queried on demand. On the other hand, generating batch reports from logs of past user actions in an experimental setup requires a stable mapping between actions and tags. One reason is that researchers need to be able to reproduce the exact same results from the same logs, whatever the technical problems
that can arise from querying web services (e.g. connection problems, interrupted ser-
vices). This constraint motivates a pre-fetching and a caching of tags into a repository
we call ‘tag cache’.

In order to populate this tag cache, URLs or web queries that would be invoked by
context sniffers and aggregators must be gathered in a list, so that they can be invoked
at once for all users logs. Notice that, gathering the URLs of all users also implies
a removal of duplicates, and thus ensures that the same tags will be used for a same
query. This effect will not necessarily be observed in a real-time setup, as each user’s
aggregator invoking a same web query might give different results, depending on the
time of invocation, and of network conditions. This first step is achieved by a script
called *extractURLs*. It generates a list of URLs that can be exported to a text file,
extracted for a list of action log files (normally, one per user) provided as command
line parameters.

From this list of URLs, the * populateTagCache *script generates a tag cache, con-
sisting of a list of * CtxResourceReport * instances serialized as JSON strings. The
* CtxResourceReport * class associates a resource identifier (in our case: the URL of
a document that has been browsed by the user) to a * CtxEventBase * instance which
contains the contextual tags associated with this resource, and the identifier of the
weighting function (also called interpreters, or * enhancers *) that was used for extracting
these tags. This output is thus human- and machine-readable, and will be leveraged
by the next step of the process. The list of weighting functions that will be used to
extract tags can be configured in the script itself.

As a tag cache might have been generated from a restricted set of weighting func-
tions, we also provide a *mergeTagCache* script that aggregates two or more tag cache
files into a common tag cache.

Indexing user contexts

In a real-time setup, context events are submitted asynchronously by users, and matches
are based on each user's last submitted context. In an experimental log-based setup,
users are not solicited, thus context events do not exist. In order to simulate these
events, a common and synchronized pulse is to be defined. At each pulse, the actions
that had been logged since the last pulse are considered as part of a new contextual
cloud, like if it was submitted by the user at the time of that pulse. That way, it is
possible to index contextual clouds of several users on a common time-line.

For achieving this process, the framework provides a script called *indexContexts*.
By passing the tag cache and a list of action logs as parameters, this script generates
a indexed list of contextual clouds per action log. Each contextual cloud is repre-
sented by a JSON-serialized *CtxCloudNorm* instance, prefixed with the index number
corresponding to the timestamp of this contextual cloud (i.e. of the pulse, in our exper-
imental setup). The index number is computed given the time-range of the experiment
(in ‘epoch’ format), using functions provided in the *Constants* class. Each index file
begins with a heading line that contains the time-range of this index, and the frequency
of the pulse, also serialized as JSON strings.

### Importing and indexing social updates

Additionally to action logs, social updates posted by users must also be listed in a log, for processing. As most social networking sites expose RSS feeds of social updates, it is possible to gather a history of users’ last social updates from every service they posted to. We developed *fetchFeed*, a script that can extract social updates from a RSS feed, and store them in a file as a list of *SocialUpdate* instances, serialized as JSON strings.

As for contexts, social updates must then be indexed on the common time-line.
For that, the *indexFeed* script transforms a list of JSON-serialized *SocialUpdate* in-
stances, provided as parameter together with their author’s context index, into a list
of *CtxSocialUpdate* instances, also serialized as JSON strings. The *CtxSocialUpdate*
class extends *CtxEventBase* with two additional attributes for attaching the *content*
and *link* of the social update. This operation is very similar to the contextual indexing
process described earlier. It results in a social update cache file, in which social updates
are indexed by pulse number, and associated to the context of their author at the time
of posting (or the last known context before posting).

### Generating matching reports and charts

Once users’ contexts and social updates are indexed on a common time-line, it is pos-
sible to generate reports, charts, and user surveys which include contextual recommen-
dation of social updates, and context similarity analyses. We provide the following
scripts:
* *genCtxReport* generates a CSV worksheet containing a list of most appearing tags, and most weighted tags, from one or several context index files.

* *genCtxLogChart* generates a CSV worksheet which can be easily rendered as a gantt-like chart to represent the repartition of users' not-null contexts on a time-line, given the corresponding context index files.

* *genCtxSim* generates two reports, given users' context index files. *CtxMatchReport* is a CSV worksheet that contains a matrix providing the cumulative number of contexts that are similar to other users' contexts, per user and per similarity threshold. *CtxSimReport* lists, for each user pair combination, the twenty most similar contexts, with their score and top common tags, between those two users.

* *genCtxSimChart* generates a CSV worksheet which can be easily rendered as a chart to represent the repartition of users' similar contexts on a time-line, given the corresponding context index files. The similarity threshold and the size of the temporal window can be set. The temporal window is the maximum temporal distance between two contexts which similarity will be evaluated, in number of pulses. From this worksheet, it is possible to render a dot-chart of similar contexts, in function of their similarity score and time.

* *recommender* generates a report which lists the relevance scores between every context and social update, given context index and social update index files.

* *genSurvey* generates survey forms in HTML, given a set of user's context index and social update index files. For each user survey, five contexts are selected and displayed on the form. And for each context, three social updates from other users are selected and displayed on the form. Each user can rate contexts, and the relevance of proposed social updates to those contexts. At the bottom of the page, a button enables to send those ratings by email.

* *genCtxFeedReport* generates a survey form in HTML, given a user's context index and social update index files. For each social update posted by the user, the corresponding contextual cloud is also displayed, and the user can rate the perceived relevance of his/her social update with this context. At the bottom of the page, a button enables to send those ratings by email.
## 5.7 Conclusion

In this chapter, we have specified the requirements of our context management framework, proposed to model contextual information as ‘*contextual clouds*’ (based on the vector space model), and a methodology for building and evaluating our filtering application based on this approach. Then, we identified several dimensions of context that can be leveraged for measuring the contextual relevance of users, and we designed an algebra for manipulating contextual clouds. We identified four filtering modalities, proposed a data flow and defined a general relevance matching function for our filter. Finally, we described and explained the implementation of this framework, the software modules involved in a real-time filtering application, and in the generation of experimental reports to evaluate the quality of the algorithms.

In the next chapter, we develop a social awareness application upon this framework, and propose an evaluation of the underlying contextual model and filtering model.
# Chapter 6

## A Social Awareness Application
and its evaluation

In the previous chapter, we designed a tag-based context management framework that can gather contextual information from physical, virtual and social sensors, and a general social matching scheme that can be used for filtering social updates.

In this chapter, we propose to consider an enterprise environment as a case study, and we apply the general social matching scheme by implementing a social awareness application, called ‘*Enterprise Contextual Notifier*’, for this specific environment. This application considers users’ web-browsing context as relevance criteria for filtering social updates between employees.

### 6.1 Case study

As a motivating case, we propose to consider an enterprise environment where employees work on individual networked computers. Employees know just a portion of other employees (mostly people working in the same department), and are spread across several offices in several cities. This traditional enterprise is organized following a hierarchical structure of managers to coordinate the efforts of workers and transfer information to the relevant parties.

We propose to set up an internal social networking system that allows every worker to share and be aware of the current status of relevant colleagues and their on-going professional activities. The time necessary for maintaining this network must be low,
and unnecessary task interruptions must be prevented. This social networking sys-
tem must allow employees to reflect their current thoughts, activities, intentions and
questions, support them for getting feedback and answers from relevant colleagues, to
promote transversal communication and collaboration across hierarchical silos.

6.2 Application of the approach

6.2.1 Enterprise context model

As users are working on computers in the case study presented above, most contextual
information about them (and their current activity) can be extracted from the software
they use. In particular, we consider that their context is defined by the projects,
subjects and topics covered by the web pages they are currently browsing. Indeed, the
web is often used to find some reference on an ongoing task, and thus the description
of these references can give a clue of the user's current activity. Therefore, users'
contextual clouds will be populated with tags that describe the web pages currently
being browsed, as depicted on Figure 6.1.

<figure><img src="image_56.png" /><figcaption>Figure 6.1: A sample contextual tag cloud, in an enterprise environment -</figcaption></figure>

6.2.2 Web-browsing-based context sensors

In chapter 3, we have seen that documents are traditionally represented by a vector
containing words that appear the most frequently in their content. In order to eval-
uate the quality of crowd-sourced/human annotations of web pages, we combine five
additional techniques for extracting descriptive tags from web documents: social book-
marking tags (annotations given by human readers), search query terms (also entered
by humans), document metadata (provided by the authors of the web pages), completed with semantic content and resource location analysis (for web pages that have not been annotated by humans).

We define six functions to extract weighted tags from browsed documents:

1) the Metadata function

counts the number of occurrences of each term *t* with different coefficients (*α*, *β*, *γ*),
depending of the position of this term in document *d*'s metadata:

$$
w_1(t, d) = \alpha * |t \in T_d| + \beta * |t \in K_d| + \gamma * |t \in D_d|
$$

where $|t \in T_d|$ is the number of occurrences of the term *t* in the *title* of the document *d*, $|t \in K_d|$ in its *keywords* set, and $|t \in D_d|$ in its *description* text.

2) the SearchQuery function

counts the number of occurrences of term *t* in a search query *Q*<sub>*d*</sub>, when the analyzed
document *d* contains a list of search results:

$$
w_2(t, d) = |t \in Q_d|
$$

This function relies on the Google<fnref n="1" /> search engine.

3) the DomainNames function

adds the domain names (including sub-domains) $N_d$ from document *d*'s URL as terms:

$$
w_3(t, d) = |t \in N_d|
$$

4) the SocialBookmarks function

counts the number of people who publicly bookmarked the document *d* using each term
*t* as tag:

$$
w_4(t, d) = \sum_{p \in P} tag(p, d, t)
$$

where each person *p* is in the set of people *P* that are using this social bookmarking
service, and where *tag(p, d, t)* has a value of 1 or 0, whether or not this person *p*

<footnote n="1">¹Google: http://www.google.com/</footnote>
# 6. A SOCIAL AWARENESS APPLICATION AND ITS EVALUATION

bookmarked the document *d* using term *t* as a tag. This function relies on the Delicious<fnref n="1" /> social bookmarking service.

## 5) the SemanticAnalyzer function

counts the number of occurrences of semantically-defined entities (i.e. concepts and instances) that are represented by each term *t*, when they are identified in the document *d*:

$$w_5(t, d) = |t \in R_d|$$

$$R_d = [\forall e \in E_d, repr(e)]$$

where *repr(e)* is the textual representation of a semantic entity *e*, *R<sub>d</sub>* is the set of textually represented entities *E<sub>d</sub>* found in the document *d*. This function relies on the SemanticProxy<fnref n="2" /> web service.

## 6) the KeyphraseExtractor function

counts the number of occurrences of each term *t* of *K<sub>d</sub>*; a set of keyphrases extracted from the document *d*:

$$w_6(t, d) = |t \in K_d|$$

This function relies on KEA, the Keyphrase Extraction Algorithm (Witten *et al.*, 1999).

### 6.2.3 A comparison of tags gathered from virtual and social sensors

In order to evaluate the quality of these tag extractors for synthesizing consistent contextual clouds that can represents users' context, based on the web pages that each user is browsing, we generated a contextual cloud with each sensor on a same web page and compared the results. We ran this experiment on a page entitled 'Getting started with extension development', which is about Mozilla Firefox extension programming. From the resulting contextual tags depicted on Figure 6.2, we can draw the following conclusions:

<footnote n="1">¹Delicious: http://delicious.com/</footnote>

<footnote n="2">²SemanticProxy: http://semanticproxy.opencalais.com/</footnote>
* The HTML-based Metadata sensor mostly emphasized keywords that were found in the title of the page. In our case, the page is quite well described by its title, that is why the results are satisfying. Nevertheless, this is not the case on all web pages.

* The KEA-based KeyphraseExtractor returned several relevant keywords (e.g. *firefox*, *xul*), including higher-level concepts (e.g. *'extension development'*), but also too many meaningless keywords (e.g. *ifest*, *EM*, *files*).

* The Semantic Proxy-based sensor picked up well-recognized terms such as names of standards. Indeed, *RDF*, *DTD*, *XML* and *XUL* technologies are very relevant keywords for the given page that are correctly emphasized by their high weight (based on the number of occurrences in the content). Lower weighted keywords are also relevant with the content of the page, but they do not really reflect its topic.

* Finally, the Delicious-based SocialBookmarks sensor returned fewer keywords but they better describe the topic of the page, and their weights are more heterogeneous, emphasizing the most consensual keywords (that are also the most descriptive, in this case), such as: *firefox*, *extension*, *development*. Even lower weighted keywords describe the topic and the utility of the page quite well, thanks to common sense emerging from the crowd of del.icio.us' users: *programming*, *tutorial*, *xul*.

<table><caption>Figure 6.2: Tag clouds extracted using four methods -</caption><thead><tr><th>Simple HTML meta</th><th>KEA</th><th>Semantic Proxy</th><th>Delicious</th></tr></thead><tbody><tr><td>extension profile folder<br>started<br>development development<br>resources knowledge install rdf<br>contents rdf creating toolbar buttons base -<br>adding terms to menu extension development dev<br>tps: disable xul cache getting started with extension<br>development mozillazine<br>profile manager packaging extensions</td><td>helloworld/chrome.man<br>ifest<br>chrome<br>EM<br>Folder<br>overlays<br>Firefox<br>files<br>XUL<br>extension development<br>MozillaZine</td><td>Ted Mielczarek (1)<br>Google (1)<br>Mozilla Development Center (1)<br>MDC (2)<br>DTD (10)<br>XUL (4)<br>Javascript (3)<br>US (2)<br>XML (7)<br>Linux (1)<br>XUL Planet (1)<br>rdf (12)</td><td>extension javascript<br>tutorial mozilla<br>firefox xul<br>development<br>programming howto<br>extensions</td></tr></tbody></table>
From this preliminary experiment, we conclude that the results from the Social-
Bookmarks sensor are the most suitable for describing current user's context based on
his/her web browsing session. Nevertheless, we must keep in mind that not every web
page is tagged on this service. Therefore, it is necessary to include tags from other
sensors, so that any web page can generate keywords that can be represented in the
contextual cloud.

6.2.4 Software implementation

The application described above was implemented in a modular architecture in which
software modules communicate through RESTful HTTP requests. In this section, we
present these modules.

<figure><img src="image_57.png" /><figcaption>Figure 6.3: Flow of the contextual aggregation and recommendation process -
in the enterprise application</figcaption></figure>

Figure 6.3 depicts this architecture and flow as it is currently implemented in our
application:

Firefox sniffer

The *Firefox*<fnref n="1" /> *plug-in* is a JavaScript-based extension which hooks on the browser's
events related to opening, closing and switching of web pages. Each occurrence of these

<footnote n="1">¹http://www.firefox.com/</footnote>
events is transmitted with the corresponding URLs to the local *Context Aggregator* for extracting tags from the sensors.

Table 6.1 lists the types of events being considered (extension of Mozilla's *visit_type* specification<fnref n="1" />).

<table><caption>Table 6.1: Types of browsing events handled from Firefox</caption><thead><tr><td><em>actiontype</em></td><td>Description</td></tr></thead><tbody><tr><td>1</td><td>The user opened a new page, by following a hyperlink</td></tr><tr><td>2</td><td>The user opened a new page, by typing its URL</td></tr><tr><td>3</td><td>The user opened a new page, by selecting a bookmark</td></tr><tr><td>5</td><td>The user was redirected to a new page (permanent redirect)</td></tr><tr><td>6</td><td>The user was redirected to a new page (temporary redirect)</td></tr><tr><td>7</td><td>The user downloaded a document</td></tr><tr><td>8</td><td>The user focused on a page (when several are opened at once, e.g. in tabs)</td></tr><tr><td>9</td><td>The user closed a page</td></tr></tbody></table>

## Context Aggregator

The *Context Aggregator* handles events (triggered by users' actions) with their attached contextual information, and runs sensors' *weighting functions* on this information to produce an aggregated and normalized contextual cloud to be submitted to the *Contextual Filter*. It also behaves as a typical web server, in order to generate the user interface that is displayed locally in Firefox's side-bar, as explained later in this section.

Table 6.2 enumerates the parameters that are expected by the *logAction* REST service, handled by the Context Aggregator.

Our *Context Aggregator* also implements the *weighting functions* defined previously in this section: six different interpreters turn URLs into contextual clouds. The *Metadata* interpreter parses the *title*, *description* and *keywords* elements from the HTML source code of each web page to produce the corresponding weighted terms. The *SearchQuery* interpreter extracts query terms from Google Search result pages. The *SocialBookmarks* interpreter gathers tags given by users about a web page, when existing on the Delicious social bookmarking service. The *SemanticAnalyzer* gathers

<footnote n="1">¹http://www.forensicswiki.org/wiki/Mozilla_Firefox_3_History_File_FORMAT</footnote>
# 6. A SOCIAL AWARENESS APPLICATION AND ITS EVALUATION

<table><caption>Table 6.2: Specification of parameters for events sent by the Firefox sniffer to the Context Aggregator</caption><thead><tr><td>Parameter</td><td>Description</td></tr></thead><tbody><tr><td>source</td><td>Identification of the sniffer<br>default: <em>webbrowser</em></td></tr><tr><td>localtime</td><td>Local timestamp of the event<br>format: UNIX/epoch timestamp (milliseconds)</td></tr><tr><td>actiontype</td><td>Type of event: <em>added</em>, <em>focus</em> or <em>removed</em></td></tr><tr><td>tabs</td><td>Array of URLs of currently opened web pages</td></tr><tr><td>focus</td><td>URL of the web page which currently has the focus</td></tr></tbody></table>

textual representations of semantic entities that were identified in the web page, thanks to the SemanticProxy web service. And the *Keyphrase Extractor* runs the Keyphrase Extraction Algorithm (Witten *et al.*, 1999) on the textual content of each web page.

## Contextual Filter

The *Contextual Filter* handles contextual clouds gathered and interpreted by users' *Context Aggregator*, computes relevance scores between them, and recommend best-ranked social updates to each user. Social updates are gathered by subscription to the users' declared third-party social feeds/streams (e.g. their Twitter account). An internal social networking system has also been implemented in order to experiment the application without relying on third-party accounts. In our online implementation of the framework, the recommendations are displayed in near-real-time in a sidebar of Mozilla Firefox, thanks to an AJAX interface provided also by this server.

## Side-bar (user interface)

The *side-bar* user interface is implemented as a XUL overlay, as a part of the Firefox plug-in introduced above. The content of this side-bar is generated as a HTML page by the local *Context Aggregator*. It consists of displaying the current contextual tag clouds, dynamically evolving as the user browses the web, a messaging field (called *tweet*) with a submission button, and a list of recommended social updates based on the last submission.
<figure><img src="image_58.png" /><figcaption>**Figure 6.4:** Screenshot of the *side-bar* user interface in Firefox - Simultaneous points of view for two users. The user's current contextual cloud is displayed on the top. It can be submitted to the Contextual Filter with an optional social update, by pressing the 'post' button. Recommended social updates are listed below, according to the last submitted contextual cloud.</figcaption></figure>

On Figure 6.4, we present the simultaneous points of view of two users: Adrien and John. Adrien is browsing web pages about *context-awareness*, including the definition of this concept on Wikipedia. John is browsing web pages about *ubiquitous computing*, including a page from *Georgia Tech* (known as *gatech*). Their contextual tag clouds respectively represent the terms corresponding to these pages, and relevant social updates are recommended according to their contextual tag cloud. On Adrien's screen, the first recommendation is a social update from John, who provides the URL of an *interesting* web page, apparently about *gatech*. In gray, the relevance score of this social update is shown: 35%. On John's screen, Adrien's last social update is recommended, also with a relevance score of 35%. This score represents the similarity between Adrien's and John's contextual tag clouds. Here, the terms *context* and *awareness* appear with a significant weight on both of them.

This online software implementation is functional and gives a good sense of the benefits of our approach.
6.3 Context-based relevance of social updates, an evalua-
tion

In order to evaluate the validity of our hypothesis on relevance of contextually recom-
mended social updates, we gathered browsing logs and social updates from volunteers
during two weeks, ran our algorithms on these logs to generate 1846 contextual clouds,
and asked the volunteers to rank the quality of selected results. In this section, we spec-
ify the requirements of the evaluation, describe the experimentation plan and setup we
followed, explain the data processing phase, then discuss the results obtained.

**6.3.1 Evaluation requirements**

The evaluation of our hypothesis relies on the quality of two data processing steps:
(i) the extraction of weighted contextual tags by sensors' contextual weighting and
aggregation functions, and (ii) the recommendation function based on the similarity of
those contextual clouds.

According to the eighth claim recommended by Terveen & McDonald (2005), ‘*Eval-
uations of social matching systems should focus on users and their goals*’, we decided
not to rely on existing evaluation data sets such as the ones from TREC, nor to follow a
scenario-based experiment. Instead, we leverage usual browsing behavior of the users,
and their own social updates, with their consent during the period of the experiment.

The scores expected from volunteers are threefold: (i) the representativity of con-
textual tag clouds, (ii) the accuracy of the relevance function for recommending social
updates that are contextually relevant, and (iii) the relevance of social updates with
the context of their author at the time of posting.

**6.3.2 Experimentation plan**

As explained above, we decided to let volunteers browse web sites and post social
updates as they usually do, without constraints or guidelines. The evaluation period
consists of a preparation phase, followed by three steps (as depicted on Figure 6.5):

1. The recruitment and briefing of volunteers: we presented and explained the fol-
lowing experimentation plan to potential volunteers and provided the necessary
software and support to install it for gathering log data. Additionally, we gathered volunteers' social feeds (e.g. their twitter accounts) to which they would post social updates, so that we can capture those for the experiment.

2. The data gathering period: during one week, volunteers browsed web pages using Firefox, while the provided sniffing extension was logging the required browsing events to a local database. At the end of this period, we provided a script to export these log entries to a human-readable text file, so that they could remove privacy-critical entries if needed (e.g. private activities, and other noisy data that is irrelevant to this study), and then send us this file.

3. The processing period: we ran our indexing and matching algorithms (as defined in the previous chapter) on the browsing logs and social updates provided by volunteers, to produce personalized survey forms containing ranked recommendations for each volunteer.

4. The volunteer feedback period: we asked each volunteer to fill two personalized surveys. In the first survey, for five contextual tag clouds generated from their own web browsing data, we ask each volunteer to rate the readability, representativity, need for modifications before sharing, and willingness to share these contextual tag clouds in an enterprise environment. In order to support them in remembering those contexts, we provide the list of web pages that were being browsed by the volunteer at that time. Still for each contextual tag cloud, we ask volunteers to rate the relevance of three random social updates in this context, one of those being a well-ranked match. In the second survey, volunteers rate the relevance of their own social updates with their contextual tag cloud at the time of posting.

<figure><img src="image_59.png" /><figcaption>Figure 6.5: Experimentation plan -</figcaption></figure>
6.3.3 Experimental setup and recommendation process

In order to gather experimental results, we generated personalized surveys for our eight
volunteers, all being office workers (including researchers) of the same company. In this
subsection, we present the process that we followed and the parameters that we set to
generate these personalized surveys from the web browsing logs provided by volunteers.

Browsing logs contain a list of browsing events defined by: the timestamp of event,
the type of event (i.e. page opened, focused, or closed), and the URL of the page
concerned by this event. We parsed these log files to gather a global list of URLs, and
executed the tag extraction and weighting functions. Some of those rely on querying
web services (e.g. Delicious, SemanticProxy), and every page also had to be downloaded
to extract tags from its metadata information (i.e. title, keywords and description).
The *Metadata* weighting function was parameterized with factors α = 50 per term
appearing in the title, β = 10 in the *keywords* field, and γ = 1 in *description* field of a
document. We stored the tag clouds resulting from the execution of tag extraction and
weighting functions for each URL, in a file called *TagCache*, so that we could reproduce
the execution of our indexing algorithm.

Because the experimental software was not interactive, we indexed contextual and
social logs (provided as ATOM/RSS feed files by volunteers) on a common time line
with a period of 10 minutes. When indexed, web browsing events are called *context
snapshots*. A context snapshot is composed of a contextual tag cloud that aggregates
all the weighted tags extracted from web pages that were opened, focused and closed
since the previous snapshot. Indexing a social update consists of associating it with
the contextual cloud of the last context snapshot at the time of posting this update. If
there is no known context information in the previous snapshot, we use the one before
the previous. Every indexed contextual cloud is processed to split multiple-word tags,
cleaned from punctuation and other non-literal characters, filtered against a stop-words
list, and then normalized so that the sum its tags' weights equals 1. Only the first 20
tags (with highest weights) are displayed. As shown on Figure 6.1, a contextual tag
cloud can contain diverse kinds of terms, such as words in various languages, word
combinations and acronyms.

Then, we ran the recommendation algorithm on the indexed contextual and social
logs in order to produce a relevance matrix for each participant. In order to generate
a participant's personalized survey, we selected 5 indexed heterogeneous contexts (i.e. the most dissimilar to each other) that were matched (by the recommender) with at least one highly-ranked social update.

The second survey was simply generated from users' feed files.

6.3.4 Results

As specified in the evaluation requirements, the results are threefold: we gathered scores
given by every participant on (i) the representativity and sharing of five contextual
clouds selected from his/her own, (ii) the perceived relevance of three selected social
updates (from other participants) for each of those contextual clouds, and (iii) the
relevance of the participant's own social updates with the context of their posting.

Representativity and sharing

In order to rate the representativity and sharing of contextual tag clouds, we asked
each participant to report on a 4-point Lickert scale their answers to the four following
questions, for each of the five personalized contextual tag clouds they were given:

* Readability: *Does it contain English/French words?* Proposed answers are: 1 - none, 2 - few are words, 3 - list of words, or 4 - list of well-spelled words.

* Reflectivity: *How well do these words reflect your browsing activity?* Proposed answers are: 1 - not at all, 2 - might give a vague clue, 3 - quite well, or 4 - very representative.

* Privacy: *Who would you share this contextual tag cloud with?* Proposed answers are: 1 - no one ever, 2 - to someone specific, 3 - to a known set of people (friends, family, co-workers), or 4 - to anyone, I don't mind.

* Sharing in the enterprise: *Would you share such contextual tag clouds with other employees of your company?* Proposed answers are: 1 - never, 2 - would only keep it for personal usage, 3 - yes if it could support work, or 4 - yes, even in real time on a blog.

As depicted in Figure 6.6, the average readability is rated 3.05 (i.e. on our [1, 4]
grade rating range), corresponding to a score of 68.33% (i.e. translated to a [0,1] grade
<figure><img src="image_60.png" /><figcaption>Figure 6.6: Distribution of ratings on contextual clouds -</figcaption></figure>

range). We observe that these ratings are quite homogeneous for each participant, despite the heterogeneity of proposed tag clouds. According to the fact that some clouds were similar across participants, we can assume that this rating is rather a general one (i.e. applies to all tag clouds, as seen in websites). The average reflectivity is rated 2.93, corresponding to a score of 64.17%. Also quite homogeneous, we observe some exceptional values in these ratings, showing that the percieved reflectivity depends more on the contextual nature of our tag clouds, and thus the set of tags extracted from web pages. These results confort us in the sense that, despite the novelty of this representation for contextual information, the concept and visualization of contextual tag clouds is understandable by users.

With an average rating of 2.38 (45.83%), the privacy acceptance rating reveals heterogeneous intended audiences for all participants, depending on the contents of the contextual clouds. With an average rating of 2.25 (41.67%), the enterprise sharing ratings are lower, and not correlated with the privacy ratings. Except one participant who would not mind transparency of his/her contextual clouds in the enterprise, most users would rather keep those for personal usage. This can be explained by the category of volunteers: enterprise workers that are not all so familiar with social networking practices. It can also be explained by the fact that not all contexts (or some of their tags) were work-related, as it was not possible for them to manipulate their contexts in this experiment.
Relevance of recommendations

As explained previously, social updates proposed to users are voluntarily not all relevant. Our goal is to observe a correlation between the relevance scores given by participants and the rankings computed by the context-based recommender. Thus, we rely on a Mean Percentage Error (based on MAE, Mean Absolute Error) to define the following accuracy function:

$$accuracy = 1 - \sum_{q=1}^{Q} |sim(C_q, U_q) - rating(C_q, U_q)|$$

in which, for each proposed social update $q$, $sim(C_q, U_q)$ is the relevance score of the social update $U_q$ with the contextual tag cloud $C_q$, as evaluated by the ranking algorithm. Whereas, $rating(C_q, U_q)$ is the actual relevance score, as given by the volunteer. Both scores are values in the range [0, 1], represented as percents. As $rating()$ scores are given by volunteers in the [1, 4] grade rating range, they are converted to percents with the following formula:

$$rating = \frac{(grade - 1)}{3}$$

<figure><img src="image_61.png" /><figcaption>Figure 6.7: Average precision values for each participant -</figcaption></figure>

As depicted on Figure 6.7, the precision values vary between 56% and 84%, with an average value of 72%.

On Figure 6.8, we observe that the distribution of relevance ratings given by participants on the 120 recommended social updates is quite close to algorithm expectations.
<figure><img src="image_62.png" /><figcaption>Figure 6.8: Comparative distribution of recommendation ratings - user-given and predicted</figcaption></figure>

As a natural behavior of recommender systems, the best-ranked ratings (mostly in *Rank 3*) are slightly overestimated by the recommendation algorithm, whereas low relevance ratings (*Rank 1*) given by participants are higher than expected.

We observe that 63% of relevance ratings expected by the recommendation system are low ranked (*Rank 1*), whereas medium-high (*Rank 3*) ratings were expected for 19% of these ratings. The high number of low-ranked scores and the medium ranking of better scores expected by the algorithm reveals that highly similar contextual tag clouds were rare in our small scaled experiment. By increasing the number of participants, more similar contexts would be found, thus the average scores would naturally increase.

## Relevance of contextualized social updates

For the third part of our study, in order to prove that contextual clouds are consistent reference documents for recommending social updates, we asked the participants to rate the relevance of each of their own social updates (e.g. their status updates and social bookmarks) to the contextual cloud representing their current situation at time of posting/sharing.

Over a total of 59 social updates, their authors rated an average relevance to context of 50.3%. Figure 6.9 shows the following distribution of ratings: 19 social updates were ranked 1 (low relevance), 10 were ranked 2, 14 were ranked 3, and 16 were ranked
<figure><img src="image_63.png" /><figcaption>**Figure 6.9: Comparative distribution of relevance ratings of social updates - to their context, per type**</figcaption></figure>

4 (high relevance). These social updates are gathered from several social feeds: 54% are status updates posted on Twitter, 29% are bookmarks instantly shared through Delicious (social bookmarking service).

By further analyzing these specific types of social streams, we discovered an average relevance score of 71% for shared bookmarks, and 38% for status updates from Twitter. It is natural that new bookmarks are more relevant to their context, as the web document that is bookmarked is usually being browsed by the user, and thus represented in the corresponding contextual tag clouds. Concerning status updates, Naaman et al. Naaman *et al.* (2010) proved that only 41% of social updates from Twitter are actual statuses about the current activity of the person (categorized as "me now" by the authors). The similarity of this proportion with our average contextual relevance score for status updates gives some proof, although preliminary, about the consistency of our results.

### 6.3.5 Discussion

In this section, we studied the potential of our novel social update recommendation approach based on contextual clouds. Despite the small scale of our experiment, our results are promising. Although participants are still quite reluctant towards sharing contextual information, they showed a positive acceptance of contextual tag clouds for representing their browsing activity. The average accuracy of selected social updates: 72%, is significant for a web recommender system. With the average relevance of social updates to their context of emission: 50.3%, we showed that our hypothesis is partly valid, depending on the type of social updates.
6.4 Matching contexts from virtual and social sensor tags,
a comparative analysis

In the previous section, we evaluated the accuracy of social update recommendation
based on contextual relevance. In order to estimate relevance, our system relies on
similarities between contextual clouds, generated by aggregating descriptive tags asso-
ciated to the web pages browsed by users. These associations are extracted from five
mechanisms relying on:

* four virtual sensors: leveraging HTML metadata and semantic entities recognized in those web pages, names extracted from sub-domains of the resource locator (URL), and search query terms entered by the user;

* and one social sensor: leveraging tags given by users to describe web pages, on a social bookmarking service.

In this section, we analyze the differences between these sensors for enabling contextual matching, based on the experimental data introduced in the previous section.

6.4.1 Description of the data

Among a list of 14625 unique URLs accessed by participants during our experiment,

* semantic entities were recognized and turned to tags from 5292 URLs by SemanticProxy,

* sub-domain tags were extracted from 3979 URLs,

* meta-tags were extracted from the title, description and keywords annotated in the HTML source code of 2898 URLs,

* descriptive tags were given on 1395 URLs by users of the Delicious social bookmarking service,

* and search query terms were gathered from 1050 Google search URLs.

It is to be noted that a rather minor portion of those 14625 URLs have been
successfully described by tags, using our virtual and social sensors. This ratio can be
explained by several reasons which prevented the sensors from analyzing them:
* some of the URLs were local (stored on the user's computer), and thus invisible to some sensors;

* some pages were personal, or protected by credentials, thus also invisible to sensors;

* some pages do not contain any HTML content, e.g. pages embedding Flash or Java components;

* some pages are not referenced by any user in the Delicious social bookmarking service;

* and it is also possible that some of those URLs were unreachable by sensors at the time of our indexing process, e.g. temporary server breakdown, loss of connectivity, or even sensor failure.

These reasons further justify the use of additional sensors that do not necessarily rely on the Internet, but none were used in the frame of our experiment.

6.4.2 Analysis of results

In order to compare the efficiency of virtual and social sensors for enabling context matching, we generated several charts depicting the number of matched contexts for each user and the distribution of similarity scores. Here, we rely on Delicious to represent social sensor data, and on SemanticProxy to represent virtual sensor data. Indeed, we observed similar results between SemanticProxy and other virtual sensor data.

On Figures 6.10 and 6.11, we represent a cumulative histogram showing the number of context matches involving each user, for ten similarity threshold values. The total number of context matches depends on the number of 10-minute chunks that were represented by tags, extracted respectively by the SemanticProxy sensor (Fig. 6.10) and by the Delicious sensor (Fig. 6.11). The more chunks associated with tags, the higher the number of contexts that can be matched with other users’ contexts. The average number of matched contexts is lower for the SemanticProxy sensor (127718) than for the Delicious sensor (156696); this observation also applies to other virtual sensors. Despite the lower number of URLs tagged on Delicious, this opposite difference is explained by the fact that the number of contexts of a user mostly depends on the
specific combinations of URLs he/she has browsed. For example, it is possible that all
URLs were opened by only one user, in one 10-minute chunk, thus indexed into only
one contextual cloud. In that case, no contexts would have been matched, despite the
high number of URLs.

Concerning the distribution of the similarity scores of matched contexts over the threshold values, we observe that in average 31.46% of Delicious-based matches have a similarity score higher than 0.1 (a score of 1 being maximal, representing contextual equality), whereas this portion represent only 5.72% of SemanticProxy-based matches. This means that, despite the higher number of URLs described as tags by SemanticProxy, Delicious performed better in providing tags that can be used for matching contexts. This distribution is further emphasized on Figures 6.13 and 6.14. We observed similar distributions among all virtual sensors.

<figure><img src="image_64.png" /><figcaption>Figure 6.10: Tags from SemanticProxy: Cumulative distribution of context matches - by similarity threshold, for each user</figcaption></figure>

Figures 6.12 and 6.15 show the same charts, after aggregation of tags from the
five sensors (virtual and social). With an average value of 330228 matched contexts,
from which 8.72% are above the 0.1 similarity score threshold, we observe that this
aggregation actually reduces the number of matched contexts above this threshold:
<figure><img src="image_65.png" /><figcaption>Figure 6.11: Tags from Delicious: Cumulative distribution of context matches
- by similarity threshold, for each user</figcaption></figure>

<figure><img src="image_66.png" /><figcaption>Figure 6.12: Tags from all sensors: Cumulative distribution of context matches
- by similarity threshold, for each user</figcaption></figure>
<figure><img src="image_67.png" /><figcaption>Figure 6.13: Number of matching contexts using only tags from SemanticProxy - per similarity threshold</figcaption></figure>

<figure><img src="image_68.png" /><figcaption>Figure 6.14: Number of matching contexts using only tags from Delicious - per similarity threshold</figcaption></figure>
<figure><img src="image_69.png" /><figcaption>Figure 6.15: Number of matching contexts using tags from all sensors - per similarity threshold</figcaption></figure>

32470, whereas 58008 were attained by using tags from Delicious only.

6.4.3 Discussion

With this analysis, we have proven the efficiency of tags gathered from social sensors for finding high similarity scores between users' contexts. In our case, we observed higher scores by relying on tags from the Delicious social bookmarking service than from virtual sensors: the SemanticProxy-based semantic entity extraction, tags extracted from HTML meta-data, URL sub-domain names, and Google search query terms.

This analysis also reflects that aggregating tags from various sensors implies lower similarity scores than relying solely on tags extracted by the Delicious social sensor. Indeed, adding tags extracted from various sensors imply a higher fragmentation of tags' weight. By applying the cosine similarity function on richer vectors (i.e. higher dimension space) with lower weights, the probability of resulting higher scores is inevitably reduced.

As explained earlier, the aggregation of complementary sources of tags (virtual and
social sensors) is however necessary to ensure a maximal coverage of contextual entities.
Relying solely on social bookmarking services would not be enough, as a small portion of web pages are tagged by users. In order to increase the chances of finding higher scored similarities while leveraging several sensors, the dimension of resulting term vectors must be reduced. For example, tags could be clustered by semantic proximity (i.e. merge synonyms, hypernyms and hyponyms into one tag, or hierarchies), or noisy tags could be identified and removed. We have identified such techniques in Chapter 3, but we did not have enough time to apply them in this project. Nevertheless, these issues are interesting directions to explore in the future.

## 6.5 Conclusion

In this chapter, we applied our general context management framework and social update filtering scheme to develop a social awareness application for computer-based enterprise environments. In this application, contextual tags are gathered from the web pages currently browsed by users, using one social sensor based on the Delicious social bookmarking system, in addition to five virtual sensors. The architecture and the implementation of its modules were described, and an experiment was carried out to evaluate our framework, and support the underlying hypothesis.

Despite the small scale of our experiment, our results are promising:

* Firstly, the average accuracy of contextually-recommended social updates: 72%, is significant for a web recommender system. However, concerning the perceived relevance between their own social updates and their corresponding contextual cloud (at the time of posting), we observed an average score 50.3% from participants. We discovered that this score was lower than expected, because status updates (a subset representing 54% of the number of social updates posted during our experiment) were often not related to the context of their author. Therefore, our hypothesis could only be validated if most social updates were actually related to the context of their author, which usually is true for only 41% of status updates, according to Naaman et al. (2010).

* Concerning the efficiency of social sensors for extracting highly descriptive contextual information from crowd-sourced repositories, in contrast to more traditional content-based approaches, we appreciated the representativity of tags associated
by the Delicious social bookmarking site. We also observed that those tags im-
plied more matches in looking for similar contexts, than tags extracted using
virtual sensors. This result proves the efficiency of our novel context modeling
approach based on crowd-sourced repositories.
6. A SOCIAL AWARENESS APPLICATION AND ITS EVALUATION
# Chapter 7

## Conclusion

In this final chapter, we summarize our research, we recall our contributions and our original findings. Regarding our results, we propose recommendations and best practices concerning social awareness systems and tag-based information management. In addition to our answers to fundamental questionings, we also asked new other questions for which we propose some opportunities for future developments and improvements.

### 7.1 Research summary

#### 7.1.1 Research context

In chapter two, and in (Joly et al., 2009; Subercaze et al., 2009), we provided a detailed presentation of various social networking systems (including microblogs), their common and specific characteristics and functions, and analyzed their usage through the enumeration of salient trends and major evolutions that we have observed since their appearance on the Internet. In particular, we presented the results of our survey about the cognitive disruptions (e.g. interruptions) caused real-time social communication platforms, such as the well-known Twitter micro-blogging service. In our analysis of those results, we made three major observations:

* More than half respondents (59%) follow the social feeds of more than 100 people, and 28% follow more than 250 people, which is higher than the Dunbar number;

* Many respondents (35%) are notified of new social updates in real time, and 62% read most of those updates;
* A majority of respondents (75%) did not set up any filtering mechanism, whereas more than 59% would like some support to filter social updates.

We supported that applying filtering techniques to those feeds could also reduce the frequency of cognitive interruptions (i.e. distraction) caused by real-time notifications of new updates. In particular, we identified that contextual information about users could be leveraged to filter social updates by relevance.

In chapter three, we reviewed several research efforts on improving social awareness, communication and collaboration using computer-based techniques. We identified information filtering and recommendation techniques that could be leveraged to regulate information streams between people. Inspired from content-based and collaborative filtering techniques, we proposed to develop a hybrid filtering system relying on tags inferred from documents currently opened by users, to represent their context, as published in (Joly, 2009a). Instead of a semantic representation, we chose a plain keyword representation for tags, as we identified that semantics could emerge from rich sets of tags (e.g. using co-occurrence and latent semantic analysis).

In chapter four, and in (Joly et al., 2008), we proposed a state of the art of ‘context-aware’ computing, as we have identified user context as a good criteria for filtering social updates by relevance. We identified scalability problems in most systems relying on semantic/ontology-based context models, supporting our choice for a keyword-based model for representing users’ context. We also found out that many context management systems, contrarily to popular context-aware application and services that do not rely on a general context management system, imply privacy concerns, preventing users from adopting applications relying on those systems. We concluded that contextual information should remain in control of their users, allowing them to visualize and modify it before sending it to a server. We described three kinds of context sensors:

* physical sensors transmit information that is sampled from a physical environment (e.g. geographical positioning of users and other entities);

* virtual sensors synthesize contextual information from computer-based activities;

* and social sensors can provide emergent contextual information from crowd-sourced content by combining various web services.