Salient event-detection in video surveillance scenarios Surveillance video data is accumulating at a staggering rate, making its manual handling impossible. Therefore, automatic tools for analysis and processing of such data are highly needed. In most video surveillance scenarios the most interesting parts of the recorded data are those where an unusual event takes place. The rest of the data, which is actually representing the greatest part, relates to usual or normal activities that are of no real value to the security task and thus its viewing, storage and processing are pure wast of resources. Therefore, automatically finding the exact spot in a surveillance video sequence where an interesting event occurred is of great importance financially and to take timely actions.In this project we investigate the detection of remarkable events in video surveillance scenarios. We look into how to distinguish events in surveillance scenarios, and further what is a remarkable event. We specifically focus our attention on the event of object dropping in public places such as airports and train stations. We try to answer some of the following questions: Is it possible to create a system for modeling salient events in surveillance scenarios? How does one determine what stands out as a remarkable event? How to distinguish between less remarkable events and more remarkable event taking place? The large amount of video surveillance data collected every day is growing very fast. This surveillance data is usually monitored by very few human operators who are overwhelmed by the number of cameras and the constant alertness such task requires. Therefore, it is becoming an important challenge to detect and respond to all abnormal events in real time or to search for an interesting event that was recorded in a previous but unknown date. In this kind of tasks we are interested in events that deviate from the normal and usual behavior of the people in the monitored space. Not having automatic tools at hand this can be a daughnting task for the a human operator, who would have to sequentially view all raw video data, and use his judgment to determine if an event is unusual or not.. In this this article we will look into the possibility of modeling abnormal events for automatic detection as seen in [8, 5, 6] . By analyzing objects behaviors [11] in video sequences over time it should be possible to define a set of criteriaŠs for events, which deviate from normal behavior. Before doing this there must be a clear understanding of what characterizes as an event in a surveillance scenario, and what distinguishes events in the sense of what is abnormal and what is not.. We specifically focus our attention on the event of object dropping in public places such as airports and train stations etc. as mentioned in [3, 13, 2, 12, 9] . In this initial work on salient event detection in video sequences; to model salient events we started off by performing a visual subjective analysis on a video training set. To be able to get the understanding of what was actually happening in the video sequence before, during and after the event of object dropping. By examining several videos with the same type of event we discovered that it was possible to define a set of criteria for the event. The set of criteria was set to act as a set of decision rules; thus an object dropping had occurred if all these predefined rules were fulfilled.. We automatically extracted several simple features from the objects in their entire presence in the scene. A combination of these low-level features did contain sufficient information to make a description on what characterizes the abnormal event of object dropping.. By making a general description of what the event in question contains, before, during and after the actual point of the object dropping we were able to derive a set of predefined decision rules. If the feature analysis fulfilled these criteria we could conclude that an object dropping had occurred, and we are able to pinpoint where this event occurred in the video sequence.. This video analysis was performed during the initial phase of our work and in the future we are planning to generalize this kind of analysis other types of events. Additionally, we will investigate automatic classification of events based on their feature vectors and learning mechanisms. Analyzing surveillance data, without the knowledge of when and where or even if an interesting event has occurred often takes place, and is time and labor intensive. In this kind of analysis the analyst is interested in extraordinary events, things that deviate from what is normal. Not having the suitable tools it can be a discouraging task for the analyst, to sequentially view all raw video data, and using his judgment to determine if an event is unusual. Currently the most accurate method, for abnormal event detection, is still based on human operator and manual labor however. Meaning that an operator sits and pays close attention to multiple screens of live video feeds from surveillance cameras.. There is a growing need to develop automatic systems to perform such tasks. Several published research papers [1, 4, 5] , and books [14] discussed such systems. Mecocci et al. [1] introduced a way to, automatically in real-time, detect abnormal events. Video surveillance systems today show two main drawbacks, namely, not being adaptive to different operative scenarios (they only work in well known and structured world), and they generally need the assistance of a human operator in order to recognize and tag specific visual events. The authors describe a system capable of automatically adapting to different scenarios without any human intervention, and use robust self-learning techniques to automatically learn the typical behavior of the targets in each specific operative environment. It uses an improved version of the Altruistic Vector Quantization algorithm (AVQ), which is a rework of the original idea by Johnson et. al. [7] . The system is capable of running in realtime, after a preliminary run-in period of about 40 minutes, in order to learn all the typical visual trajectories.. The authors of [4] suggest an automatic surveillance system based on a robust and simple algorithm for foreground and tracking of multiple objects in complex environments. It is explained that it is possible to obtain enough information from the characteristics of the detected objects. The system analysis the video stream and gives an alarm to call for the operators attention if the features fulfill predefined requirements. It is based on very simple algorithms in order to allow a single computer to analyze up to eight video streams in real-time. Blob and scene characteristics are used to create a low-level description of predefined events. It detects events by comparing the sequence's parameters with those associated with the events for the current scenario.. In [5] the authors present an unsupervised technique for detecting unusual activity in a large video set using many simple features. Low-level features, as mentioned in [10] , are defined to be those basic features that can be extracted automatically from an image without any shape information (information about spatial relationship). Thresholding is a form of low-level feature extraction performed as a point operation. Similar approaches can be used in high-level feature extraction, where we find shapes in images. In the proposed technique there are no complex activity models and no supervised feature selections used. The video is divided into equal length segments and extracted features are classified into prototypes, from which a prototype-segment co-occurrence matrix is computed.. A method to utilize the hard to describe but easy to verify property of unusual events without building explicit models of normal events are presented here. One compares each event with all others observed to determine how many similar events exists. If there are many similar events, one can conclude that the events are most likely to be normal, and not of much interest. If an event is not similar to others it is most likely to be an unusual and interesting event. Detecting unusual events in a large data set does not require modeling of normal events, but rather the ability to compare two events and measure their similarity.. The overall goal is to extract simple and reliable features which are descriptive, which is similar to goal of this article, i.e. can be used by an unsupervised algorithm to discover the important image features in a large video set in order to detect any unusual events.. In the rest of this paper we will present the proposed system for event detection. Section 3 gives an overview of the system and Section 4, shows some initial experimental results. Conclusions and future work are presented in Section 5.

B O O B B O B B O O O O O O O O O O O O O O O O B O O O O O O O O O O B B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O B O O O O O O O O O O O O O O O O B O O O O B B O O O O O O O O O B O O O O O O O B O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O B O O O O O O O O O O B O O O O O O B B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O B B B O B O O O B O O O O O O O O O B O O B O O O O O O O O O O O O O O O O B O O O O O O B O B O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O B B O O O O O O O O B O O O O O O O O O O O B O O O B O O B B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O


A study of mass-mailing worms Mass-mailing worms have made a significant impact on the Internet. These worms consume valuable network resources and can also be used as a vehicle for DDoS attacks. In this paper, we analyze network traffic traces collected from a college campus and present an in-depth study on the effects of two mass-mailing worms, SoBig and MyDoom, on outgoing traffic. Rather than proposing a defense strategy, we focus on studying the fundamental behavior and characteristics of these worms. This analysis lends insight into the possibilities and challenges of automatically detecting, suppressing and stopping mass-mailing worm propagation in an enterprise network environment. In recent years the Internet has experienced an increasing number of malicious programs propagating through electronic mail. High-profile worms such as I love you [1] and Sircam [2] are but a few historic al examples. The more recent outbreak of MyDoom [4] substantially degraded network services and was directly responsible for a massive Distributed Denial of Service (DDoS) attack. An earlier outbreak, SoBig [3] , utilized sophisticated hybrid propagation techniques to replicate over network shares and emails, resulting in a rapid infection rate that surpassed those achieved Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. previously. Both SoBig and MyDoom disrupted normal network operations and caused unexpected downtime and an increase in associated IT expenses.. The severity of these attacks can be attributed to a numbe r of factors, including efficient mechanisms for targeting victims and insufficient defense mechanisms. Previous work on defendi ng email-based attacks focuses primarily on filtering techniques implemented at the outgoing mail servers [16] . Unfortunately, newer instances of mass-mailing worms come equipped with their own SMTP engines [3, 4] , and therefore are capable of bypassing filters or detection mechanisms deployed at the outgoing mail server. An alternative approach would be examining incoming mails. Unfortunately, the prevalence of email messages with malicious conten t makes it difficult and costly to perform comprehensive filtering before the emails reach end users. The worm is then unlikely to be detec ted in the end user's mailbox because most users update their anti-virus software/signatures infrequently, if at all. As a resul t, we will continue to see outbreaks of email-based attacks, the frequency and sophistication of which will only increase with time.. In this paper we present an in-depth study of two massmailing worms, SoBig and MyDoom, from real traffic traces. The purpose of this paper is not to propose any particular defense mechanism. Rather, we focus on studying fundamental behavior and characteristics of the worms, which may lead to new insights to automatically detect, suppress and stop their propagation.. The remainder of this paper is organized as follows. Section 2 describes related work. Section 3 gives a brief background on SoBig and MyDoom. Section 4 describes the trace data set. Section 5 describes the analysis techniques and presents the the results of our analysis. Section 6 discusses the implications of our findings and we conclude in Section 7. From our network trace data analysis, it is clear that SoBig and MyDoom both induced noticeable abnormalities in the traffic of the infected hosts. In addition, during massmailing worm outbreaks, the normal traffic patterns of outgoing mail on the mail servers were also disr upted, despite the fact that infected hosts used their own SMTP engines to forward mail. We conjecture that these disruptions are due to the backscatter mail traffic. In addition, anti-virus software deployed on the mail server is not only ineffect ive against mail worms, it also generates excess traffic by sending infection notifications to spoofed addresses.. We examined our trace data in a number of ways, all of which exhibited abnormalities at the time of the worm outbreaks. These included exploring the aggregate number of outgoing TCP flows, the number of outgoing TCP flows to distinct destination ad dresses, and the relationship between TCP flows and DNS traffic. We found that each "viewpoint" provided us with unique insight into characterizing and contrasting each worm's behavior and im pact on our local network. One interesting observation gathered from our data is that SoBig was generally more virulent than MyD oom. SoBig caused more outgoing infection attempts per infected host and contacted more distinct destination addresses. Recent work in rate limiting schemes such as traffic throttling [15] and secure NICs [8] show promise for mitigating widespread worm attacks. However, these techniques have not been tested using sufficiently realistic network data. In addition, we have outlined some problems in using these techniques to defend specifically against mail worms. Future work will focus on alternative defense approaches that are more tolerant of a diversified network environment. In this section we describe our study of the SoBig and MyDoom traces. We begin by examining traffic patterns before and during the worm outbreaks. More specifically, we investigated change in the rate of outgoing TCP, SMTP, and DNS-related traffic. We found that traffic anomalies were easily detectable from monitoring client machine traffic patterns.. For our analysis, it is advantageous to identify the infected hosts and contrast their behavior with that of normal hosts. We devised a simple heuristic to identify infected hosts.. 2 The heuristic involves contrasting outgoing SMTP connections from a host before and during the outbreak. An uninfected host should make no or very few outgoing SMTP connections (other than to its designated mail server). If a host changes its behavior by creating a large number of SMTP connections during the outbreak, it is marked as a candidate for being infected. We further refined the heuristic by considering the sizes of the message payload-those with payload similar to that of the worms reported by Symantec are suspects [3, 4] . Note that this is only a heuristic and we cannot identify infected hosts with 100% certainity. Using this heuristic, we identified 5 infected hosts during the SoBig outbreak and 6 infected hosts during the MyDoom outbreak. In the analysis that follows, those infected hosts refer to the sets identified by this heuristic.. In the following sections we analyze data before, during, and after the worm outbreaks. We study the number of outgoing TCP traffic flows, the number of distinct IPs contacted by TCP flows, and the relationship between DNS traffic and TCP flow data. In each section we present results by comparing average traffic of normal clients to the infected clients. We also present the effects of the mass-mailing worms on the outgoing traffic patterns of mail servers. From the TCP flows statistics, we can see that SoBig employed a more aggressive propagation strategy than MyDoom. Although there were less clients infected by SoBig in our network, the total volume of flows generated by its infected clients was 50% greater than MyDoom's. Our data suggests that traffic analysis on the mail server would be ineffective against MyDoom, since mail server traffic during the MyDoom outbreak does not appear to be distinctively unusual. However, for Sobig, it is possible that one can detect its presence due to the large volume increase in the SMTP flows.. Contrary to our original belief that distinct IPs contacted should remain relatively stable during outbreaks of email worms (since domain names are scanned from web caches and files from victims machines), hosts contacted an noticeably large number of distinct mail servers. This is interesting because it means that rate limiting schemes such as Williamson's [15] that work by limiting the rate of distinct IPs contacted can still be effective. However, it should be noted that an email worm may send a large number of emails to the same mail server, hence straightforward . Our study of the infected hosts shows that associating DNS translations with TCP flow data reveals a spike in DNS related traffic, and it hints at the "secondorder" effects of the worm, such as additional DNS traffic generated and additional data loaded into the local nameservers' caches. We also observed that during times of infection client translations uncharacteri stically resulted in more refreshes than new entries. This phenomenon could provide the basis for an interesting detection or de fense strategy, such as rate-limiting DNS responses that only contain refreshed entries to clients.. What was not accounted for in our study was local DNS traffic, which may look similar to the the figures in Sec-tion 5.1 if little address caching is performed on the end hosts. Also unaccounted for are intermediate translations that result from "walking the DNS hierarchy" to arrive at the DNS server delegated for a particular host name. If these translations are included, we could see a sizable increase in DNS traffic attributed to worm activity. Given the increase in both TCP flows and DNS translations on the infected clients and mail servers, one would expect that similar trends exist in the aggregate traffic of the network. However, we found that HTTP traffic significantly dominates TCP network traffic and DNS translations. Figure 10 shows aggregate SMTP-related, HTTP-related, and TCP-related DNS translations in our network for the period of SoBig. Figure 11 shows the overall successful outgoing TCP flows in our network for the same period. In the previous section we studied the effects of mass mailing worms from a variety of perspectives. We found that mass-mailing worms' activity on end host machines is apparent whether one monitors the number of outgoing TCP flows, the number of distinct IP addresses contacted by those TCP flows, or the DNS lookup pattern required to resolve the address. An advantage of monitoring DNS patterns to detect a worm is that one could implement a mail-worm watchdog at the DNS server, rather than analyzing all outgoing packets at the router. Containment possibilities at the DNS server are also intriguing. In particular, the DNS server may be able to slow down the rate that mass-mailing worms propagate by delaying answers to translation requests. Another possibility is that a DNS server could attempt to "quarantine" suspicious emails by replacing the address of the requested mail server with that of a mail "holding bay". These suspicious mails could then be inspected by a more heavy weight filtering mechanism.. The analysis in Section 5.1 shows that mail worms that create messages with a legitimate return address will place additional load on mail servers by causing noticeable "bounced" mail messages to the domains. In our analysis, we discovered that SoBig caused more "backscatter" mail traffic than MyDoom, presumably because SoBig was more aggressive in attempting to guess addresses or because MyDoom often did not provide a properly formed return address. Any assessment of the load placed on a network by a worm should factor in this "spoofed mail" effect, as well as the increase of DNS traffic attributable to it (see Figure 9) . In order to limit the load on the mail server, one could employ a strategy limiting the number of bounced mail messages (for example, those in response to mail with attachments).. Additionally, we found that most DNS traffic generated by the worms was to refresh unexpired entries in the DNS cache rather than to input new entries. This is intuitive, because addresses associated with mail servers often have longer TTLs. This suggests that the additional DNS traffic generated by worms may be superfluous, and that loads on DNS servers may be minimized by honoring the TTLs of their current entries rather than refreshing them prematurely. Finally, the total number of outgoing flows of overall traffic shows that DNS traffic attributed to TCP is dominated by HTTP, not SMTP. Therefore, in order to detect the presence of a mass mailing worm, it is necessary to apply additional filtering techniques to overall traffic. By examining traffic on the SMTP port or looking at sets of particular hosts or servers, we were able to distinguish the worms' presence in an academic network setting. Future work will focus on a more in depth investigation of the DNS traffic, with an eye towards separating IPs corresponding to MX (mail exchange) records from others. If this proves fruitful, it may be helpful to add logic to the DNS server that flags internal hosts requesting an abnormal number of mail server address translations as part of a wider defense or detection mechanism.. Our study suggests existing defenses specifically designed for monitoring outgoing mail on SMTP servers do not work for newer types of mass mailing worms, because almost all of these worms have their own SMTP engine. Defenses should instead be deployed where most network traffic can be seen -at the edge router or at individual hosts. In addition, selective filtering should be employed, or noise from other TCP traffic (e.g. HTTP) will obscure the traffic patterns of email worms.. Williamson's [15] and Ganger's [8] schemes will be less effective against email worms than random scanning worms. Ganger's detection scheme is based on using a secure network interface to detect and rate-limit connections to IP addresses that were not introduced to the host by a DNS translation. Since email worms require DNS lookups (for MX records) to send mail to their targets, this scheme would be ineffective.. Although the data did suggest a spike in the distinct IP addresses contacted by the infected hosts, Williamson's scheme [15] that restricts connections to different IP addresses could be easily circumvented: the worms we observed sent an average of 10 infected emails to each IP address it contacted, and this ratio could be further increased by sending more infection attempts to a single friendly (or compromised) external SMTP server. Williamson also proposed a different scheme that would rate limit mail to distinct email addresses [16] . Other work has shown such rate limiting schemes are ineffective unless implemented on almost all host machines [17] . Another defense mechanism is for a router to filter out all outgoing SMTP traffic from non-mail servers. This solution would be possible and quite effective in an enterprise network setting where clients are mostly homogeneous. However, it may prove to be too restrictive for academic computing environments.. Future worms may attempt to hide under the radar by sending emails at a slower rate, or by intelligently delegating targeted addresses across other infected intranet hosts. We believe that an effective traffic monitoring scheme would use different filtering mechanisms and correlate their results. For instance, traffic monitoring could be done on both perclient and aggregate basis, and traffic could be monitored using a variety of metrics, including the number of total flows and unique destination IPs per each destination port. The more specifically we can examine traffic, and the more vantage points we employ to examine it, the easier it will be to mitigate false positives and detect abnormal traffic. Several documented studies have investigated malicious code and the ways in which it propagates. Moore et al. [11] analyzed the propagation of the scanning worm Slammer and its effect on the Internet as a whole. Our work instead focuses on the effects of mass-mailing worms o n a single subnet.. Researchers have also studied the use of epidemiological models to model the spread of a virus or worm within a network [7, 10, 13, 14] . Staniford et al. [12] presented a study of different types of worms and how they can cause damage on the Internet. Zou et al. [18] modeled the propagation of the Code Red worm. These studies focus on propagation simulation and our work differs by using traffic analysis to identify anomalies and behavior of email worms.. In the area of worm defense, Williamson [15] proposed the idea of host-based rate limiting by restricting the number of new outgoing connections. He further applied this mechanism to email worms by rate limiting emails to distinct recipients [16] . Wong et al. [17] studied the effects of various rate limiting deployment strategies. Ganger et al. [8] proposed a scheme that analyzes and limits network traffic based on abnormal DNS lookup patterns. Chen et al. [6] devised a detection mechanism based on the premise that a worm-infected host will have more failed connections. Gupta et al. [9] used a heuristic based approach for detection of email worms. Our approach is different from previous work since we focus on the studying the fundamental behavior of email worms. Our work is focused on email worms. In this section, we give a brief description of email worms and the general characteristics that set them apart from other instances of malicious code.. An email worm is a program that propagates by sending copies of itself to recipients via electronic mail. Once a recipient opens the email attachment, the malicious program executes on the victim's machine and further propagates itself. Typically, the worm program chooses its targets by harvesting email addresses from the victim's address book, web cache, and hard disk. The worm then sends mails containing its own code to targets in an attempt to repeat the infecti on cycle.. Email worms are different from scanning worms such as Code Red and Slammer [5, 18] . The latter typically exploits a vulnerability (e.g., buffer overflow) in the network services running on a target machine, and they must perform IP scanning to find vulnerable targets. Email worms infect hosts by tricking users into inadvertently executing malicious code. Unlike a scanning worm, which needs to aggressively search for new victims to compromise, an email worm has much higher hit rate because it obtains targets from victim machines.. Unlike traditional email viruses, mass-mailing worms do not limit their targets strictly to those in a victim's address book. W orms like SoBig and MyDoom utilize aggressive spreading techniques such as harvesting legitimate domain names from victim machines (e.g., by scanning web caches and hard disks) then attempting to construct probable addresses. These worms also use social engineering techniques (e.g., disguise as a system error message) to lure users to open t he malicious attachment. Some mass-mailing worms even spread using avenues other than mail. For in stance, both SoBig and MyDoom replicated over file sharing clients (e.g., Kazza).. Both SoBig and MyDoom exhibit unique and atypical traffic patterns. For instance, clients infected with SoBig and MyDoom used their own SMTP engines in propagation attempts. Under normal circumstances, connections to outside SMTP servers are highly unusual for most enterprise clients.

O O O O B O B O O O O O O O B O B O O B O O O O O O O O O O O O O O B O O B B B O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O B O O O O B O B O O O O O O O O O O O O O O O B O O B B O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O B B O O O O B O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O B O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O B O O O B B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O B O O O O O O O O O O O O B O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O B O O O O O O O O O B O O O O O O O B I O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O B O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O


Package

Fingerprints:

A

visual

summary

of

package

interface

usage

Context:

Object-oriented

languages

such

as

Java,

Smalltalk,

and

C++

structure

their

programs

using

packages.

Maintainers

of

large

systems

need

to

understand

how

packages

relate

to

each

other,

but

this

task

is

complex

because

packages

often

have

multiple

clients

and

play

different

roles

(class

container,

code

ownership,

etc.).

Several

approaches

have

been

proposed,

among

which

the

use

of

cohesion

and

coupling

metrics.

Such

metrics

help

identify

candidate

packages

for

restructuring;

however,

they

do

not

help

maintainers

actually

understand

the

structure

and

interrelationships

between

packages.

Objectives:

In

this

paper,

we

use

pre-attentive

processing

as

the

basis

for

package

visualization

and

see

to

what

extent

it

could

be

used

in

package

understanding.

Method:

We

present

the

Package

Fingerprint,

a

visualization

of

the

references

made

to

and

from

a

package.

The

proposed

visualization

offers

a

semantically

rich,

but

compact

and

zoomable

views

centered

on

packages.

We

focus

on

two

views

(incoming

and

outgoing

references)

that

help

users

understand

how

the

package

under

analysis

is

used

by

the

system

and

how

it

uses

the

system.

Results:

We

applied

these

views

on

four

large

systems:

Squeak,

JBoss,

Azureus,

and

ArgoUML.

We

obtained

several

interesting

results,

among

which,

the

identification

of

a

set

of

recurring

visual

patterns

that

help

maintainers:

(a)

more

easily

identify

the

role

of

and

the

way

a

package

is

used

within

the

system

(e.g.,

the

package

under

analysis

provides

a

set

of

layered

services),

and

(b)

detect

either

problematic

situations

(e.g.,

a

single

package

that

groups

together

a

large

number

of

basic

services)

or

opportunities

for

better

package

restructuring

(e.g.,

removing

cyclic

dependencies

among

packages).

The

visualization

generally

scaled

well

and

the

detection

of

different

patterns

was

always

possible.

Conclusion:

The

proposed

visualizations

and

patterns

proved

to

be

useful

in

understanding

and

maintaining

the

different

systems

we

addressed.

To

generalize

to

other

contexts

and

systems,

a

real

user

study

is

required.

To

cope

with

the

complexity

of

large

object-oriented

software,

developers

organize

classes

into

packages

or

modules.

This

organization

usually

follows

conceptual

interrelationships

between

classes,

that

the

stake-holders

would

like

to

maintain

over

ineluctable

system

evolution.

As

the

system

modular

structure

changes,

its

maintenance

is

required.

However,

where

approaches

of

system

remodularization

succeed

in

proposing

system

refactorings

they

do

not

provide

good

ways

for

understanding

and

assessing

the

changes

they

propose.

There

is

a

wide

range

of

work

to

define

new

modularization

algorithms

but

little

support

to

understand

the

proposed

results

and

their

impact

on

existing

systems..

It

is

important

to

understand

the

concrete

organization

of

packages

and

their

interrelationships.

Ideally,

packages

should

be

kept

as

less

coupled

and

as

much

cohesive

as

possible

We

distinguish

two

main

approaches

of

package

cohesion

in

the

existing

literature

The

first

approach

defines

the

cohesion

of

a

package

in

terms

of

the

interconnections

between

its

internal

classes.

The

second

approach

defines

cohesion

according

to

how

the

system

uses

the

package

classes.

For

instance,

if

two

classes

of

a

package

are

used

from

the

same

client

package,

then

they

are

considered

as

conceptually

related,

regardless

of

the

explicit

relationships

that

exist

between

them

This

second

approach

is

more

meaningful

to

us,

because

we

consider

a

package

as

a

functionality

provider

and

not

only

a

structural

grouping

of

coupled

classes.

Many

metrics

of

package

cohesion

have

been

defined

and

help

maintainers

determine

packages

that

are

candidates

for

restructuring.

However,

those

approaches

do

not

help

maintainers

of

large

systems

when

they

face

the

problem

of

understanding

how

packages

are

structured

in

general

and

how

packages

are

in

relation

with

each

other

in

their

provider/client

roles.

Severalpreviousworksonsoftwarevisualizationprovideinformation

on

packages

and

their

relationships,

by

visualizing

software

artifacts

or

metrics

about

their

structure

or

evolution

While

these

approaches

are

valuable,

they

fall

short

of

providing

a

fine-grained

view

of

packages

that

would

help

maintainersunderstandthestructureofpackages,theirinterrelationships

within

the

system,

and

identify

their

roles

within

a

system..

In

this

article,

we

present

the

Package

Fingerprint,

a

compact,

rich

and

zoomable

visualization

to

better

support

the

understanding

of

a

package

and

its

relationships.

The

goal

of

this

visualization

is

to

help

maintainers

during

their

early

contacts

with

unknown

packages.

We

propose

two

complementary

variants

of

the

Package

Fingerprint,

structured

around

the

distribution

of

references

from

or

to

the

classes

of

the

analyzed

package:

the

incoming

fingerprint

shows

how

the

system

uses

the

package

classes,

and

highlights

the

cohesion

of

the

analyzed

package,

as

defined

in

the

outgoing

fingerprint

shows

how

the

package

classes

use

the

system.

This

article

is

an

extension

of

previous

work

The

new

contributions

presented

here

are:

the

description

of

the

outgoing

fingerprints

and

their

use,

the

application

of

fingerprints

to

large

industrial

systems,

and

additional

fingerprint

patterns..

In

Section

we

discuss

the

challenges

for

understanding

packages.

Then

we

present

the

principles

of

the

incoming

and

the

outgoing

fingerprints

in

Section

In

Section

we

show

how

to

use

the

incoming

fingerprint

for

analyzing

and

understanding

packages

in

practice.

Section

presents

the

different

zoom

levels

of

a

fingerprint

and

shows

how

to

read

a

fingerprint

from

far

away.

Section

presents

the

outgoing

fingerprint

via

a

simple

example,

and

Section

lists

the

relevant

visual

patterns

in

fingerprints.

Finally,

we

discuss

our

approach

and

related

works

in

Sections

and

In

this

paper,

we

tackled

the

problem

of

understanding

the

details

of

package

relationships

from

a

usage

perspective.

We

described

the

package

fingerprints,

and

their

use

as

a

visual

approach

for

understanding

package

relationships,

contextual

cohesion,

and

the

conceptual

coupling

of

their

classes.

While

designing

fingerprint,

we

exploited

pre-attentive

processing

using

color

properties

and

saving

placeholders

principle.

We

also

introduced

interactivity

and

multi-selection

mechanism

to

help

the

user

during

the

analysis

task..

We

successfully

applied

the

visualization

to

several

large

systems

and

we

have

been

able

to

quickly

point

out

badly

designed

packages,

and

to

extract

relevant

patterns.

While

applying

fingerprints

to

large

systems

that

contain

radically

different

packages

in

terms

of

size

and

references,

the

visualization

generally

scaled

well

without

hindering

the

detection

of

different

patterns.

Several

works

focus

on

understanding

packages.

These

can

be

by

supporting

high-level

analyses

of

package

relationships,

visualizations,

software

metrics

or

package

evolution..

High-level

analyses.

We

are

interested

here

on

those

based

on

visualizations.

Sangal

et

al.

adapted

the

dependency

structure

matrix

(DSM)

from

the

domain

of

process

management

to

analyze

architectural

dependencies

in

software

DSM

presents

a

con-sistent

visualization

that

offers

a

system

overview.

While

the

visualization

scales

for

large

systems,

it

is

poor

in

terms

of

precise

information

about

the

package.

DSM

cells

contain

a

number

indicating

the

number

of

references

made

between

packages.

However

DSM

did

not

focus

on

packages

cohesion

and

co-use

or

co-usage

of

classes..

X.

Dong

et

al.

present

the

High-level

Object

Dependency

Graph

(HODG)

that

helps

capturing,

from

a

high-level

point

of

view,

possible

usage

dependencies

among

coarse-grained

software

entities,

namely

packages.

In

their

approach,

they

interpret

the

usage

dependencies

between

classes

in

the

context

of

their

hierarchy

and

present

a

new

graph

of

the

system

under

analysis.

While

the

given

graph

is

helpful

for

understanding

the

considered

system

from

a

high-level

point

of

view,

it

does

not

give

any

information

about

package

cohesion

nor

about

the

co-use

or

the

similarity

between

classes.

Also,

their

graph

visualization

still

difficult

to

be

interpreted

by

human

eyes

because

within

it,

the

nodes

have

different

sizes

but

without

any

meaningful

dimension.

The

HODG

has

not

visual

semantics

and

it

uses

numbers

to

visualize

almost

information..

Program

visualization

and

navigation.

Package

Fingerprints

are

based

on

similar

principles,

but

provide

more

visual

information

and

help

identify

groups

of

packages

with

similar

dependencies.

A

fingerprint

exploits

pre-attentive

processing

using

color,

contrast,

and

the

principle

of

placeholders.

In

addition,

a

fingerprint

by

focusing

on

a

package

at

a

time

qualifies

in

a

finer-grained

way

the

dependencies..

A

Package

Blueprint

presents

a

condensed

view

of

a

package

in

terms

of

the

references

made

between

packages.

It

acts

as

a

map

and

puts

in

situation

the

references

between

packages.

While

package

blueprint

provides

a

compact

view

and

shows

dependencies

on

a

per-class

basis,

it

does

not

help

to

group

the

client/provider

packages

in

terms

of

their

dependencies

to

the

package

under

analysis..

Kuhn

et

al.

used

information

retrieval

to

exploit

linguistic

information.

He

introduced

semantic

clustering

to

group

source

artifacts

that

use

similar

vocabulary

He

uses

vocabulary

topics

to

reveal

the

intention

of

the

code

and

the

similarity

between

its

artifacts,

then

he

provides

a

consistent

visualization..

Several

works

explore

packages

and

their

structure

but

few

of

them

reveal

information

on

their

relationships

and

dependencies.

In

Softwarenaut,

Lungu

et

al.

help

system

discovery

by

guiding

exploration

of

nested

packages

Storey

et

al.

also

worked

on

system

exploration,

supporting

zoom-out

facilities

and

forcesbased

graph

layouts

However

the

work

did

not

focus

on

couse

or

co-usage

of

classes..

Software

Metrics.

There

is

a

plethora

of

software

metrics

on

cohesion:

from

the

simple

and

bogus

LCOM

to

more

advanced

LCOM*

metrics

Ponisio

et

al.

introduced

the

notion

of

use

cohesion

which

is

at

the

foundation

of

the

fingerprint.

E.

Hautas

defines

a

new

metric

that

indicates

the

percentage

of

changes

to

be

made

in

order

to

make

a

package

structure

acyclic

While

he

focuses

only

on

the

cyclic

dependencies,

he

does

not

provide

any

utility

that

helps

understanding

packages

or

indicating

their

cohesion

or

similarity..

Package

evolution.

A

number

of

approaches

give

summarized

information

on

package

relationships

and

their

evolution:

the

Butterfly

by

Ducasse

et

al.

gives

a

high-level

client/provider

trend

of

package

dependencies

Pzinger

et

al.

show

the

evolution

of

package

metrics

using

Kiviat

diagrams

Chuah

and

Eick

use

rich

glyphs

to

characterize

software

artifacts

and

their

evolution

(number

of

bugs,

number

of

deleted

lines,

kind

of

language...)

In

particular,

the

timewheel

exploits

pre-attentive

processing,

and

the

infobug

presents

many

different

data

sources

in

a

compact

way;

finally,

D'Ambros

et

al.

reveal

package

coupling

by

showing

evolutions

that

are

correlated

in

time

Co-evolution.

Other

works

treat

and

visualize

information

about

software

co-change

evolution,

looking

at

coupling

from

a

temporal

perspective,

and

software

development

teams

and

activities

Such

approaches

are

completementary

to

ours

in

the

sense

that

we

only

focus

on

the

static

nature

of

the

packages

and

their

relationships.

While

those

approaches

are

valuable

and

provide

fine-grained

views

of

packages

that

may

help

understanding

the

contextual

coupling

and

cohesion

inside

packages,

they

fall

short

on

the

analysis

of

a

single

version

of

a

system.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O


Quadratic Kernelization for Convex Recoloring of Trees The CONVEX RECOLORING (CR) problem measures how far a tree of characters differs from exhibiting a so-called "perfect phylogeny". For an input consisting of a vertex-colored tree T , the problem is to determine whether recoloring at most k vertices can achieve a convex coloring, meaning by this a coloring where Comput. Syst. Sci. 74:850-869, 2008) who showed that CR is NP-hard, and described a search-tree based FPT algorithm with a running time of O(k(k/ log k) k n 4 ). The Moran and Snir result did not provide any nontrivial kernelization. In this paper, we show that CR has a kernel of size O(k 2 ). The historically first and most common definition of fixed-parameter tractability (or, in short: in FPT) for parameterized problems is solvability in time O(f (k)n c ), where n is the input size, k is the parameter, and c is a constant independent of n and k. Background and motivation for the general subject of parameterized complexity and algorithmics can be found in the books [13, 14, 23] . This basic view of the subject is by now well-known.. Less well-known is the point of view that puts an emphasis on FPT kernelization. For an extensive overview of kernelization, see [17] . Kernelization is central to FPT as shown by the lemma:

O B O B I O B O B I O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O B O O O O O O O O O


Refunctionalization at Work We present the left inverse of Reynolds's defunctionalization and we show its relevance to programming and to programming languages. We propose two methods to transform a program that is almost in defunctionalized form into one that is actually in defunctionalized form, and we illustrate them with a recognizer for Dyck words and with Dijkstra's shunting-yard algorithm.Revised version of BRICS RS-07-7. * This tutorial documents an invited talk at MPC '06 [26] . † IT-parken, This article is a continuation of Danvy and Nielsen's earlier article "Defunctionalization at Work" [33] , that outlined the extent to which Reynolds's defunctionalization [59] is pervasive in writing and transforming programs, and in specifying and implementing programming languages. Our goal here is to show that the left inverse of defunctionalization, i.e., refunctionalization, is also relevant to programming and to programming languages. We first review defunctionalization and its left inverse, refunctionalization (Section 2). We then propose two steps for restating programs from almost being in defunctionalized form to actually being in defunctionalized form: disentangling and merging apply functions (Section 3), and we illustrate these techniques with two parsing examples: recognizing Dyck words (Section 4) and Dijkstra's shunting-yard algorithm (Section 5). After reviewing other applications of refunctionalization (Section 6), we conclude (Section 7). We have investigated the applicability of refunctionalization for transforming programs and implementing programming languages. Elsewhere, we have illustrated its relevance to programming [13, 30, 33] and to specifying programming languages [2-4, 11, 24, 32] .

B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Foundations of Instance Level Updates in Expressive Description Logics In description logic (DL), ABoxes are used for describing the state of affairs in an application domain. We consider the problem of updating ABoxes when the state changes, assuming that update information is described at an atomic level, i.e., in terms of possibly negated ABox assertions that involve only atomic concepts and roles. We analyze such basic ABox updates in several standard DLs, in particular addressing questions of expressibility and succinctness: can updated ABoxes always be expressed in the DL in which the original ABox was formulated and, if so, what is the size of the updated ABox? It turns out that DLs have to include nominals and the '@' constructor of hybrid logic for updated ABoxes to be expressible, and that this still holds when updated ABoxes are approximated. Moreover, the size of updated ABoxes is exponential in the role depth of the original ABox and the size of the update. We also show that this situation improves when updated ABoxes are allowed to contain additional auxiliary symbols. Then, DLs only need to include nominals for updated ABoxes to exist, and the size of updated ABoxes is polynomial in the size of both the original ABox and the update. Description Logics (DLs) are a traditional family of knowledge representation formalisms which, in recent years, have played an important role as a logical underpinning of ontology languages such as the W3C recommendation OWL [1] .. Email addresses: liuhkai@tcs.inf.tu-dresden.de (Hongkai Liu), clu@uni-bremen.de (Carsten Lutz), maja.milicic@manchester.ac.uk (Maja Miličić), frank@csc.liv.ac.uk (Frank Wolter) In DLs, a knowledge base (KB) typically consists of two parts: a TBox to store intensional knowledge, i.e., a general formalization of the relevant concepts and relationships of the application domain; and an ABox to store extensional knowledge, i.e., instance level assertions that describe the current state of affairs in the application. Just like database systems, DL knowledge bases are not static entities, but have to be modified when the application domain evolves. This raises the fundamental update problem, which consists of rewriting the knowledge base to incorporate new information from the application without unnecessarily losing any existing knowledge. In the case of a DL knowledge base, at least three different incarnations of the update problem can be distinguished:. • TBox updates, triggered by changes of the intensional knowledge of the application domain;. • ABox updates, which have to be carried out when the intensional knowledge remains stable, but the state of affairs in the application changes;. • KB updates, which do not distinguish between the two levels of knowledge and allow simultaneous modification of the TBox and the ABox.. In typical applications, instance level knowledge stored in the ABox tends to change frequently whereas intensional knowledge in the TBox often remains stable for longer periods of time. Moreover, automatic TBox modifications are rarely desired because the TBox is typically the result of a careful and timeconsuming manual modeling process, and thus its syntactic structure should not be changed in a radical way. These observations lead us to study ABox updates as a fundamental and basic form of updates in a DL context. A central property of DL ABoxes is that they store incomplete knowledge, reflected by an open world semantics and the use of compound logical expressions that can involve disjunction and existential quantification. It follows that, technically, updating DL ABoxes is equivalent to updating logical theories, a problem with a long tradition in both the database and AI communities [4, 5, 6, 7, 8] .. In the database and AI literature, for a long time no proper distinction was made between updates as studied in this paper and the related notion of a revision. While the purpose of update is to bring the knowlegde base up to date when the world described by it changes, revision aims at incorporating new knowledge that was obtained about a static world. Katsuno and Mendelzon [6] discuss this distinction in detail, show that update and revision are fundamentally different operations, and give 8 postulates that any rational update operator should satisfy. The protoypical update semantics that complies with these postulates is Winslett's well-known PMA semantics [4] whose general idea can, in our context, be stated as follows. The models of the original knowledge base K are viewed as those states of the world that are considered possible; when K is to be updated with new information U , then the models of the resulting updated knowledge base K should satisfy U , but also be 'as close as possible' to the models of K (the principle of minimal change). In the case of updating propositional theories and logical databases as considered in [6, 4] , the difficulty of defining what 'as close as possible' means mainly derives from the following two features: (i) the newly added information may be non-deterministic, e.g. when it involves disjunction; and (ii) the updated theory must satisfy additional domain constraints stated in the form of a logical background theory. As discussed in more detail in Section 6 of this paper, the combination of these features with the first-order quantification present in description logics leads to serious semantic difficulties and also to computational problems. For this reason, we concentrate on a simple, yet fundamental form of update where (i) the newly added information U consists of a set of ground literals, i.e., sets of ABox assertions A(a) or r(a, b) and their negations, where A is a concept name and r a role name (thus both are atomic); and (ii) no background theory is present, i.e., the knowledge base K comprises only an ABox, but no TBox. In this case, there seems to be only one sensible formalization of 'as close as possible': the models of K are obtained from the models of K by (deterministically!) applying the changes dictated by the ground literals in U . This semantics, which we adopt in the current paper, can thus be viewed as an incarnation of Winslett's semantic that avoids the potentially controversial cases.. As a starting point for the current paper, we observe that, in standard 'expressive' DLs such as those between ALC and ALCQIO, we can find an ABox A and update U of the restricted form described above such that the result of updating A with U cannot be expressed in the given DL. As a concrete example, take the following ABox A, which is formulated in ALC, the basic expressive DL with Boolean operators. It states that John is a parent with only happy children, that Peter is his child, and that Mary is a person: To understand why A is appropriate, note that A provides no information about whether or not Mary is a child of John. Because we cannot exclude that this is the case, John may now have an unhappy child, which is Mary. Thus, the new knowledge concerning Mary also resulted in an update of the knowledge concerning John. Using the nominal {mary} in the assertion for john is actually unavoidable as it can be shown that there is no ALC-ABox that is equivalent to the ALCO-ABox A . As a consequence, the update of the ALC-ABox A with U cannot be expressed in ALC.. We say that a description logic L does not have updates if there are an L-ABox A and update U such that the result of updating A with U cannot be expressed in L. The first main aim of this paper is to understand how the problem of non-expressibility of updated ABoxes can be overcome. In particular, we consider two options: (i) increasing the expressive power of DLs by adding additional constructors and (ii) relaxing the definition of updated ABoxes.. Regarding (i), we show that the addition of nominals (as in the example above) and the '@' constructor from hybrid logic suffices to ensure the existence of updated ABoxes in all DLs between ALC and ALCQIO. Intuitively, the '@' constructor enables 'jumps' between individuals by allowing the formation of concepts such as @ a C which is satisfied at any point of an interpretation whenever the individual a satisfies the concept C. We also show that the '@'-constructor (but not nominals) can be replaced by Boolean ABoxes, i.e., ABoxes that admit Boolean operators to be applied to ABox assertions.. Regarding (ii), we consider the following definitions of updated ABoxes. An ABox A is. • a semantic update of A with U if the models of A are precisely those interpretations that can be obtained from models of A by making the assertions in U true (the standard definition);. • an approximate update of A with U regarding a DL L if A entails exactly the same L-ABox assertions as the semantic update of A with U ;. • a projective update of A with U if the models of A are precisely the models of the semantic update, after projecting both to the symbols in A and U ;. • a projective approximate update of A with U if A entails exactly the same L-ABox assertions ϕ as the semantic update of A with U as long as ϕ uses only symbols from A and U . Observe that projective updates allow the use of fresh, auxiliary symbols in A , and so do projective approximate updates. Also note that (projective and nonprojective) approximate updates have an additional parameter, which is the DL L in which entailed ABox assertions are formulated. It is not hard to see that every semantic update is also an approximate update and a projective update, which in turn are also projective approximate updates. Moreover, semantic updates and approximate updates can be proved to be logically equivalent, if the former exists. Due to the new symbols, this is in general not true for projective updates. Similar forms of updates have been considered e.g. in [9, 11] , see Section 7 for more details.. Unfortunately, it turns out that the more relaxed definitions of updated ABoxes only rarely help to overcome the problem of DLs not having updates. Actually, the 'real' definitions of projective updates and projective approximate updates are slightly stronger and use different (but more complicated to describe) sets of symbols. We refer to Definition 4 for full details.. More precisely, all DLs considered in this paper have approximate updates if, and only if, they have semantic updates. Projective updates are slightly more well-behaved: using a simple trick, one can see that a DL L has projective updates if the extension of L with the '@' constructor has semantic updates. Thus, we can ensure the existence of projective updates in the DLs between ALC and ALCQIO by adding only nominals (but not the '@' constructor). Further relaxing projective updates to projective approximate updates turns out to not improve this situation.. The second main question studied in this paper concerns the size of updated ABoxes. The first relevant observation is that our construction of semantic updates in DLs that comprise nominals and the '@' constructor incurs an exponential blowup in the size of the update U and in the role depth of the original ABox A, i.e., the nesting depth of existential and universal restrictions in A. Although both measures are typically small in real applications, this raises the questions (i) whether an exponential blowup of semantic updates can be avoided by a more careful construction and (ii) whether other forms of update help to avoid an exponential blowup. Concerning (i), we show that an exponential blowup cannot be avoided unless NP ∩ co-NP is contained in the non-uniform version of the complexity class NC 1 , which is considered very unlikely in complexity theory. Note that similar results such as those obtained by Cadoli et al. in [9] are not applicable due to the restricted form of updates considered in this paper (however, our results strengthen some of the results obtained by Cadoli et al.) Giving a positive answer to (ii), we then show that switching from semantic updates to projective ones dramatically improves the situation: in DLs that comprise nominals, not only the existence of projective updates is guaranteed, but it is even possible to construct projective updates whose size is polynomial both in the size of the original ABox and the update. Thus, projective updates are particularly well-suited for use in practical applications. We note that similar observations have already been made for the case of propositional logic in [9] , where it is shown that for some update semantics, the projective version of updates is more succinct than the non-projective one.. Finally, we extend our update constructions to conditional updates, which can express statements such as 'A(a) is true after the update if C(b) was true before'. We then apply the extended results to reasoning about actions in a DL context, as recently proposed and studied in [12, 13, 14] . In particular, we show that our construction of updated ABoxes can be used to implement a progression approach to the central projection problem [7] , reproving a number of tight upper complexity bounds for this problem that have originally been obtained using the method of regression. This paper is organized as follows. In Section 2, we provide a brief introduction to description logics, define the various kinds of ABox updates studied in this paper, and present some basic results regarding these updates. In particular, we interrelate the various definitions of updated ABoxes and prove that the '@' constructor is intimately related to projective updates, and to Boolean ABoxes.. The non-existence of updated ABoxes in standard expressive DLs of the ALC family is established in Section 3. We prove that DLs between ALC and ALCQI @ do not have approximate projective updates, and that DLs between ALCO and ALCQIO do not have approximate updates. This section is actually the only one where we explicitly consider (projective or non-projective) approximate updates since the positive results in subsequent sections all hold for stronger, non-approximating definitions of updated ABoxes.. In Section 4, we show that adding nominals and the '@' constructor to the DLs ALC, ALCI, ALCQ, and ALCQI suffices to have semantic updates. We also establish the single-exponential size of updated ABoxes announced before, both for the case of single updates and iterated updates. We prove that an exponential blowup of updated ABoxes cannot be avoided, subject to the complexity-theoretic assumption that NP ∩ co-NP ⊆ NC 1 . Finally, we consider an extension of ALCQIO @ with certain role constructors which allows a simple construction of updated ABoxes that are exponential only in the size of the update, but not in the size of the original ABox.. We then focus on projective updates in Section 5, showing that they are enjoyed by all DLs between ALCO and ALCQIO @ , including those that do not comprise the '@' constructor. We also show that projective updates can be constructed in polynomial time such that the resulting updated ABox is of polynomial size. With a small trick, these time and space bounds also apply to the case of iterated updates.. Section 6 is devoted to conditional updates and their application to reasoning about actions using DLs. Finally, Section 7 wraps up the paper, analyzing some possible extensions of our results (e.g. with TBoxes) and discussing related work. This paper is a significantly extended and revised version of [15] . We have studied updates of description logic ABoxes in the restricted, yet fundamental case where no compound concepts are admitted in the ABox and no TBoxes are present. Our results show that, while many description logics do that for DLs between ALCO and ALCQIO @ , the update operator considered in this paper is query compactable. not have updates, by choosing the right DL (one that includes nominals), it is possible to guarantee the existence of updates. Moreover, by choosing the right notion of update (projective semantic), it is even possible to compute updated ABoxes in polynomial time. We have also described an application of our results in reasoning about action.. Regarding future work, it would be interesting to study less restricted cases where the update U is allowed to contain compound concepts and TBoxes are admitted. Note, though, that this involves some rather serious challenges that we have identified and discussed in Section 6. It would also be interesting to consider ABox revision instead of ABox update, for which a number of competing semantics are available; see [8, 9] and references therein. We believe that the results and techniques established in this paper would also be useful to deal with those semantics.

O O O O B O O B I O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O B O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O B O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Effective symbolic dynamics, random points, statistical behavior, complexity and entropy Abstract. We consider the dynamical behavior of Martin-Löf random points in dynamical systems over metric spaces with a computable dynamics and a computable invariant measure. We use computable partitions to define a sort of effective symbolic model for the dynamics. Through this construction we prove that such points have typical statistical behavior (the behavior which is typical in the Birkhoff ergodic theorem) and are recurrent. We introduce and compare some notions of complexity for orbits in dynamical systems and prove: (i) that the complexity of the orbits of random points equals the KolmogorovSinaï entropy of the system, (ii) that the supremum of the complexity of orbits equals the topological entropy.

B I I O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B


A spatio-temporal approach to collaborative filtering In this paper, we propose a novel spatio-temporal model for collaborative filtering applications. Our model is based on low-rank matrix factorization that uses a spatio-temporal filtering approach to estimate user and item factors. The spatial component regularizes the factors by exploiting correlation across users and/or items, modeled as a function of some implicit feedback (e.g., who rated what) and/or some side information (e.g., user demographics, browsing history). In particular, we incorporate correlation in factors through a Markov random field prior in a probabilistic framework, whereby the neighborhood weights are functions of user and item covariates. The temporal component ensures that the user/item factors adapt to process changes that occur through time and is implemented in a state space framework with fast estimation through Kalman filtering. Our spatio-temporal filtering (ST-KF hereafter) approach provides a single joint model to simultaneously incorporate both spatial and temporal structure in ratings and therefore provides an accurate method to predict future ratings. To ensure scalability of ST-KF, we employ a mean-field approximation for inference. Incorporating user/item covariates in estimating neighborhood weights also helps in dealing with both cold-start and warm-start problems seamlessly in a single unified modeling framework; covariates predict factors for new users and items through the neighborhood. We illustrate our method on simulated data, benchmark data and data obtained from a relatively new recommender system application arising in the context of Yahoo! Front Page. Matrix factorization (MF) is an effective prediction technique that has been successfully applied to several collaborative filtering applications [17, 13, 18, 19] . However, most existing MF-based collaborative filtering algorithms do not consider the following facts:. • Real-world recommender systems are often dynamic in nature and require adaptive estimation of user and item characteristics for accurate predictions. In fact, for several well studied movie recommender problems, user factors are non-stationary over time and require an adaptive estimation procedure that emphasizes recent user ratings more than his/her past behavior. In several other applications, item characteristics, such as the popularity of a news article, may also be dynamic and change rapidly in a rather short time period. In such scenarios, "static" matrix factorization is sub-optimal. In fact, most collaborative filtering algorithms have been designed and evaluated based on random split of training and test sets without considering the temporal structure.. • In addition to the actual ratings, we often have extra information about users or items, such as implicit feedback (e.g., who rated what) and/or some side information (e.g., user demographics, browsing history). Incorporating side information becomes especially useful when available ratings for users and items are scarce.. In this paper, we provide a new method to address the above-mentioned problems through a model that simultaneously incorporates the spatial and temporal structure in rating history. We show that this joint model, referred to as spatio-temporal Kalman filtering (ST-KF), improves prediction accuracy over standard matrix factorization on synthetic data, Movielens data and the Yahoo! Front Page data. Our work enhances the basic factor model for collaborative filtering that models the rating of item i by user u as. where p (u) and q (i) are k-dimensional latent user and item factors respectively, and ǫ (ui) denotes observation error. Estimation of these factors is obtained through some regularization on factors to prevent over-fitting; it is customary to constrain the factors by regulating the L2 norm of factors. The main contribution of our work is to enhance the regularization through a spatio-temporal filtering approach to obtain better estimates of user and item factors. The spatial component, discussed in Section 2, regularizes the factors by exploiting correlations across users and/or items. The temporal component, introduced in Section 3, ensures that user/item factors adapt to process changes that occur through time. In particular, temporal change in factors is modeled in a state space framework with fast estimation through Kalman filtering. Our spatio-temporal filtering approach (named ST-KF, see Sections 4-5) provides a single joint model to simultaneously incorporate both spatial and temporal structure in ratings using a "product of experts" approach that results in an accurate method to predict future ratings. Although classical Kalman filtering is a fast estimation method for linear state-space models, simultaneous estimation of both spatial and temporal components induces a model that is computationally intensive; we ensure scalable inference for ST-KF through mean-field approximation. Our model also handles both cold-start and warm-start problems seamlessly in a single unified modeling framework by incorporating user/item covariates in estimating neighborhood weights; covariates predict factors for new users and new items through the neighborhood. We presented an efficient spatio-temporal approach to collaborative filtering, and showed its efficacy on both synthetic and real-world data sets. Our future work will focus on better models for joint and dynamical estimation of user factors and item factors, including bilinear filtering model based on Sigma-point Kalman filter, and parameterized regression models as in [10] . We tested the proposed algorithms on synthetic data, the MovieLens data, and Yahoo! Front Page data. On synthetic data and MovieLens, the task is to predict the missing entries based on the (sparsely) observed ones. The prediction performance is measured as the root of mean square error on test set,. where test stands for the set for all the entries in test set. For Yahoo! Front Page data, the error is measured by the more relevant ROC curve for "click-or-not" prediction.

O O O O B I O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Operator Splittings, Bregman Methods and Frame Shrinkage in Image Processing We examine the underlying structure of popular algorithms for variational methods used in image processing. We focus here on operator splittings and Bregman methods based on a unified approach via fixed point iterations and averaged operators. In particular, the recently proposed alternating split Bregman method can be interpreted from different points of view-as a Bregman, as an augmented Lagrangian and as a Douglas-Rachford splitting algorithm which is a classical operator splitting method. We also study similarities between this method and the forward-backward splitting method when applied to two frequently used models for image denoising which employ a Besov-norm and a total variation regularization term, respectively. In the first setting, we show that for a discretization based on Parseval frames the gradient descent reprojection and the alternating split Bregman algorithm are equivalent and turn out to be a frame shrinkage method. For the total variation regularizer, we also present a numerical comparison with multistep methods. In this paper, we described main ideas behind some important minimization techniques for image restoration, in particular split Bregman methods. It had previously been observed that the split Bregman method can be interpreted as a classical proximal point method or, equivalently, as an augmented Lagrangian method. Since there is still a hard subproblem to solve in each iteration of the split Bregman algorithm, the alternating split Bregman method had been proposed. It turned out that this algorithm is equivalent to other well-known methods, namely, the Douglas-Rachford splitting and the alternating direction method of minimizers. We also studied the convergence properties in more detail, especially for the primal variable.. As an application, image denoising via two popular models which apply B 1 1,1 and total variation regularizers, respectively, was considered. In the B 1 1,1 case and using a special setting based on Parseval frames, the alternating split Bregman and gradient descent reprojection methods led to the same algorithm which is essentially iterated frame shrinkage. This underlines the common roots of these techniques. A certain similarity was also established for the total variation regularization functional. Finally, we have compared these methods numerically with multistep algorithms for a forward difference discretization of the total variation regularizer. We found that if the step length parameter is optimized the alternating split Bregman method performs well, especially with respect to the primal function value. This can be motivated by relating it to a special Newton method.. The minimization algorithms we considered here can be applied to other problems in image processing, e.g., to the minimization of nonlocal total variation functionals, cf., Gilboa et al. (2006) , Gilboa and Osher (2008) , Kindermann et al. (2005) . See also Buades et al. (2008) for more on the idea of using nonlocal operators in image processing. Moreover, used the alternating split Bregman method for segmentation and surface reconstruction. As pointed out in Esser (2009) , the alternating split Bregman method is especially useful for more complicated minimization problems, e.g., for image deblurring in the presence of noise. For Gaussian and impulse noise this was discussed in Osher (2009), Esser (2009) . The application to deblurring in the presence of Poisson noise and multiplicative noise can be found in Figueiredo and Bioucas-Dias (2009 ), Setzer et al. (2010 . An alternating split Bregman algorithm for image denoising in the presence of multiplicative noise using a nonlocal total variation term was recently proposed by Steidl and Teuber (2010) .

O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O B I O O O O O O O O O O O O O O O O O O O O O