Eulerian video magnification for revealing subtle changes in the world and output (bottom) videos plotted over time shows how our method amplifies the periodic color variation. In the input sequence the signal is imperceptible, but in the magnified sequence the variation is clear. The complete sequence is available in the supplemental video.Our goal is to reveal temporal variations in videos that are difficult or impossible to see with the naked eye and display them in an indicative manner. Our method, which we call Eulerian Video Magnification, takes a standard video sequence as input, and applies spatial decomposition, followed by temporal filtering to the frames. The resulting signal is then amplified to reveal hidden information. Using our method, we are able to visualize the flow of blood as it fills the face and also to amplify and reveal small motions. Our technique can run in real time to show phenomena occurring at the temporal frequencies selected by the user. The human visual system has limited spatio-temporal sensitivity, but many signals that fall below this capacity can be informative.. For example, human skin color varies slightly with blood circulation. This variation, while invisible to the naked eye, can be exploited to extract pulse rate [Verkruysse et al. 2008; Poh et al. 2010; Philips 2011] . Similarly, motion with low spatial amplitude, while hard or impossible for humans to see, can be magnified to reveal interesting mechanical behavior [Liu et al. 2005] . The success of these tools motivates the development of new techniques to reveal invisible signals in videos. In this paper, we show that a combination of spatial and temporal processing of videos can amplify subtle variations that reveal important aspects of the world around us.. Our basic approach is to consider the time series of color values at any spatial location (pixel) and amplify variation in a given temporal frequency band of interest. For example, in Figure 1 we automatically select, and then amplify, a band of temporal frequencies that includes plausible human heart rates. The amplification reveals the variation of redness as blood flows through the face. For this application, temporal filtering needs to be applied to lower spatial frequencies (spatial pooling) to allow such a subtle input signal to rise above the camera sensor and quantization noise.. Our temporal filtering approach not only amplifies color variation, but can also reveal low-amplitude motion. For example, in the supplemental video, we show that we can enhance the subtle motions around the chest of a breathing baby. We provide a mathematical analysis that explains how temporal filtering interplays with spatial motion in videos. Our analysis relies on a linear approximation related to the brightness constancy assumption used in optical flow formulations. We also derive the conditions under which this approximation holds. This leads to a multiscale approach to magnify motion without feature tracking or motion estimation.. Previous attempts have been made to unveil imperceptible motions in videos. [Liu et al. 2005] analyze and amplify subtle motions and visualize deformations that would otherwise be invisible. [Wang et al. 2006] propose using the Cartoon Animation Filter to create perceptually appealing motion exaggeration. These approaches follow a Lagrangian perspective, in reference to fluid dynamics where the trajectory of particles is tracked over time. As such, they rely Overview of the Eulerian video magnification framework. The system first decomposes the input video sequence into different spatial frequency bands, and applies the same temporal filter to all bands. The filtered spatial bands are then amplified by a given factor α, added back to the original signal, and collapsed to generate the output video. The choice of temporal filter and amplification factors can be tuned to support different applications. For example, we use the system to reveal unseen motions of a Digital SLR camera, caused by the flipping mirror during a photo burst (camera; full sequences are available in the supplemental video).. on accurate motion estimation, which is computationally expensive and difficult to make artifact-free, especially at regions of occlusion boundaries and complicated motions. Moreover, Liu et al. [2005] have shown that additional techniques, including motion segmentation and image in-painting, are required to produce good quality synthesis. This increases the complexity of the algorithm further.. In contrast, we are inspired by the Eulerian perspective, where properties of a voxel of fluid, such as pressure and velocity, evolve over time. In our case, we study and amplify the variation of pixel values over time, in a spatially-multiscale manner. In our Eulerian approach to motion magnification, we do not explicitly estimate motion, but rather exaggerate motion by amplifying temporal color changes at fixed positions. We rely on the same differential approximations that form the basis of optical flow algorithms [Lucas and Kanade 1981; Horn and Schunck 1981] .. Temporal processing has been used previously to extract invisible signals [Poh et al. 2010] and to smooth motions [Fuchs et al. 2010] . For example, Poh et al. [2010] extract a heart rate from a video of a face based on the temporal variation of the skin color, which is normally invisible to the human eye. They focus on extracting a single number, whereas we use localized spatial pooling and bandpass filtering to extract and reveal visually the signal corresponding to the pulse. This primal domain analysis allows us to amplify and visualize the pulse signal at each location on the face. This has important potential monitoring and diagnostic applications to medicine, where, for example, the asymmetry in facial blood flow can be a symptom of arterial problems. Fuchs et al. [2010] use per-pixel temporal filters to dampen temporal aliasing of motion in videos. They also discuss the high-pass filtering of motion, but mostly for non-photorealistic effects and for large motions (Figure 11 in their paper). In contrast, our method strives to make imperceptible motions visible using a multiscale approach. We analyze our method theoretically and show that it applies only for small motions.. In this paper, we make several contributions. First, we demonstrate that nearly invisible changes in a dynamic environment can be revealed through Eulerian spatio-temporal processing of standard monocular video sequences. Moreover, for a range of amplification values that is suitable for various applications, explicit motion estimation is not required to amplify motion in natural videos. Our approach is robust and runs in real time. Second, we provide an analysis of the link between temporal filtering and spatial motion and show that our method is best suited to small displacements and lower spatial frequencies. Third, we present a single framework that can be used to amplify both spatial motion and purely temporal changes, e.g., the heart pulse, and can be adjusted to amplify particular temporal frequencies-a feature which is not supported by Lagrangian methods. Finally, we analytically and empirically compare Eulerian and Lagrangian motion magnification approaches under different noisy conditions. To demonstrate our approach, we present several examples where our method makes subtle variations in a scene visible. We described a straightforward method that takes a video as input and exaggerates subtle color changes and imperceptible motions. To amplify motion, our method does not perform feature tracking or optical flow computation, but merely magnifies temporal color changes using spatio-temporal processing. This Eulerianbased method, which temporally processes pixels in a fixed spatial region, successfully reveals informative signals and amplifies small motions in real-world videos. Sensitivity to Noise. The amplitude variation of the signal of interest is often much smaller than the noise inherent in the video. In such cases direct enhancement of the pixel values will not reveal the desired signal. Spatial filtering can be used to enhance these subtle signals. However, if the spatial filter applied is not large enough, the signal of interest will not be revealed (Figure 11 ).. Assuming that the noise is zero-mean white and wide-sense stationary with respect to space, it can be shown that spatial low pass filtering reduces the variance of the noise according to the area of the low pass filter. In order to boost the power of a specific signal, e.g., the pulse signal in the face, we can use the spatial characteristics of the signal to estimate the spatial filter size.. Let the noise power level be σ 2 , and our prior on signal power over spatial frequencies be S(λ). We want to find a spatial low pass filter with radius r such that the signal power is greater than the noise in the filtered frequency region. The wavelength cut off of such a filter is proportional to its radius, r, so the signal prior can be represented as S(r). The noise power σ 2 can be estimated by examining pixel values in a stable region of the scene, from a gray card, or by using a technique as in [Liu et al. 2006] . Since the filtered noise power level, σ 2 , is inversely proportional to r 2 , we can solve the following equation for r,. where k is a constant that depends on the shape of the low pass filter. This equation gives an estimate for the size of the spatial filter needed to reveal the signal at a certain noise power level.. Eulerian vs. Lagrangian Processing. Because the two methods take different approaches to motion-Lagrangian approaches explicitly track motions, while our Eulerian approach does notthey can be used for complementary motion domains. Lagrangian approaches, e.g. [Liu et al. 2005] , work better to enhance motions of fine point features and support larger amplification factors, while our Eulerian method is better suited to smoother structures and small amplifications. We note that our technique does not assume particular types of motions. The first-order Taylor series analysis can hold for general small 2D motions along general paths.. In Appendix A, we further derive estimates of the accuracy of the two approaches with respect to noise. Comparing the Lagrangian error, εL (Eq. 29), and the Eulerian error, εE (Eq. 31), we see that both methods are equally sensitive to the temporal characteristics of the noise, nt, while the Lagrangian process has additional error terms proportional to the spatial characteristics of the noise, nx, due to the explicit estimation of motion (Eq. 27). The Eulerian error, on the other hand, grows quadratically with α, and is more sensitive to high spatial frequencies (Ixx). In general, this means that Eulerian magnification would be preferable over Lagrangian magnification for small amplifications and larger noise levels.. We validated this analysis on a synthetic sequence of a 2D cosine oscillating at 2 Hz temporally and 0.1 pixels spatially with additive white spatiotemporal Gaussian noise of zero mean and standard deviation σ (Figure 12 ). The results match the relationship of error-to-noise and error-to-amplification predicted by the derivation (Figure 12(b) ), as well as the expected region where the Eulerian approach outpeforms the Lagrangian results (Figure12(a)-left) . The Lagrangian method is indeed more sensitive to increases in spatial noise, while the Eulerian error is hardly affected by it (Figure 12(c) ). While different regularization schemes used for motion estimation (that are harder to analyze theoretically) may alleviate the Lagrangian error, they did not change the result significantly (Figure 12(a)-right) . In general, our experiments show that for small amplifications the Eulerian approach strikes a better balance between performance and efficiency. Comparisons between the methods on natural videos are available on the project webpage. The results were generated using non-optimized MATLAB code on a machine with a six-core processor and 32 GB RAM. The computation time per video was on the order of a few minutes. We used a separable binomial filter of size five to construct the video pyramids. We also built a prototype application that allows users to reveal subtle changes in real-time from live video feeds, essentially serving as a microscope for temporal variations. It is implemented in C++, is entirely CPU-based, and processes 640 × 480 videos at 45 frames per second on a standard laptop. It can be sped up further by utilizing GPUs. A demo of the application is available in the accompanying video. The code is available on the project webpage.. Given an input video to process by Eulerian video magnification, there are four steps the user needs to take: (1) select a temporal bandpass filter; (2) select an amplification factor, α; (3) select a spatial frequency cutoff (specified by spatial wavelength, λc) beyond which an attenuated version of α is used; and (4) select the form of the attenuation for α-either force α to zero for all λ < λc, or linearly scale α down to zero. The frequency band of interest can be chosen automatically in some cases, but it is often important for users to be able to control the frequency band corresponding to their application. In our real-time application, the amplification factor and cutoff frequencies are all customizable by the user. : Eulerian video magnification used to amplify subtle motions of blood vessels arising from blood flow. For this video we tuned the temporal filter to a frequency band that includes the heart rate-0.88 Hz (53 bpm)-and set the amplification factor to α = 10. To reduce motion magnification of irrelevant objects, we applied a user-given mask to amplify the area near the wrist only. Movement of the radial and ulnar arteries can barely be seen in the input video (a) taken with a standard point-and-shoot camera, but is significantly more noticeable in the motion-magnified output (b). The motion of the pulsing arteries is more visible when observing a spatio-temporal Y T slice of the wrist (a) and (b). The full wrist sequence can be found in the supplemental video. We first select the temporal bandpass filter to pull out the motions or the signal desired to be amplified (step 1 above). The choice of filter is generally application dependent. For motion magnification, a filter with a broad passband is preferred; for color amplification of blood flow, a narrow passband produces a more noise-free result. Figure 9 shows the frequency responses of some of the temporal filters used in this paper. We use ideal bandpass filters for color amplification, since they have passbands with sharp cutoff frequencies. Low-order IIR filters can be useful for both color amplification and motion magnification and are convenient for a real-time implementation. In general, we used two first-order lowpass IIR filters with cutoff frequencies ω l and ω h to construct an IIR bandpass filter.. Next, we select the desired magnification value, α, and spatial frequency cutoff, λc (steps 2 and 3). While (14) can be used as a guide, in practice, we may try various α and λc values to achieve a desired result. Users can select a higher α that violates the bound to exaggerate specific motions or color changes at the cost of increasing noise or introducing more artifacts. In some cases, one can account for color clipping artifacts by attenuating the chrominance components of each frame. Our approach achieves this by doing all the processing in the YIQ space. Users can attenuate the chrominance components, I and Q, before conversion to the original color space.. For human pulse color amplification, where we seek to emphasize low spatial frequency changes, we may force α = 0 for spatial wavelengths above λc. For motion magnification videos, we can choose to use a linear ramp transition for α (step 4).. We evaluated our method for color amplification using a few Figure 9 : Temporal filters used in the paper. The ideal filters (a) and (b) are implemented using DCT. The Butterworth filter (c) is used to convert a user-specified frequency band to a second-order IIR structure and is used in our real-time application. The secondorder IIR filter (d) also allows user input. These second-order filters have a broader passband than an ideal filter.. videos: two videos of adults with different skin colors and one of a newborn baby. An adult subject with lighter complexion is shown in face (Figure 1) , while an individual with darker complexion is shown in face2 (Figure 8) . In both videos, our objective was to amplify the color change as the blood flows through the face. In both face and face2, we applied a Laplacian pyramid and set α for the finest two levels to 0. Essentially, we downsampled and applied a spatial lowpass filter to each frame to reduce both quantization and noise and to boost the subtle pulse signal that we are interested in. For each video, we then passed each sequence of frames through an ideal bandpass filter with a passband of 0.83 Hz to 1 Hz (50 bpm to 60 bpm). Finally, a large value of α ≈ 100 and λc ≈ 1000 was applied to the resulting spatially lowpass signal to emphasize the color change as much as possible. The final video was formed by adding this signal back to the original. We see periodic green to red variations at the heart rate and how blood perfuses the face.. baby2 is a video of a newborn recorded in situ at the Nursery Department at Winchester Hospital in Massachusetts. In addition to the video, we obtained ground truth vital signs from a hospitalgrade monitor. We used this information to confirm the accuracy of our heart rate estimate and to verify that the color amplification signal extracted from our method matches the photoplethysmogram, an optically obtained measurement of the perfusion of blood to the skin, as measured by the monitor. Figure 10: Selective motion amplification on a synthetic sequence (sim4 on left). The video sequence contains blobs oscillating at different temporal frequencies as shown on the input frame. We apply our method using an ideal temporal bandpass filter of 1-3 Hz to amplify only the motions occurring within the specified passband.. In (b), we show the spatio-temporal slices from the resulting video which show the different temporal frequencies and the amplified motion of the blob oscillating at 2 Hz. We note that the space-time processing is applied uniformly to all the pixels. The full sequence and result can be found in the supplemental video.. To evaluate our method for motion magnification, we used several different videos: face (Figure 1 ), sim4 (Figure 10 ), wrist (Figure 7 ), camera (Figure 2) , face2, guitar, baby, subway, shadow, and baby2 ( Figure 8 ). For all videos, we used standard Laplacian pyramid for spatial filtering. For videos where we wanted to emphasize motions at specific temporal frequencies (e.g., in sim4 and guitar), we used ideal bandpass filters. In sim4 and guitar, we were able to selectively amplify the motion of a specific blob or guitar string by using a bandpass filter tuned to the oscillation frequency of the object of interest. These effects can be observed in the supplemental video. The values used for α and λc are shown in Table 1 .. For videos where we were interested in revealing broad, but subtle motion, we used temporal filters with a broader passband. For example, for the face2 video, we used a second-order IIR filter with slow roll-off regions. By changing the temporal filter, we can magnify the motion of the head instead of amplifying skin color change. Accordingly, α = 20, λc = 80 were chosen to magnify the motion.. By using broadband temporal filters and setting α and λc according to the bound, our method is able to reveal invisible motions, as in the camera and wrist videos. For the camera video, we used a highspeed camera with a sampling rate of 300 Hz to record a Digital SLR camera capturing photos at about 1 exposure per second. The vibration, caused by the moving mirror in the SLR, though invisible to the naked eye, is revealed by our Eulerian video magnification approach. To verify that the vibrations amplified are indeed caused by the flipping mirror in the SLR, we secured a laser pointer to the camera and recorded a video of the laser light, appearing at a distance of 3 to 4 meters from the source. At that distance, the laser light visibly oscillates with each exposure, and the oscillations were in sync with the magnified motions.. Our method is also able to exaggerate visible, yet subtle motion, as seen in the baby, face2, and subway videos. In the subway example we deliberately amplified the motion beyond the derived bounds of where the 1st order approximation holds in order to increase the effect and to demonstrate the algorithm's artifacts. We note that most of the examples in our paper contain oscillatory movements because such motion generally has longer duration and smaller amplitudes. However, our method can be used to amplify non-periodic motions as well, as long as they are within the passband of the temporal bandpass filter. In shadow, for example, we process a video of the sun's shadow moving linearly yet imperceptibly over 15 seconds. The magnified version makes it possible to see the change even within this short time period. Finally, some videos may contain regions of temporal signals that do not need amplification, or that, when amplified, are perceptually unappealing. Due to our Eulerian processing, we can easly allow the user to manually restrict magnification to particular areas by marking them on the video (this was used for face and wrist).

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Automating HAZOP studies using D-higraphs In this paper, we present the use of D-higraphs to perform HAZOP studies. D-higraphs is a formalism that includes in a single model the functional as well as the structural (ontological) components of any given system. A tool to perform a semi-automatic guided HAZOP study on a process plant is presented. The diagnostic system uses an expert system to predict the behavior modeled using D-higraphs. This work is applied to the study of an industrial case and its results are compared with other similar approaches proposed in previous studies. The analysis shows that the proposed methodology fits its purpose enabling causal reasoning that explains causes and consequences derived from deviations, it also fills some of the gaps and drawbacks existing in previous reported HAZOP assistant tools. Throughout the history of the Process Industry there have been a lot of accidents of different consideration, from mild to catastrophic. The gravity of an accident can be analysed in terms of economical, human or environmental losses. Kletz (1999b Kletz ( , 2001 has performed a deep review of several accidents, analysing the causes and proposing modifications that could have avoided them. From the information gathered of the accidents new safety procedures, process modifications and working methods have been implemented, however, prevention should have come first.. "It is better to see the hazards afterwards than not see them at all, as we may pass the same way again, but it is better still to see them when they still lie ahead". (Kletz, 2001) In economic terms, operating a process plant in a safe way-lack of accidents-reduces the losses associated to shut-downs, reparations, compensations and fines. But, at the same time, the incomes increase because productivity does. As a consequence of costs decrease and incomes increase, profits grow. Of course, there are other aspects, such as environmental impact, pollution or operators occupational health that also benefit from a safe operation.. Apart from economical issues, tighter regulations, public concern on industrial incidents and a greater presence of chemical industry accidents in the media are making accident prevention a major task in process industry. It involves the analysis of the process itself, the control system, the additional safety systems, the backup systems, the operation procedures and so on. They are analysed during the design stage and during the operation of the plant. In these papers we will focus in the former, the design stage.. Process Hazard Analyses (PHA) are carried out to identify the potential safety problems but also to propose possible solutions such as process changes, new control strategies or the use of safety instrumentation. To perform PHA there is a wide range of techniques: What-If, Checklist, Hazard and Operability (HAZOP), Failure Modes and Effects Analysis and Fault Tree Analysis. However, HAZOP studies gather the two following features (Zhao, Bhushan, & Venkatasubramanian, 2005a) : (1) it is a tool easy to learn and use and (2) it is reusable and adaptable for almost all industrial processes.. Hence, HAZOP is widely accepted as the method for conducting the PHA analysis. It was first developed in the late 1960s at Imperial Chemical Industries (ICI), nowadays taken over by AkzoNobel. HAZOP studies provide a systematic methodology to identify and reduce the potential risks or process hazards (Swann & Preston, 1995) . This tool will be discussed further in the present article.. Although HAZOP studies are easy to learn, reusable and systematic, it is a procedures that consumes a lot of time and effort which can be translated to money. During the last two decades, there have been developed different tools that automate the HAZOP studies, some of them are:. • HAZOPExpert, a model-based intelligent system for HAZOP of continuous processes (Venkatasubramanian, Zhao, & Viswanathan, 2000) .. • PHASuite, an automated HAZOP analysis tool for chemical processes (Zhao, Bhushan, & Venkatasubramanian, 2005b; Zhao et al., 2005a ).. • Functional HAZOP assistant, a functional modeling based methodology (Rossing, Lind, Jensen, &j0rgensen, 2010a ).. • Layered Digraph Model (Cui, Zhao, Qju, & Chen, 2008) • PetroHAZOP (Zhao, Cui, Zhao, Qju, & Chen, 2009) The main objective of this paper is to present the use of Dhigraphs to perform HAZOP studies. The key idea of a D-higraph is to capture the functional as well as the structural aspects of process plants. In other words, the aim of a D-higraph model is to gather activity and ontological features of the system modeled in an integrated model (Rodriguez & Sanz, 2009) .. D-higraphs, as a functional modeling technique, can be used in the same way than Multilevel Flow Modeling (Lind, 1994 (Lind, , 2005 is used to develop the 'Functional HAZOP assistant' proposed by Rossing et al. (2010a Rossing et al. ( , 2010b . D-higraphs provide almost the same functional knowledge than MFM but as they integrate structural information, the correlation with the real system is much more direct and easier. However, using the extensions presented in Lind (2010) , MFM also integrates structural information. At this point it should be noticed that a D-higraph is not a hypergraph, thus abstraction is not facilitated, in contrast to MFM.. The proposed methodology is compared with the above mentioned tools and also with conventional HAZOPs. To that end, we have developed the D-higraph of the Indirect Vapor Recompression Distillation pilot Plant (IVaRDiP) at the Department of Chemical and Biochemical Engineering at Technical University of Denmark (DanmarksTekniske Universitet, DTU) and we have also performed a HAZOP analysis using the D-higraphs framework. Rossing et al. (2010a Rossing et al. ( , 2010b used this case study, so a direct comparison can be made.. The paper is structured as follows. Next section introduces an overview of HAZOP studies. The following section briefly describes the D-higraphs technique. The D-higraphs HAZOP assistant is presented in the succeeding section. Then, the methodology is applied to the IVaRDiP system. Next, the methodology is compared to other tools. Finally, the 'assistant' is analysed, the conclusions are drawn and further work is proposed.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Interval Forecasting of Electricity Demand: A Novel Bivariate EMD-based Support Vector Regression Modeling Framework  Proposing a novel interval-valued electricity demand forecasting approach. BEMD and SVR are integrated for interval forecasting of electricity demand. The EMD-based modeling framework are extended to deal with interval forecasting  BEMD is used to decompose both the lower and upper bounds electricity demand series. The proposed modeling framework is justified with real world data sets. Interval Forecasting of ElectricityHighly accurate interval forecasting of electricity demand is fundamental to the success of reducing the risk when making power system planning and operational decisions by providing a range rather than point estimation. In this study, a novel modeling framework integrating bivariate empirical mode decomposition ( To supply high quality electric energy to the customer in a secure and economic manner, an electricity operator faces many technical and economical problems in operation, planning, control and reliability management of an electric power system.. In achieving this goal, accurate forecast of electricity demand is the first prerequisite.. According to extent literature investigation, it is not hard to find that a wide variety of methodologies and techniques have been used for electricity demand forecasting with many degrees of success, such as exponential smoothing models [1] , regression models [2] , fuzzy logic approach [3] , fuzzy inference system [4] , grey-based approaches [5] , wavelet transforms and adaptive models [6] , kernel-based method [7] , artificial neural networks [8, 9] , support vector machines [10] , semi-parametric method [11] and hybrid model [12, 13] . The reader is referred to [14] for a recent survey of the presented methodologies and techniques employed for electricity demand forecasting. However, an important point to note from past studies mentioned above is their preoccupation with point forecasting rather than interval one.. An interval forecasting of electricity demand has the advantage of taking into account the variability and/or uncertainty so as to reduce the amount of random variation relative to that found in classic single-valued load time series. As. García-Ascanio and Maté [15] pointed out, the interval-valued time series forecasting (ITS) methods as a potential tool that will lead to a reduction in risk when making power system planning and operational decisions. To date, several suitable tools for managing ITS have been developed (see [16] for a recently review), such as interval 1 Holt's exponential smoothing methods [17] , interval multi-layer perceptrons (iMLP) [18] , vector autoregressive (VAR) model [15] , vector error correction (VEC) model [19] , etc. Most of the conventional methodologies available for ITS in the literature propose the use of computational methods or modeling schema, accounting for the capability of dealing with interval-valued data. For instance, neural networks are an area that has recently witnessed substantial improvements in interval analysis.. Notable earlier work on interval analysis using neural networks includes that of Simoff [20] , while Beheshti, Berrached [21] developed a multi-layer perceptron (MLP) in which inputs, weights, biases, and outputs are intervals, and proposed an algorithm so as to obtain the optimal weights and biases. Recently, Roque, Maté [18] proposed and analyzed a new model of MLP (named iMLP in [18] ) based on interval arithmetic that facilitates handling input and output interval-valued data, but where weights and biases are single-valued and not interval-valued.. Our study focuses on extending the EMD-based time series modeling framework to adapt to the scenario of interval forecasting of electricity demand. Following the philosophy of 'divide and conquer', EMD-based time series modeling framework has been recently well-established and well-justified in energy market [22] [23] [24] [25] , tourism management [26] , hydrology [27] , and transportation research [28] . 4 This produces an interval-valued time series where each observation is formed by an interval that collects, as its lower bound, the minimum value of the electricity demand and, as its upper bound, the maximum value of the electricity demand for a specific hour, month and year.. This paper is structured as follows. In Section 2, we provide brief introduction of BEMD and illustrate the data representation of interval-valued electricity demand time series analysis. Afterwards, the proposed BEMD-based SVR modeling framework are discussed in detail in Section 3. Section 4 details the research design on data process and preliminary analysis, accuracy measure, methodologies implementation, and experimental procedure. Following that, in Section 5, the experimental results are discussed. Section 6 finally concludes this work. Interval forecasting of electricity demand plays an increasingly important role in the electricity industry, as it could prove to be a potential tool for both power generators and consumers to make their plans. Following the philosophy of the 'divide and conquer', this study proposes a novel BEMD-based SVR modeling framework for monthly interval-valued electricity demand per hour forecasting in PJM Interconnection. The experimental study shows that the proposed modeling framework can improve prediction performance significantly and outperform statistically some well-established counterparts in terms of forecast accuracy measure and equality of accuracy of competing forecasts test. This indicates that the proposed BEMD-based SVR modeling framework is a promising tool for interval forecasting of electricity demand.. Besides electricity demand, the proposed BEMD-based SVR modeling framework might be used for other tough interval-valued time series forecasting task in energy market such as electricity price, which appeals further evidence.. Furthermore, this study restricts attention exclusively to one-step-ahead forecasting, multi-step-ahead forecasts are of greater value to decision-makers than one-step-ahead forecasts in energy market. We will look into these issues in the future research. This section demonstrates the usefulness of the models through statistical evaluations in experiments using interval-valued electricity demand series. The experiments are detailed below.. The aim of this evaluation is to highlight the advantages of using the proposed For each row of the Table 3 below, the entry with the smallest value is set in boldface and marked with an asterisk, and the entry with second smallest value is heighted in bold. Several observations can be drawn from Table 3. 19 <Insert Table 3 here>. Overall, the top three models across 24 hours turn out to be BEMD-SVR, then iMLP, and then VEC. It is clear that the proposed BEMD-SVR model is with position within top two. Indeed, the BEMD-SVR produces the most accuracy forecasts at 20. hours and obtains the second best forecasts at the remaining 4 hours. It is conceivable that the reason for the superiority of the proposed BEMD-SVR model is that decomposition strategy does effectively improve prediction performance of ITS.. As far as the comparison between the proposed BEMD-SVR and classical EMD-SVR, the BEMD-SVR is consistently the best performing model for ITS forecasting, this attests to the value which is added by decomposing simultaneously both the lower and upper bounds of interval-valued electricity demand series using BEMD technique.. Concerning the comparison between the interval artificial intelligence models (i.e., BEMD-SVR model and iMLP) and the traditional statistic models (i.e., Holt I and VEC), it is clear that the BEMD-SVR and iMLP outperform the Holt I and VEC, particularly in the period from 10 to 24 h. As it is known, the hourly electricity power demand is influenced by various factors, such as temperature, day of the week, etc, exhibiting strong nonlinearity. Fig. 5 shows an hourly average demand curve comparing the summer period vs. the winter period for 2011. Looking at Fig. 5 , it is clear that in the period from 10 to 24 h, the demand shows the highest variability, as it is behavior differs considerably from summer to winter due to the temperature effect.. In contrast, the demand curve from 1 to 9 h is relatively stable in both seasons due to 20 the majority of companies and factories are closed and most of people are asleep.. According to the variability of the demand in both seasons and the superiority of the BEMD-SVR and iMLP from 10 to 24 h, thus, we can conclude that the interval artificial intelligence methods are more efficient than the traditional statistic methods for forecasting the interval-valued time series with highly variability. According to the obtained results in Table 4 , one can deduce the following observations. First, the BEMD-SVR significantly outperforms the counterparts for the 22 overwhelming majority of hours. Second, the iMLP ranks the first at 2, 7, 11, and 15. h, but the superiority of iMLP at these hours is not significant. Third, the difference in prediction performance between iMLP and VEC is not significant (with the exception of the 4, 10, and 14 h). Fourth, the EMD-SVR performs significantly worse than the counterparts for the majority of hours. Fifth, the difference of prediction performance between two transformations (Eq. (1) and Eq. (2)) is not significant at 0.05 level.. Finally, concerning the traditional statistic models, the difference in prediction performance between VEC and Holt I is not significant at the 0.05 level (with the exception of the 12 and 21 h).. <Insert Table 4 here>

O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Video image mosaic implement based on planar-mirror-based catadioptric system Abstract Planar-mirror-based catadioptric method is one of the most hot topics in recent years. To overcome the disadvantages of the planar-mirror-based catadioptric panoramic camera, described by Nalwa (1996, 2001, 2000), such as the requirement for high-precision optical device designing and the stitching lines in the resulting images, we proposed a planar-mirror-based video image mosaic system with high precision for designing. Firstly, we designed a screw nut on our system, which can be adjusted to locate the viewpoints of the cameras' mirror images at a single point approximately. It provides a method for those who have difficulties in their designing and manufacturing for high precision. Then, after the image distortion correction and cylinder projection transforms, we can stitch the images to get a wide field of view image by template matching algorithm. Finally, nonlinear weighting fusion is adopted to eliminate the stitching line effectively. The experimental results show that our system has good characteristics such as video rate capture, high resolution, no stitching line and without affection by the depth of field of view. Wide field of view (FOV) and high-resolution image acquisition have become an important subject for research in visionbased applications. Due to its high practicability, it has been widely used in intelligent video surveillance systems, such as moving object detection [4] [5] [6] [7] and tracking [8] [9] [10] [11] . Techniques for constructing wide FOV image can be classified into two categories: dioptric methods and catadioptric methods [12] .. Dioptric methods mean that only refractive elements (such as lens) are employed. Fisheye lens [13] [14] [15] , rotating cameras [16, 17] , and camera clusters [18] [19] [20] [21] [22] are most widely used in this kind of methods. Although real-time wide FOV images can be acquired by fisheye lens, the resolutions of the images are usually low due to the limitation of the single image sensor in the fisheye lens. Also, the distortion of near lens scene caused by fisheye lens cannot be resolved. Rotating cameras can achieve high resolution of the FOV images. However, as the single camera can only get a limited visual angle of the scene, it is impossible to acquire real-time wide FOV images. Camera clusters that capture images simultaneously can acquire wide FOV images at real-time video rate. But the optic center of the cameras cannot locate at a single point because of the cameras' space limitations, thus the parallax caused by these limitations always leads to ghost phenomenon in the mosaic image as affected by the depth of field of view.. Catadioptric methods mean a combination of refractive components and reflective components. There are two specific methods: the curved mirror method and the planar-mirror-based catadioptric method. The curved mirror methods [23] [24] [25] [26] [27] use the combination of curved mirror and single camera, enlarge the visual angle of the single camera by the reflective effects of the curved mirror. These methods are able to acquire wide FOV images in real time without stitching line. But as the same as the fisheye lens, it has the disadvantages of low resolution. The planar-mirror-based catadioptric methods are widely used to acquire wide FOV images in recent years. These methods are firstly proposed by Nalwa [1] [2] [3] . He obtained 360 • cylindrical panoramic video images by planar-mirror-based catadioptric system consisting of 4 cameras and 4 trapezoid mirrors and achieved ideal performance. According to Nalwas' idea, Hua [28, 29] obtained cylindrical panoramic video images with the wide vertical view angle by bilayer planar mirrors. Gao [30] also obtained real-time hemisphere panoramic video images using hexahedral planar mirrors. The innovation of planarmirror-based catadioptric methods is that after focusing different cameras' optic centers into a single point by planar mirrors, parallax between images captured by different cameras can be eliminated and the final wide FOV images have the advantages of good real-time performance, high resolution and not affected by the depth of field. However, Nalwas' method uses planar-mirror-based catadioptric system to focus optical axis of different cameras into a single point strictly; the accuracy of the mirrors and the equipments is really high. Because there is no overlap region between images, it is impossible to make image fusion at the stitch areas. Thus, there will be obvious stitching lines in the resulting wide FOV images.. Image registration is the basis of video image mosaic. Commonly used registration methods are mainly divided into three classes: template matching, mutual information, and features-based methods. Template matching methods firstly get a matching template by selecting a window of gray information from the overlap areas directly [31] and then search in another image until reach the highest matching score. This kind of methods can solve the image registration with low computational complexity. Mutual information methods were firstly proposed by Kughn and Hines [32] in 1975. The early mutual information methods can only be used for image registration when there is only pure translations between two images. Then, Keller et al. [33] introduced the polar Fourier transform into the mutual information methods, which can achieve the improved image registration stability with translation, rotation, and scale transformation. Zhou et al. [34] propose a Bayesian-based mutual information technique, combined with an established affine transformation model, which can register images with affine transformation efficiently and accurately. However, mutual information methods often require the mosaic images have larger overlap area and would fail in a smaller percentage of overlap situation. Corner features algorithm was proposed by Harrs [35] in 1998, which is the most representative method in features-based registration. This method is invariant with respect to difference of rotation and image brightness and can be widely used in 2D image registration [36] and 3D reconstruction [37] . But it has higher computational complexity, which make it not suitable for real-time demanding registration occasion.. In this paper, to describe the fault of the traditional planarmirror-based catadioptric methods, which need high optical designing precision, we first establish a mathematical model to demonstrate the influence of the parallax to the image mosaic and then proposed a mirror pyramid camera which has undemanding requirement of machining precision and can focus 3 cameras' optic centers into one point approximately by adjusting the screw nuts. Thus, the final FOV images can be achieved by image registration. Finally, to eliminate the stitch line caused by optical difference effectively, we proposed an improved algorithm based on nonlinear weighting fusion. To solve the problems of mirror pyramid cameras, such as too high precision required for designing and video images with inevitable stitching line, we proposed a mirror pyramid camera with undemanding precision for designing and processing. First, we devised a screw nut on our design system to adjust the cameras' height to make their optic centers located at a single point. Then, by the method of nonlinear weighting fusion, we can eliminate the stitching line effectively. The experimental results indicate that our system has the characteristics of good real time, high resolution, without stitching line and not affected by depth of field. According to the theory in this paper, it is easy to design a panoramic camera with ideal mosaicing performance.. However, our catadioptric image mosaicing device has a limited imaging range (in this paper, the range is from 0.4 meter to infinity); if the users want to get the better mosaicing result within 0.4 meter, we need to improve our system to reduce the optical parallax between cameras. On the other hand, when we design our catadioptric image mosaicing system, we do not consider the cameras' difference, which would be an adverse factor in image mosaicing. So in our future work, we will make more precise calibration for the camera selection to solve this problem.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A comparison of models for predicting early hospital readmissions In this work we describe and compare several predictive models, some of which have never been applied 34 to this task and which outperform the regression methods that are typically applied in the healthcare lit-35 erature. In addition, we apply methods from deep learning to the five conditions CMS is using to penalize 36 hospitals, and offer a simple framework for determining which conditions are most cost effective to 37 target. readmissions within 30 days of discharge, with 76% of these being 63 potentially avoidable [3] . In total, these readmissions accounted for increased to a minimum of 3% of a hospital's Medicare reimburse-71 ment, and also included several more conditions [1] . 72 Hospital leaders recognize that scrutiny over readmission rates 73 will continue to grow over the next few years, and that the finan- ing patient readmission risk, using many types of available data. 88 Some methods, such as in [4] , leverage a variety of data sources, 89 including patient demographic and social characteristics, medica- based on only a single source of data, for instance, solely on admin-92 istrative claims data, as in [5] . A thorough review of past models 93 can be found in [6] . With the exception of [7] , all of these methods 94 are logistic regressions on independent variables typically chosen 95 by hand. 96 Our aim is to compare in detail existing methods used to predict 97 readmission with many other statistical methods. These methods 98 include ''local'' models tailored to particular patient subpopula-99 tions as well as ''global'' models fit to the entire dataset. We com-100 pare penalized linear models as well as non-linear models such as 101 random forests and deep learning. Due to the increased difficulty of 102 training deep models, we conduct a smaller set of experiments to 103 validate their performance. 104 The remainder of this paper will be organized as follows. forces some coefficients to be exactly zero. The Elastic Net of 233 [10] is a combination of the two, and is also commonly used. .

O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Shakespeare's complete works as a benchmark for evaluating multiscale document navigation techniques In this paper, we describe an experimental platform dedicated to the comparative evaluation of multiscale electronic-document navigation techniques. One noteworthy characteristic of our platform is that it allows the user not only to translate the document (for example, to pan and zoom) but also to tilt the virtual camera to obtain freely chosen perspective views of the document. Second, the platform makes it possible to explore, with semantic zooming, the 150,000 verses that comprise the complete works of William Shakespeare. We argue that reaching and selecting one specific verse in this very large text corpus amounts to a perfectly well defined Fitts task, leading to rigorous assessments of target acquisition performance. For lack of a standard, the various multiscale techniques that have been reported recently in the literature are difficult to compare. We recommend that Shakespeare's complete works, converted into a single document that can be zoomed both geometrically and semantically, be used as a benchmark to facilitate systematic experimental comparisons, using Fitts' target acquisition paradigm. This paper is concerned with the establishment of scientific norms within the human-computer interaction (HCI) research community for the quantitative evaluation of multiscale electronic-document navigation techniques, a domain in which the innovation rate is currently fairly high [1] [5] [12] [13] [17] [19] . Twenty seven years of research after Card et al.'s study of simple, single-scale, pointing tasks [4] have recently led to an ISO recommendation for the experimental evaluation of "non-keyboard input devices" [14] , which apparently has already exerted an appreciable impact on practice in input research [20] . Research on target acquisition in multiscale documents, however, is of more recent appearance [9] . We believe that collectively adopting a standard multiscale document, held as a benchmark, would help researchers to draw comparisons among the variety of available techniques and thus constitute a useful first step towards standardization in this important field of HCI research.. Below, we start by reminding the basics of Fitts target acquisition task and Fitts' law, and recall why this paradigm is useful. Next, we turn to multiscale documents, showing that Fitts' paradigm is still perfectly appropriate to assess performance in the multiscale case. We then list the main conditions that we believe have to be met for performance data from different studies to be comparable, taking examples from the current literature. We end with the suggestion that Shakespeare's complete works, implemented in a single, highly flexible platform which we have recently developed, and which we are making available to the HCI community, might constitute a suitable evaluation benchmark.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I


Fast Factorization by Similarity of Fuzzy Concept Lattices with Hedges The paper presents results on factorization by similarity of fuzzy concept lattices with hedges. A fuzzy concept lattice is a hierarchically ordered collection of clusters extracted from tabular data. The basic idea of factorization by similarity is to have, instead of a possibly large original fuzzy concept lattice, its factor lattice. The factor lattice contains less clusters than the original concept lattice but, at the same time, represents a reasonable approximation of the original concept lattice and provides us with a granular view on the original concept lattice. The factor lattice results by factorization of the original fuzzy concept lattice by a similarity relation. The similarity relation is specified by a user by means of a single parameter, called a similarity threshold. Smaller similarity thresholds lead to smaller factor lattices, i.e. to more comprehensible but less accurate approximations of the original concept lattice. Therefore, factorization by similarity provides a trade-off between comprehensibility and precision. We first describe the notion of factorization. Second, we present a way to compute the factor lattice directly from input data, i.e. without the need to compute the possibly large original concept lattice. Third, we provide an illustrative example to demonstrate our method.

O B O B O O O O O O O O O O O B O B O O O O O O O O B I O O O O O O O O O O O O O O O B O B O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O B O O O O B I O O B O O B O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O B I O B O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O


PluralisMAC: a generic multi-MAC framework for heterogeneous, multiservice wireless networks, applied to smart containers Developing energy-efficient MAC protocols for lightweight wireless systems has been a challenging task for decades because of the specific requirements of various applications and the varying environments in which wireless systems are deployed. Many MAC protocols for wireless networks have been proposed, often custommade for a specific application. It is clear that one MAC does not fit all the requirements. So, how should a MAC layer deal with an application that has several modes (each with different requirements) or with the deployment of another application during the lifetime of the system? Especially in a mobile wireless system, like Smart Monitoring of Containers, we cannot know in advance the application state (empty container versus stuffed container). Dynamic switching between different energy-efficient MAC strategies is needed. Our architecture, called PluralisMAC, contains a generic multi-MAC framework and a generic neighbour monitoring and filtering framework. To validate the real-world feasibility of our architecture, we have implemented it in TinyOS and have done experiments on the TMote Sky nodes in the w-iLab.t testbed. Experimental results show that dynamic switching between MAC strategies is possible with minimal receive chain overhead, while meeting the various application requirements (reliability and low-energy consumption). Developing energy-efficient MAC protocols for lightweight wireless systems has been a challenging task for decades because of the specific requirements of various applications and the varying environments in which wireless systems are deployed. Many MAC protocols for wireless networks have been proposed, often custommade for a specific application. However, we have noticed two evolutions in wireless (sensor) networks. First, there is a shift from simple monitoring applications towards multiple applications (or an application with several modes) in the same network. Second, there is a shift from homogeneous devices towards heterogeneous devices (e.g. mobile battery-powered devices with limited processing power and a minimum required lifetime of 2 years, fixed mains-powered devices with many communication interfaces, or battery-powered portable devices which are easy to recharge on a daily or weekly basis). So, how should a MAC layer deal with an application that has several modes (each with different requirements) or with the deployment of another application during the lifetime of the system? This is exactly the problem we have encountered during the design and implementation of a solution of a smart container monitoring system. The architecture is a complex system of mobile and lightweight devices with stringent energy consumption requirements and varying requirements for the MAC layer because of the peculiarities of the monitoring application.. To the best of the authors' knowledge, we have not found a MAC protocol that can meet all these requirements. Although hybrid MAC protocols exist, these are not sufficient in mobile systems, like the smart container monitoring system. This smart container use case, which will be described in Section 2, has inspired us to create a multi-MAC framework and a neighbour management framework (NMF). These frameworks are generic, so they can be deployed in a broader context and deal with other use cases (e.g. building automation, smart cities, etc.) as well that have conflicting requirements that cannot be addressed by monolithic or inflexible solutions.. Instead of designing one monolithic MAC protocol, we will switch between MAC strategies (the so-called maclets) based on the application state and requirements. In Section 3, we will show why one MAC does not fit for all our requirements. Section 4 gives an overview of the PluralisMAC design goals and our architecture is described in Section 5. We will offer common functionalities of a MAC protocol in shared primitives. This way, maclets do not have to implement these themselves. We have implemented this in TinyOS and have done experiments on the w-iLab.t testbed (Section 6). The results of our experiments are presented in Section 7. In Section 8, an overview of related work is given. Lastly, we conclude by a summary in Section 9. In this article, we have described the smart container monitoring use case which made clear that multiple MAC strategies were needed to achieve a low-power system able to handle wireless communication with heterogeneous devices. We have presented our generic multi-MAC framework and NMF. We have implemented our architecture in TinyOS and have done experiments on the w-iLab.t testbed. We have shown that our framework works, and it does not introduce a significant overhead (e.g. 4 ms in the receive chain for both the NMF and the multi-MAC framework). We will start to implement more (complex) reference maclets, composed primitives and neighbour filters in the near future and hope that our framework will be used by many MAC designers to implement and test their design on real hardware.

O O O B B O O O B I O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B O O O O O O O O O O B B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B I O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O B B O O B I B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O


Learning classifier systems with memory condition to solve non-Markov problems In the family of Learning Classifier Systems, the classifier system XCS has been successfully used for many applications. However, the standard XCS has no memory mechanism and can only learn optimal policy in Markov environments, where the optimal action is determined solely by the state of current sensory input. In practice, most environments are partially observable environments on agent's sensation, which are also known as non-Markov environments. Within these environments, XCS either fails, or only develops a suboptimal policy, since it has no memory. In this work, we develop a new classifier system based on XCS to tackle this problem. It adds an internal message list to XCS as the memory list to record input sensation history, and extends a small number of classifiers with memory conditions. The classifier's memory condition, as a foothold to disambiguate non-Markov states, is used to sense a specified element in the memory list. Besides, a detection method is employed to recognize non-Markov states in environments, to avoid these states controlling over classifiers' memory conditions. Furthermore, four sets of different complex maze environments have been tested by the proposed method. Experimental results show that our system is one of the best techniques to solve partially observable environments, compared with some well-known classifier systems proposed for these environments. Learning Classifier System (LCS) is a rule-based machine learning technique based on Genetic Algorithm and reinforcement learning. It can map the input stimuli to output actions by a population of compact and general "condition-action-payoff rules", called classifiers. Through an evolutionary mechanism, LCS can also learn to adapt to new or changing environments. The original Learning oped an enhanced version of XCSM, named "XCSMH" (Lanzi 1998b; Lanzi and Wilson 2000) . Compared with XCSM, XCSMH consists of (i) a different policy for updating the contents of the internal memory and (ii) a hierarchical exploration strategy to select actions. Lanzi's experiments show that XCSMH can evolve an optimal policy for most maze environments previously presented. Especially, when applied to a more difficult environment, such as Maze10 (shown in Fig. 11(b) ), XCSMH still converges to a near optimal solution, outperforming XCSM (Lanzi 1998b; Lanzi and Wilson 2000) .. And then, Hamzeh et al. developed two distributed-architecture classifier systems, named Parallel Specialized XCS (PSXCS) (Hamzeh and Rahmani 2008) and Recursive PSXCS (RPSXCS) (Hamzeh et al. 2009 ). PSXCS and RPSXCS detect aliased environmental states and assign their handling to some subordinate XCS. So, in their systems, some special sub-XCSs equipped with history windows are responsible for each detected aliasing state respectively, which causes the structure of the systems is somewhat complicated. Experimental results show that their systems can perform optimally in some well-known benchmark problems.. In order to further improve the performance of XCS within non-Markov problems, we will discuss a different way to include memory within XCS. Our proposal adds XCS an internal message list as the memory list, in which the length of a list element equals to that of strings from the detectors.. Then, only extends a small number of classifiers with memory conditions, to cope with non-Markov states within the environments. Besides, in order to make the memory condition match the earlier environmental state, not just the state at the last time step, our system uses the memory list to store some recent experiences, i.e. recent perception inputs from detectors and, if necessary, corresponding actions for these perceptions. But the memory condition must keep away from those aliasing states in the memory list. Because if the memory condition of a classifier matches an aliasing state in the memory list, this classifier will be confused at the aliasing state, and have no benefit to the system1. Therefore, to overcome this problem, a method is needed to detect those aliasing states. We introduce and improve an aliasing state detection method (Dung et al. 2008; Hamzeh et al. 2009; Hamzeh and Rahmani 2008) to our system. With the help of the detection method, aliasing states in the environment will be recognized, thus the memory condition can always avoid these states and become efficient and effective. From the above, our proposed method is based on XCS, including an internal memory list, an extended condition in the classifier (memory condition), and an aliasing state detection mechanism.. Later, we will refer to our proposal as "XCSMD", and the "MD" stands for "memory" and "detection". Experimental results show that XCS with memory adopting our method can perform optimally or near-optimally in partially observable environments compared with some of the best classifier systems. Additionally, XCSMD holds a clear memory mechanism, which is easy to understand.. The rest of the paper is organized as follows. Section 2 reviews XCS. Section 3 briefly introduces the maze environments and the perceptual aliasing problem, including some related work on it. The implementation of our proposal, "XCSMD", is described in Section 4. Experiments with XCSMD are presented in Section 5 and Section 6. Finally, Section 7 ends the paper, presents our main conclusions and some directions for future research.

O O O O B I O O B I O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B I O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B I O O B I I O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Deformable

models

with

sparsity

constraints

for

cardiac

motion

analysis

Deformable

models

integrate

bottom-up

information

derived

from

image

appearance

cues

and

top-down

priori

knowledge

of

the

shape.

They

have

been

widely

used

with

success

in

medical

image

analysis.

One

limitation

of

traditional

deformable

models

is

that

the

information

extracted

from

the

image

data

may

contain

gross

errors,

which

adversely

affect

the

deformation

accuracy.

To

alleviate

this

issue,

we

introduce

a

new

family

of

deformable

models

that

are

inspired

from

the

compressed

sensing,

a

technique

for

accurate

signal

reconstruction

by

harnessing

some

sparseness

priors.

In

this

paper,

we

employ

sparsity

constraints

to

handle

the

outliers

or

gross

errors,

and

integrate

them

seamlessly

with

deformable

models.

The

proposed

new

formulation

is

applied

to

the

analysis

of

cardiac

motion

using

tagged

magnetic

resonance

imaging

(tMRI),

where

the

automated

tagging

line

tracking

results

are

very

noisy

due

to

the

poor

image

quality.

Our

new

deformable

models

track

the

heart

motion

robustly,

and

the

resulting

strains

are

consistent

with

those

calculated

from

manual

labels.

Physics-based

deformable

models

and

their

variations

have

been

studied

extensively

in

recent

decades

and

widely

used

in

computer

vision,

computer

graphics

and

medical

image

analysis

(Metaxas,

McInerney

and

Terzopoulos,

Metaxas,

Nealen

et

al.,

They

are

able

to

solve

diverse

types

of

problems,

such

as,

but

not

limited

to,

image

segmentation

image

registration

(Rueckert

et

al.,

Shen

and

Davatzikos,

shape

reconstruction

Metaxas

and

Terzopoulos,

and

motion

analysis

Paragios

and

Deriche,

The

name

''deformable

models''

is

derived

from

the

nonrigid

body

mechanics,

which

describes

how

elastic

objects

respond

to

applied

forces.

Starting

from

an

initial

shape,

the

model

is

usually

deformed

by

two

types

of

forces,

i.e.,

internal

and

external

forces.

The

external

force

drives

the

model

to

fit

the

observations,

while

the

internal

force

constrains

the

geometric

flexibility

of

the

shape.

For

examples,

in

the

image

segmentation

problem,

the

external

force

computed

from

the

image

intensity

drives

the

model

to

the

estimated

boundary,

while

the

internal

force

keeps

the

boundary

smooth.

In

the

motion

analysis

(e.g.,

cardiac

motions

(Park

et

al.,

Haber

et

al.,

Hu

et

al.,

Chen

et

al.,

Wang

and

Amini,

and

the

shape

manipulation

problems

(Nealen

et

al.,

control

points

are

employed

as

the

external

force

to

drive

the

model,

and

the

internal

force

maintains

the

smoothness

and

preserves

shape

details.

The

control

points

are

tracked

along

a

motion

sequence,

and

then

an

initial

model

is

deformed

to

fit

the

control

points

in

each

following

frame.

This

is

often

measured

by

the

distances

between

the

control

points

and

the

corresponding

points

on

the

initial

model.

In

fact,

in

the

context

of

motion

analysis

and

shape

manipulation,

many

previous

methods

(Sorkine

et

al.,

Zhou

et

al.,

Yan

et

al.,

Wang

et

al.,

use

Euclidean

distance

or

norm

as

the

distance

metric

for

penalty

functions.

This

assumes

intrinsically

that

the

errors

of

the

target

points

follow

a

Gaussian

distribution

with

small

variances.

Nevertheless,

this

is

not

always

true

in

practice.

Since

the

control

points

are

usually

from

automated

detections,

they

may

contain

not

only

Gaussian

noise,

but

also

some

gross

errors

or

outliers

due

to

the

erroneous

detection.

Therefore,

the

accuracy

of

the

traditional

deformable

models

depends

heavily

on

the

accuracy

of

the

control

point

detection..

In

this

paper,

we

focus

on

improving

the

robustness

of

traditional

deformable

models,

particularly

for

the

problems

of

cardiac

motion

analysis.

Inspired

by

the

robust

recovery

power

of

the

compressed

sensing

approach

(Donoho,

we

propose

a

new

class

of

deformable

models

using

sparse

regularization.

Recent

research

in

compressed

sensing

shows

that

using

an

norm

can

dramatically

increase

the

Elsevier

B.V.

All

rights

reserved..

probability

of

accurate

signal

recovery,

even

when

there

are

both

sparse

outliers

and

moderate

Gaussian

noise

Thus,

we

design

a

robust

deformable

model

by

integrating

seamlessly

an

norm

regularization

with

a

modified

Laplacian

deformable

model

(Sorkine

et

al.,

Yu

et

al.,

This

new

model

is

able

to

handle

outliers

or

gross

errors.

In

addition,

it

is

designed

as

a

convex

optimization

problem,

and

can

be

efficiently

solved

within

a

constrained

solution

space.

However,

when

the

variances

of

the

Gaussian

noise

are

large,

solely

using

the

norm

may

cause

overfitting

problems

due

to

its

nature

of

pursuing

the

sparse

structure

(Candes

et

al.,

Therefore,

we

propose

a

deformable

model

using

hybrid

norm

regularization

that

is

able

to

handle

both

the

Gaussian

errors

and

gross

errors.

We

also

generalize

these

two

models

in

a

unified

formulation,

named

as

sparse

deformable

models..

In

the

following

section,

we

discuss

the

relevant

work

of

deformable

models

and

compressed

sensing.

Our

proposed

sparse

deformable

models

(SDM)

are

presented

in

detail

in

Section

In

Section

we

validate

our

models

on

a

clinically

important

and

challenging

problem,

i.e.,

the

left

ventricle

(LV)

motion

analysis

in

mouse

cardiac

tagged

MRI.

Section

presents

experimental

results

demonstrating

the

robustness

of

our

models

on

mouse

heart

motion

tracking

even

with

inaccurate

results

of

control

point

detection.

The

last

section

draws

conclusions

on

model

advantages,

and

discusses

directions

of

future

work.

In

this

paper,

we

introduce

a

group

of

sparse

deformable

models.

Benefitted

from

the

sparsity

techniques,

these

deformable

models

are

able

to

handle

outliers

or

gross

errors.

Therefore,

they

are

robust

to

deal

with

noisy

images

or

tracking

errors.

We

have

validated

these

methods

on

the

synthetic

data

and

the

mouse

cardiac

motion

tracking.

Both

qualitative

and

quantitative

results

demonstrate

that

our

methods

outperform

and

are

more

robust

than

previous

ones.

It

is

also

noteworthy

that

the

applications

of

our

proposed

methods

are

not

limited

to

cardiac

motion

analysis.

It

is

flexible

enough

for

many

other

medical

image

problems..

In

the

future,

we

plan

to

extend

the

deformable

models

by

using

structured

priors.

The

LV

is

conventionally

separated

into

segments.

This

inspires

us

to

add

group

constraints

to

the

current

sparse

model.

The

group

sparsity

and

other

structure

sparsity

constraints

will

further

improve

the

robustness

of

the

model.

The

current

regularization

term

is

only

related

to

the

external

force

based

on

noisy

observations.

It

is

also

possible

to

extend

the

other

parts

of

the

model.

The

problem

for

modeling

arbitrary

internal

force

is

that

the

resulting

model

may

not

be

a

convex

problem.

The

traditional

finite

difference

method

can

be

employed

to

find

a

local

minimum,

while

the

performance

should

then

be

further

analyzed.

Consider

a

set

of

points

V,

where

each

point

has

a

neighborhood

structure

The

goal

of

our

deformable

models

is

to

track

the

motion

of

the

whole

shape,

given

the

position

V

This

will

be

further

discussed

in

the

internal

force.

The

deformation

of

each

point

i

is

parameterized

by

an

affine

Â

transformation

matrix

T

i

where

the

first

three

columns

are

linear

transformation

and

the

last

column

is

translation.

In

our

model,

the

internal

force

preserves

the

local

shape

structure

by

using

the

Laplacian

coordinate,

and

the

external

force

minimizes

the

difference

between

the

shape

and

the

control

points

via

our

proposed

sparse

constraints.

B I O O O O B I I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I I O O O O O O O O B I O O O O O O O O O O B I O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Applicative functors and fully transparent higher-order modules we present a variety of the Standard ML module system where parameterized abstract types (i.e. functors returning generative types) map provably equal arguments to compatible abstract types, instead of generating distinct types at each applications as in Standard ML. This extension solves the full transparency problem (how to give syntactic signatures for higher-order functors that express exactly their propagation of type equations), and also provides better support for non-closed code fragments.

O O O O O O O O O O O O O O B B O O O O B O O O O B O O O O O O O B O O O O B O O B O O O B O O O O O B O O O O O O O O O O O O O B O O O O O O O B O O B O