Patent Description:
It is more and more common for people to carry mobile computing devices that include built-in cameras, such as smartphones or tablet computing devices. As the processing speed and storage capacities of these devices increase, people may more commonly use the devices to capture videos or various series of pictures (e.g., captured by holding down the shutter button to capture many pictures over a short period of time). These videos and series of pictures can capture a scene, in which objects or humans move through the scene from image to image.

This document describes techniques, methods, systems, and other mechanisms for detecting motion in images and selectively storing images. The motion-detection mechanism may compare a recently-received image to one that was previously-received to identify which objects moved in the scene that is shown by the images. Because the camera may have been moving, which would cause stationary objects in the background to appear at different locations in each of the images, the motion-detection mechanism may analyze the images to identify how the camera moved. It may then use this analysis to modify the previously-received image to show its content from an estimated orientation of the camera when the recently-received image was captured. In this way, the background may appear to remain substantially the same and stationary between the modified previously-received image and the currently-received image. This enables the system to analyze the two images to identify objects that are moving independent of the background. The motion data is used to select whether to save an image or discard the image.

Particular implementations can, in certain instances, realize one or more of the following advantages. The technology described in this disclosure allows a device to determine when an object in a scene that is being captured by a camera moves, even when the camera itself is moving. As such, the device is able to distinguish movement of an object in a scene from an apparent movement of the background of the scene that is caused by the camera movement. The device may distinguish foreground from background movement by compensating for movement of the camera in all eight degrees of movement. The device is not only able to determine the region of the image at which movement occurs, but may be able to generate a general indication of motion saliency, for example, an indication of significance of movement in the scene. Moreover, the processes described herein may not require significant processing power and may be able to fully-compensate for eight degrees-of-freedom of camera motion, and thus may be suitable for real-time computation on mobile computing devices.

This document generally describes detecting motion in images. A computing system performs a motion-detection process by comparing two images in order to identify which portions of the image show objects that were moving in real life, and to generate a value that identifies a level of significance to this movement (e.g., a person jumping through the air may be more significant than many small leaves drifting in the wind). The computing system compensates for movement of the camera, which helps the computing system distinguish stationary objects that appear to be moving from one image to the next due to movement of the camera, from those objects that are actually moving in real life with respect to the stationary objects.

The identification of which portions of an image are moving, and a level of significance of that movement is used by the computing system. The computing system determines which images, of a series of images that is captured by the computing system, to save and which images to delete. As an illustration, should a user hold down a shutter button to capture a series of images, many of those images may be nearly identical and it may be unhelpful to permanently store all of the nearly-identical images or even provide them for display to a user. As such, the computing system may determine which images represent a significant level of movement with respect to other images, and may store only those that images show that significant level of movement. This process is graphically illustrated and explained with respect to <FIG>.

<FIG> shows graphical illustration of a process for detecting motion in images. In this illustration, a user of a mobile computing device (a phone in this example, but it could also be a laptop or a stand-alone camera, for example) has captured a series of images A-D of a friend crouching in place and then jumping through the air. The user may have oriented the camera lens of his phone camera to face the friend, and the user may have pressed and held down a shutter button just before the friend jumped to cause the phone to capture a series of images of the friend jumping. The series of images may include two images of the friend preparing to jump (Images A and B), one of the friend leaping into the air (Image C), and one of the friend coming back to the ground (Image D).

Although this illustration shows the phone capturing four images for ease of illustration, it is possible that the phone captured dozens of images over the same time period. It may not make sense to permanently store all of these images because the images occupy potentially-valuable computer memory, and because some of the images may be nearly identical. As such, the phone may be programmed to estimate which of the images the user may be the most interested in viewing, and may delete the remaining images without even providing the deleted images for user review or display. As an example, the phone may store the captured images in a buffer, but once the buffers fills up, the computing system may delete images that scored low in order to allow the buffer to store more-highly scoring images that are being received. The computing system may perform or at least start the scoring process with each newly-received image before the computing system captures the next image.

A first step to identify motion in the images may be to identify two images for comparison. The two images may be images that were captured adjacent to each other in time. For example, the images may be adjacent frames in a video.

Next, the computing system compensates for movement of the phone. Because the phone is a mobile device, the user may move the phone as the user is capturing the series of pictures (e.g., by translating the phone or twisting the phone). Due to the movement of the phone, it can be difficult to compare the two images directly to each other, because the position of items that were stationary in the scene that was captured by the images may appear in different positions of the images due to the camera movement.

The computing system compensates for movement of the camera by generating, using the first image and the second image, a mathematical transformation that indicates movement of the camera from the first image to the second image with respect to a scene that is reflected in the first image and the second image (box <NUM>). The mathematical transformation (item <NUM>) may be a mathematical number, series of numbers, matrix, or algorithm that indicates or can be used to indicate movement of a camera with respect to a scene from one image to the next. That transformation may be generated by identifying the locations of the same features in each of the images, and identifying how the features have moved from one image to the next. As described below, the mathematical transformation <NUM> can be used to modify pixels of one of the images to estimate the capture of the same scene at the same time from a different location (e.g., a location at which the other of the two images was captured).

The computing system then generates, using the first image and the mathematical transformation, a modified version of the first image. The modified version of the first image presents the scene that was captured by the first image from a position of the camera when the second image was captured (box <NUM>). In other words, the computing system may take the first image and run it through a mathematical process that also uses the mathematical transformation <NUM> as an input. The effect of the mathematical process may be to move at least some of the pixels in the first image to new positions, in a manner that is specified by or indicated by the mathematical transformation. This rearrangement may generate a new image that is a "warped" version of the original image, and that appears to show the original image from a different camera perspective. The modified first image is illustrated in <FIG> as Image B' (item <NUM>). A position (e.g., location and/or orientation) of the camera when the first image was captured may be different from a position of the camera when the second image was captured.

The computing system then compares the modified version of the first image to the second image (box <NUM>). A computational output of this comparison includes an indication of a portion of the second image at which an object moved (e.g., with respect to a stationary background). The one or more outputs of these comparison processes are illustrated in <FIG> as motion data <NUM>. The comparison processes are described in additional detail with respect to <FIG>. Generally though, the comparison processes identify which portions of the images changed (after having compensated for camera movement). The comparison processes may additionally identify which portions of the images represent edges of objects, and the computations may emphasize changes at positions of edges over changes to features with less-prominent edges.

The phone uses the motion data to select whether to save an image or discard the image (box <NUM>). For example, as described above, the device may capture more images than is necessary to store or show to a user. Thus, the device removes some of the captured images from memory before the user is given a chance to view the images (e.g., the images are removed without user input). The computing system may perform these removal operations as a result of the computing system having either a fixed number of images that it is configured to store for any given captured sequence of images (e.g., a fixed buffer size) or whenever an image has a image score that falls below a given threshold (e.g., delete uninteresting images, even if the buffer may not be full).

An example input for determining which images are interesting or are not interesting (e.g., an input that is used to calculate the above-describe image score), is an input that specifies the saliency or importance to the motion in the image, which may be determined based on the above description. This motion-identifying input may be used with other inputs (e.g., a score that specifies whether people in the image have their eyes open, and a score that specifies whether the image is not blurry) to generate an overall score for the image. That overall score may be used to determine whether or not to remove the image or keep the image for later presentation to a user. Of course, the motion-detection techniques described herein may be used to achieve other results, for example, to track a location of an object.

<FIG> shows a graphical illustration of a process for comparing two images. The processes shown in <FIG> may represent additional detail regarding the comparison operation that is previously described at box <NUM> (<FIG>).

The comparison may include the device initially computing some statistical information about the modified first image and about the second image. For example, the device may compare the images to identify a temporal gradient <NUM> (box <NUM>). The temporal gradient data <NUM> may represent the pixel difference between the images. Because the modified first image represents the image taken from the position of the camera when it captured the second image, portions of the images that represent stationary features may have similar pixel values. As such, the pixel difference at such image locations may be zero or near zero. On the other hand, at locations in the images at which an object moved there may be a notable pixel difference (e.g., either a location at which an object was but is no more, or at a location at which an object was not but is now). The temporal gradient may represent the temporal or time difference from one image to the next, and may be calculated for multiple pixels (e.g., each pixel in the image).

The device may also calculate and identify the spatial gradient from the second image (box <NUM>). The calculation may generate spatial gradient data <NUM>, which may indicate how the image differs from one pixel to the next in a certain direction in the image. For example, a horizontal gradient may identify, for any given pixel in the image, how the grayscale value for the pixel to the left of the given pixel differs from the grayscale value for the pixel to the right of the given pixel. As another example, a vertical gradient may identify, for any given pixel, how the grayscale value for the pixel to the top differs from the grayscale value for the pixel to the bottom. Significant spatial gradient values may indicate the presence of edges in the image.

The computing device may use these statistical values to identify locations of the image at which motion occurs. This analysis may be performed on patches or regions of the image. As such, the computing system may generate a grid of patches or a grid of multiple regions (box <NUM>). In the following detailed description, the term grid of patches is used. Generating the grid of patches may include generating a grid of evenly-spaced points on an area representative of one of the images, and generating a patch (e.g., a <NUM>-pixel by <NUM>-pixel square) that is centered on each of the evenly-spaced points. The patches in the grid of patches <NUM> may or may not overlap, or may abut each other (they are shown in <FIG> with space between each patch, such that they do not overlap or abut each other).

The device may then calculate a motion score for each patch, using the temporal gradient data <NUM> and the spatial gradient data <NUM> (box <NUM>). The computation of the motion score for each patch is described in additional detail with regard to <FIG>, but this calculation may generate a score map <NUM>. The score map <NUM> may include one value for each patch that indicates the saliency of the motion in that patch. It is this score map <NUM> (or a reduced version thereof) that may be used by the device to indicate at which regions of the image motion is occurring. In <FIG>, the highest values in score map <NUM> are shown at the region of Images B' and C at which the friend moved. The values in score map <NUM> are illustrated as ranging from <NUM>-<NUM>, but the values may occupy other ranges, such as a range from <NUM> and <NUM>.

The device may then compute an overall motion score value (box <NUM>). In particular, the device may use the values in score map <NUM> to generate overall motion score value data <NUM>. In various examples, computing the overall motion score value data <NUM> may include averaging the values in the score map <NUM>. In some examples, the overall motion score value data <NUM> is calculated using a nonlinear mapping function, which normalizes values to a standard range (e.g., between <NUM> and <NUM>), as described in additional detail with respect to <FIG>.

<FIG> show a flowchart illustration of a process for detecting motion in images. The process described with respect to <FIG> provide additional description regarding at least some aspects of the process described with respect to <FIG>.

At box <NUM>, the computing system receives an image. The received image may be an image that was captured by a camera (e.g., an image sensor) of a computing device most recently. The computing system may downsample the image (box <NUM>) in order to reduce a level of processing that is needed to perform the motion-detection processes that are described herein. For example, the received image may have a resolution of 1920x1080 pixels and the downsampling process may convert the received image to a smaller resolution of 320x180 pixels. In some implementations, the computing system also converts the received image (e.g., before or after downsampling, independent of any downsampling) from color to grayscale. The computing system may store the received image (and/or the downsampled and color-converted version thereof) in an image buffer <NUM>, so that the system has access to previously-captured images. The images on which the processing is performed, whether the original image or a downsampled and color converted version thereof, is designated as I(x,y).

Determining which portions of the image represent an object that is moving can involve comparing the received image to a previously-received image, although this is not forming part of the invention. But, if the camera is moving, all or most of the received image and the previously-received image may be different due to the camera being at a different position at different points of time. Thus, it can be helpful to "warp" the previously-received image so that it is shown from the vantage point of the received image. Doing so can involve analyzing both images to identify how the camera moved and generating a transformation that indicates or otherwise identifies the motion of the camera, as described in additional detail below.

At box <NUM>, the computing system estimates camera motion and generates a transformation that indicates the motion of the camera. The generated transformation may be a matrix that is created using at least two images as input (e.g., I and I_previous, from the image buffer). The transformation may be designated "H_interframe. " This frame-to-frame motion matrix may be a homography transform matrix. A homography transform matrix may be a matrix that can represent movement of a scene or movement of a camera that was capturing the scene, from one image to the next (e.g., from I_previous to I).

As an illustration, suppose that a first image represents a picture taken of a square from directly in front of the square, so that the square had equal-length sides with ninety-degree angles in the image (in other words it appeared square). Suppose now that the camera was moved to the side (or the square itself was moved) so that a next image displayed the square as skewed, with some sides longer than each other and with angles that are not ninety degrees. The location of the four corner points of the square in the first image can be mapped to the location of the four corner points in the second image to identify how the camera or scene moved from one image to the next.

The identified mapping of these corner points to each other in the images can be used to generate a homography transform matrix that represents the motion of the camera viewpoint with respect to the scene that it is capturing. Given such a homography transform matrix, a system can combine the first image with the generated homography transform matrix to recreate the second frame, for example, by moving pixels in the first frame to different locations according to known homography transformation methods.

The homography transform matrix that is described above can represent not only translational movement of a camera, but also rotation, zooming, and non-rigid rolling shutter distortion. In this way, the homography transform matrix can represent movement of the camera in eight degrees-of-freedom. To compare, some image comparison techniques only account for translational movement (e.g., up/down and left/right movement).

The above-described homography transform matrix may be a 3x3 homography transform matrix, although other types of homography matrices may be used (and other mathematical representations of movement from one image to another, even if not a homography matrix or even if not a matrix, may be used). The system may determine the 3x3 matrix (H_interframe) in the following manner. First, the computing system may identify a set of feature points (sometimes corner points) in a current image, where those points may be denoted [x'_i, y'_i], i = <NUM>. N (N is the number of feature points). Then, the computing system may identify corresponding feature points in the previous frame, where the corresponding feature points may be denoted [x_i, y_i]. Note that the points are described as being in the GL coordinate system (i.e., the x and y ranges from -<NUM> to <NUM>, with the frame center as the origin). If the points are in the image pixel coordinate system in which x ranges from <NUM> to the image width and y ranges from <NUM> to the image height, then the points can be transformed to the GL coordinate system or the resulting matrix can be transformed to compensate.

The above-described H_interfame matrix may be a 3x3 matrix which contains <NUM> elements:.

Given a set of corresponding feature points, an example algorithm for estimating the matrix is described in the following computer vision book at <NPL>.

The above-described algorithm <NUM> follows. Given n ≥ <NUM>2D to 2D point correspondences {xi ↔x'i}, determine the 2D homography matrix H such that x'i = Hxi. (i) For each correspondence xi ↔ x'l compute the matrix Ai from (<NUM>). Only the first two rows need to be used in general. (ii) Assemble the n <NUM> x <NUM> matrices Ai into a single 2n x <NUM> matrix A. (iii) Obtain the SVD of A (section A4. <NUM>(p585)). The unit singular vector corresponding to the smallest singular value is the solution h. Specifically, if A = UDVT with D diagonal with positive diagonal entries, arranged in descending order down the diagonal, then h is the last column of V. (iv) The matrix H is determined from h as is (<NUM>).

The above-described algorithm <NUM> follows. Compute the 2D homography between two images. (i) Interest points: Compute interest points in each image. (ii) Putative correspondences: Compute a set of interest point matches based on proximity and similarity of their intensity neighbourhood. (iii) RANSAC robust estimation: Repeat for N samples, where N is determined adaptively as in algorithm <NUM>: (a) Select a random sample of <NUM> correspondences and compute the homography H. (b) Calculate the distance d⊥ for each putative correspondence. (c) Compute the number of inliers consistent with H by the number of correspondences for which d⊥ < t = √<NUM>σ pixels. Choose the H with the largest number of inliers. In the case of ties choose the solution that has the lowest standard deviation of inliers. (iv) Optimal estimation: re-estimate H from all correspondences classified as inliers, by minimizing the ML cost function. (v) Guided matching: Further interest point correspondences are now determined using the estimated H to define a search region about the transferred point position. The last two steps can be iterated until the number of correspondences is stable.

The above-described algorithm <NUM> follows. N = ∞, sample count= <NUM>. While N > sample count Repeat. - Choose a sample and count the number of inliers. - Set = <NUM> - (number of inliers)/(total number of points). - Set N from and (<NUM>) with p = <NUM>. - Increment the sample count by <NUM>.

At box <NUM>, the computing system smooths the current image. For example, the computing system may smooth the input image with a Gaussian filter to generate a smoothed input image (I_smoothed). Smoothing the input image can increase the robustness of the process, since downsampling and transforming the image can create aliasing artifacts or other noise, which the smoothing can remove or reduce. The computing system may store the smoothed image in a smoothed image buffer <NUM>, that stores smoothed images from previous iterations of this process on previous-received images. In this disclosure, description of operations that are performed on an image includes operations that are performed on either the image or a smoothed version of the image.

At box <NUM>, the computing system uses the transformation matrix to warp the previously-smoothed image into a new image (e.g., by warping I_smoothed_previous into I_smoothed_previous_warped). Doing so effectively shifts the location of the camera from when the previous image was taken so that it matches the location of the camera from when the current image was taken. As such, after the warping, the background, static portions of I_smoothed_previous_warped and I_smoothed may roughly match each other. This allows the computing system to compare the images to identify which portions of the image are non-background portions that are moving. The computing system can determine the coordinates for I_smoothed_previous_warped from the coordinates of I_smoothed_previous using H_interframe, as follows:.

For each pixel [x, y] in I_smoothed_previous, the computing system is able to determine the position [x', y'] in I_smoothed_previous_warped using the above transformation, and the computing system can copy the pixel value from [x, y] in I_smoothed_previous to [x', y'] in I_smoothed_previous_warped.

At box <NUM>, the computing system calculates the temporal gradient (e.g., the difference between pixels) between the current image and the warped version of the previous image. It may do this for each pixel as follows:.

The temporal gradient values may be further from zero the more change occurred at the location from one image to the next. As such, higher numbers (at least once the absolute value is taken) may identify portions of the image at which movement occurred.

At box <NUM>, the computing system additionally or alternatively calculates a rate of variation in one or more directions across the image (e.g., the spatial gradient). It may do so in the x direction as follows:.

It may do so in the y direction as follows:.

The rate of variation is greater if the pixel lies along an edge or border (e.g., because the pixel intensity may change more between the pixel on the left and the pixel on the right when the pixel lies along an edge or border, than if the pixel was located in a portion of the image without much variation. As such, higher numbers may identify edges.

At box <NUM>, the computing system computes a grid of points, from which a grid of patches may be generated. The computing system may calculate the grid p(i,j), with i=<NUM> → gridWidth and j=<NUM> → gridHeight. The calculation of the grid may exclude a margin at the edges of the image, for example, three percent of the image at the edges. The grid points may be evenly spaced, for example, <NUM> pixels apart along the x direction, and <NUM> pixels apart along the y direction. As an illustration, if the frame size is 320x180, the computing system may exclude <NUM> pixels on the left and right (<NUM>*<NUM>%=<NUM> pixels) and <NUM> pixels on the top and bottom (<NUM>*<NUM>%=<NUM> pixels). This provides a grid with a gridWidth=<NUM> and a gridHeight=<NUM>.

For each point p(i,j) in the grid, the computing system may identify a patch from I_smoothed that is based off a location of the point (e.g., the patch may be centered on the point) (box <NUM>). As an illustration, the patch may have a patchWidth of <NUM> and a patchHeight of <NUM>. The patches can overlap, be separated from each other, or be adjacent to and abut each other (e.g., like a checkerboard).

At box <NUM>, the computing system computes one or more statistics for each patch. These statistics may use the previously-calculated temporal and spatial gradients.

A first statistic that the computing system may calculate is an average of the horizontal rates of variation in the patch, for example, as follows:.

This calculation may multiply the horizontal spatial gradient values to emphasize the presence of vertical edges over smooth changes.

A second statistic that the computing system may calculate is an average of the vertical rates of variation in the patch, for example, as follows:.

This calculation may multiply the vertical spatial gradient values to emphasize the presence of horizontal edges over smooth changes.

A third statistic that the computing system may calculate is an average rate of the diagonal variations in the patch, for example, as follows:.

A fourth statistic that the computing system may calculate is a value that identifies vertical edges that are moving in the image, by combining the horizontal spatial gradient at a given position with the temporal gradient at that position, to generate a value that identifies if a vertical edge moved at that point, for example, as follows:.

A fifth statistic that the computing system may calculate is a value that identifies horizontal edges that are moving in the image, by combining the vertical spatial gradient at a given position with the temporal gradient at that position to generate a value that identifies if a horizontal edge moved at that point, for example, as follows:.

The computation of the statistics can be optimized by using integral images.

At box <NUM>, the computing system selects those patches that have texture (e.g., by ignoring those patches that may not have texture, and may just be a portion of the image that represents a blank wall). In other words, the computing system may determine if each patch has enough texture, and for those that do not, may set a motion score of "<NUM>" to the patch (box <NUM>). The process for selecting patches with texture can include identifying a Hessian 2x2 matrix of the patch:
{Ixx Ixy Ixy lyy}.

The computing system can determine the determinant of the matrix (det). The larger eigenvalue may be denoted max_eigenvalue and the smaller eigenvalue may be denoted min_eigenvalue. The computing system may select a patch as having texture if it satisfies the following conditions:.

The determinant may be greater than zero when the edges in the image have at least modest x and y components to them (e.g., the edges are not purely horizontal or purely vertical, in which case it may be difficult to identify motion in a horizontal or vertical direction, respectively).

This condition may guarantee that there are at least some edges in any given direction. EigenvalueThreshold is manually tuned and an example value may be <NUM>.

This condition may guarantee that edges in a dominant direction may not overwhelm edges in another direction. EigenvalueRatioThreshold is also manually tuned and an example value may be <NUM>. If a patch failed the above condition check, the computing system can set the motion vector for that patch to be motion_x = motion_y = <NUM>.

At box <NUM>, for each patch that is identified as having enough texture, the computing system estimates a motion of the patch (e.g., an object depicted by the pixels in the patch) by calculating a motion vector for the patch, for example, as follows:.

In some examples, the computing system applies the Lucas-Kanade differential method for optical flow estimation.

At box <NUM>, the computing system computes a motion score for each patch. The motion scores can be combined to generate a motion score map <NUM>. A map of motion scores may be calculated as follows:.

In this equation, motionParam may be manually set by a user, and may have a value of <NUM>. In some examples, the computing system may downsample the collection of scores (e.g., one score for each patch, with some having a <NUM> value) to a smaller motion score map: score_small(k,l), k=<NUM> → scoreWidth, l=<NUM> → scoreHeight (box <NUM>). An example interpolation method to downsample the score map is to average a window of multiple points to get one value. For example, to downsample by <NUM>, average every 3x3 window to get one pixel. As such, the computing system may end up with a 10x10 grid of scores rather than a 50x50 grid of scores. Description in this disclosure relating to a motion score map can refer to a motion score map or a downsampled version thereof.

At box <NUM>, the computing system calculates the entropy value of the score map, as follows:.

Epsilon may be a small number to avoid problems caused by <NUM>. The entropy value may identify the disorder in the image, which can illustrate a difference in movement throughout the image. For example, if all or most of the image is moving (e.g., because a camera focused on the side of a large truck that is pulling away), then there is not much disorder because all or most of the image is moving. On the other hand, there is a great deal of disorder and a high entropy if there are multiple people running around in an image, because many portions of the image are moving and many portions are not moving. Entropy may be large if motion is highly concentrated in a few portions of the image.

The computing system may use the generated entropy value to generate a motion saliency score. This score may identify an importance to the motion in the image. The motion_saliency_score may be a value between <NUM> and <NUM> that can be generated using the following nonlinear mapping function:.

The computing system outputs the motion saliency score <NUM> to inform another process or device how salient the motion is in the image. The computing system may also output the motion score map to inform another process or device where motion is occurring within a frame.

In the above description, a previously-received image is warped to match the camera position of a later-received image, and then various operations are performed on the later-received image, for example, calculations of the spatial gradient. Skilled artisans would understand that similar results could be achieved by applying the processes to the other of the two images. For example, the later-received image can be the image that is warped to match the position of the previously-received image, and subsequent operations such as the calculation of the spatial gradient could be performed on the previously-received image. Moreover, these operations (e.g., the spatial gradient) can be performed on the image that was warped, whether that is the previously-received image or the later-received image. As such, portions of this disclosure may refer to operations being performed on either the first image "or" the second image, to illustrate the various manners in which the motion estimation mechanisms may be performed.

In various implementations, operations that are performed "in response to" or "as a consequence of" another operation (e.g., a determination or an identification) are not performed if the prior operation is unsuccessful (e.g., if the determination was not performed). Operations that are performed "automatically" are operations that are performed without user intervention (e.g., intervening user input). Features in this document that are described with conditional language may describe implementations that are optional. In some examples, "transmitting" from a first device to a second device includes the first device placing data into a network for receipt by the second device, but may not include the second device receiving the data. Conversely, "receiving" from a first device may include receiving the data from a network, but may not include the first device transmitting the data.

"Determining" by a computing system can include the computing system requesting that another device perform the determination and supply the results to the computing system. Moreover, "displaying" or "presenting" by a computing system can include the computing system sending data for causing another device to display or present the referenced information.

<FIG> is a block diagram of computing devices <NUM>, <NUM> that may be used to implement the systems and methods described in this document, as either a client or as a server or plurality of servers. Computing device <NUM> is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Computing device <NUM> is intended to represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations described and/or claimed in this document.

The processor <NUM> can process instructions for execution within the computing device <NUM>, including instructions stored in the memory <NUM> or on the storage device <NUM> to display graphical information for a GUI on an external input/output device, such as display <NUM> coupled to high-speed interface <NUM>.

The high-speed controller <NUM> manages bandwidth-intensive operations for the computing device <NUM>, while the low speed controller <NUM> manages lower bandwidth-intensive operations. Such allocation of functions is an example only.

Additionally, the processor may be implemented using any of a number of architectures. For example, the processor may be a CISC (Complex Instruction Set Computers) processor, a RISC (Reduced Instruction Set Computer) processor, or a MISC (Minimal Instruction Set Computer) processor.

External interface <NUM> may provided, for example, for wired communication in some implementations, or for wireless communication in other implementations, and multiple interfaces may also be used.

The information carrier is a computer- or machine-readable medium, such as the memory <NUM>, expansion memory <NUM>, or memory on processor <NUM> that may be received, for example, over transceiver <NUM> or external interface <NUM>.

Additionally computing device <NUM> or <NUM> can include Universal Serial Bus (USB) flash drives. The USB flash drives may store operating systems and other applications. The USB flash drives can include input/output components, such as a wireless transmitter or USB connector that may be inserted into a USB port of another computing device.

Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), peer-to-peer networks (having ad-hoc or static members), grid computing infrastructures, and the Internet.

Claim 1:
A computer-implemented method for selectively storing images, comprising:
receiving, by a computing system, a first image that was captured by a camera;
receiving, by the computing system, a second image that was captured by the camera;
generating (<NUM>), by the computing system and using the first image and the second image, a mathematical transformation (<NUM>) that indicates movement of the camera from the first image to the second image with respect to a scene that is reflected in the first image and the second image;
generating (<NUM>), by the computing system and using the first image and the mathematical transformation, a modified version (<NUM>) of the first image that presents the scene that was captured by the first image from a position of the camera when the second image was captured, wherein a position of the camera when the first image was captured is different from the position of the camera when the second image was captured; and
determining, by the computing system, a portion of the first image or second image at which a position of an object in the scene moved, by comparing (<NUM>) the modified version of the first image to the second image,
the method further comprising:
receiving, by the computing system, a sequence of images that includes at least the first image and the second image, in addition to multiple other images;
determining, by the computing system, a level of movement (<NUM>) reflected by the first image or the second image based on the comparison of the modified version of the first image to the second image; and determining (<NUM>), by the computing system and based on the determined level of movement reflected by the first image or the second image, to:
(i) maintain the first image or the second image in computer storage, at least until user input removes the first image or the second image from the computer storage, and
(ii) remove at least one of the multiple other images from storage, without receipt of user input that specifies that the at least one of the multiple other images is to be removed from storage.