System and method of image upsampling

A method includes receiving an image having a first resolution and generating an upsampled image having a second resolution based on the image. A multi-dimensional data structure corresponding to a multi-dimensional image space is generated from the upsampled image. Each node of the data structure is determined based on a weighted sum of values of one or more pixels in the upsampled image. Each of the one or more pixels corresponds to a pixel in the received image and is located within a region of the image space having a vertex defined by the node. A filter modifies the values of the nodes and a second upsampled image is generated based on the modified values. Each pixel of the second upsampled image not corresponding to a pixel in the received image is determined based on a weighted sum of the modified values of one or more nodes.

FIELD OF THE DISCLOSURE

The present disclosure is generally related to image upsampling.

BACKGROUND

As high-resolution (e.g., high definition (HD)) displays become common, image/video upsampling has been used with such displays. For example, image/video upsampling may be used to convert lower-resolution (e.g., standard definition (SD)) images/video to higher resolution (e.g., HD) images/video. Typically, upsampling may be performed via interpolation. For example, given an original image of 100×100 pixel resolution, generating an upsampled image that has a resolution of 200×200 may include “spreading out” the 100×100 original pixels and determining interpolated values for “gap” pixels in the upsampled image.

In interpolation, a model used to describe the relationship between high-resolution pixels and low-resolution pixels may affect the performance of the interpolation. Typically, an interpolation algorithm determines the value of a pixel in the higher resolution image as a weighted average of the values in the neighborhood of the pixel in the lower resolution image. The weighted average scheme enables formulation of the interpolation algorithm using linear convolution techniques. However, such linear interpolation schemes that are based on space-invariant models may fail to capture the changing statistics around edges and may consequently produce interpolated images with blurred edges and annoying artifacts. Despite these drawbacks, linear interpolation remains popular due to its computational simplicity.

Blurring in the upsampled image may be removed via iterative feedback-controlled de-blurring. However, this type of de-blurring may be slow due to repeated application of a de-blurring operator and may therefore be unsuitable for real-time video upsampling.

DETAILED DESCRIPTION

A system and method to upsample image/video with improved output image quality and reduced computational complexity is disclosed. The disclosed techniques may utilize an adapted filtering framework that is sensitive to edges and other local image features and that uses a multi-dimensional lattice data structure for computation. The disclosed techniques may be used for real time or near-real time image upsampling (e.g., upsampling of SD video to HD video).

The disclosed techniques may provide various advantages over certain interpolation and other upsampling schemes. As used herein, the notation Iprepresents the value of an image at a pixel p. F[I] denotes the output of a filter applied to an image I. S denotes the set of pixel locations in the image (the spatial domain), while R refers to the set of image intensities (or range domain). In practical situations, due to quantization and digital imaging limitations, both S and R may be discrete sets. Further, |•| denotes the absolute value of a real number and ∥•∥ denotes the L2norm (or Euclidean distance).

An image filtered by a Gaussian Filter (denoted GF) may be represented by the expression:

Gaussian filtering is a weighted average of the intensity of the adjacent pixels with weights decreasing as the spatial distance decreases to a center pixel p. The weight corresponding to pixel q is defined by the Gaussian kernel Gσ(∥p-q∥) where σ is a parameter defining the neighborhood size.

Bilateral filtering is an example of an image processing algorithm that can be considered in the framework of high-dimensional Gaussian filtering. Bilateral filtering may attempt to preserve edges in image data as pixel values are smoothed. In bilateral filtering, for a first pixel to influence a second pixel, the first pixel should not only occupy a nearby spatial location but should also have a similar intensity value as the second pixel. The bilateral filter (BF) may be represented by the expression:

Parametric image processing methods typically rely on a specific signal model of interest and seek to compute parameters of the model in the presence of noise. In contrast, nonparametric methods rely on the data itself to dictate the structure of the model. Let f( ) denote a continuous function generating an image and assume that f is locally smooth to some order k. Given image information on L pixels (corresponding to a low-resolution image), if yiis the image intensity at a pixel xi=(x(i)i,x(2)i), then the known data can be expressed as
yi=f(xi)+ηi,1≦i≦L(Equation 2.1)
where ηidefines noise and is generated by independent and identically distributed zero mean random variables.

Based on the smoothness assumption for f, the function value at any point x can be estimated using a bivariate Taylor's series expansion around the point. Specifically:

For the case of k=2:
f(xi)=f(x)+{∇f(x)}T(xi−x)+½(xi−x)T{Hf(x)}(xi−x)+ . . .
f(xi)=α0+α1T(xi−x)+½α2Tvec((xi−x)(xi−x)T)+  (Equation 2.3),
where ∇ and H are gradient and Hessian operators, and vec( ) is the vectorization operator. The a terms can be estimated using a least squares formulation. Since this is a local estimate, nearby samples may be weighted more than farther away samples. Thus, the optimization problem may be defined as

Kσis a kernel function that penalizes samples far away (in the spatial domain) from the origin. This kernel function may be selected to satisfy a few conditions, such as K( ) is greater than or equal to zero everywhere, K( ) is unimodal with a maximum at the origin, K( ) is symmetric about the origin, and that the first and second moments of K( ) are zero and u, respectively. It will be appreciated that one example of such a kernel function is the Gaussian function Gσ(x). For upsampling, k=0, which leads to the a term:

However, the above approach, while penalizing samples farther away in the spatial domain, may not be sensitive to local image features. Thus, similar to interpolation methods, the above approach may suffer from blurring and other annoying artifacts.

Instead, if the kernel function is changed to not only depend on the spatial domain but also the range domain, the kernel function may adapt to local image features such as edges. Let σsbe the spatial variance of the kernel and σrbe the range variance. Given an image sample y=f(x), let the variable z denote (x/σs,y/σr) εS×R. The least squares formulation then becomes:

It will be appreciated that if the Gaussian function is used as the kernel, the above expression becomes the previously described bilateral filter BF (see Equation 1.3). Further, the subscript of K has changed from σ to 1, due to the transformation to z. As k changes, different expressions for the value of α0are obtained. This may be interpreted as a generalization of a bilateral filter.

It will also be noted that in the context of upsampling, the expression on the right-hand side of the equation has a dependence on z, and therefore y, inside the kernel function. However, because we are trying to estimate y using the weighted least squares optimization, this leads to a circular dependency. To break the circular dependency, an initial estimate of y may be obtained using another upsampling scheme, such as linear interpolation.

Accordingly, the present disclosure describes an upsampling system and method that generates an initial upsampled image from a low-resolution input image using an interpolator (e.g., a bi-cubic interpolator). To generate an improved upsampling, the initial upsampled image is then subjected to adaptive filtering. Adaptive filtering may include the steps of “splatting,” “filtering,” and “slicing” using a multi-dimensional lattice structure to generate a second upsampled image. The second upsampled image may be of higher quality than the first upsampled image.

During splatting, a first upsampled (e.g., interpolated) image may be used to populate a multi-dimensional lattice structure. Notably, only pixels in the first upsampled image that correspond to pixels from the original (e.g., low-resolution) image may be used during splatting. Each pixel from the original image may contribute (via barycentric weighting) to the nodes that define the vertices of a multi-dimensional region (i.e., a simplex) that the pixel belongs to. During filtering, a filter (e.g., a 1-2-1 filter) may be applied to blur (i.e., smooth) the values of the nodes in the lattice. Slicing may be considered the reverse of splatting. During slicing, the blurred values in the nodes may contribute via barycentric weights to the “gap” pixels in a second upsampled image. The second upsampled image may optionally be subjected to a single-iteration image deconvolution (i.e., de-blurring) algorithm.

The disclosed techniques may employ a data-adaptive kernel framework for image/video upsampling that is sensitive to fast changing image statistics around edges and other image features. The disclosed techniques may be implemented using a graphics processing unit (GPU) to achieve real time or near-real time performance rates of approximately 25 frames per second on 480 p video (progressive scan video with 480 lines of vertical resolution, usually 640×480 pixels in 4:3 aspect ratio material and 854×480 pixels in 16:9 aspect ratio material).

In a particular embodiment, a method includes receiving an image having a first resolution and generating a first upsampled image based on the image, where the first upsampled image has a second resolution that is greater than the first resolution. The method also includes generating a multi-dimensional data structure corresponding to a multi-dimensional image space based on the first upsampled image. Each node of the multi-dimensional data structure has a value that is determined based on a weighted sum of values of one or more pixels in the first upsampled image, where each of the one or more pixels corresponds to a pixel in the received image and is located within a region of the multi-dimensional image space that has a vertex defined by the node. For example, each contributing pixel that is located within a particular multi-dimensional simplex may contribute to the nodes of the multi-dimensional data structure that make up the vertices of the particular multi-dimensional simplex. The method further includes applying a filter to modify the values of the nodes of the multi-dimensional data structure and generating a second upsampled image based on the modified values of the nodes in the multi-dimensional data structure. Each pixel of the second upsampled image that does not correspond to a pixel in the received image has a pixel value that is determined based on a weighted sum of the modified values of one or more nodes of the multi-dimensional data structure.

In another particular embodiment, a system includes a processor and an interpolator executable by the processor to generate a first upsampled image based on a received image. The received image has a first resolution and the first upsampled image has a second resolution that is greater than the first resolution. The apparatus also includes a splatter that is executable by the processor to generate a multi-dimensional data structure corresponding to a multi-dimensional image space based on the first upsampled image. Each node of the multi-dimensional data structure has a value that is determined based on a weighted sum of values of one or more pixels in the first upsampled image, where each of the one or more pixels corresponds to a pixel in the received image and is located within a region of the multi-dimensional image space that has a vertex defined by the node. For example, each contributing pixel that is located within a particular multi-dimensional simplex may contribute to the nodes of the multi-dimensional data structure that make up the vertices of the particular multi-dimensional simplex. The apparatus further includes a filter that is executable by the processor to modify the values of the nodes of the multi-dimensional data structure and a slicer that is executable by the processor to generate a second upsampled image based on the modified values of the nodes in the multi-dimensional data structure. Each pixel of the second upsampled image that does not correspond to a pixel in the received image is determined based on a weighted sum of the modified values of one or more nodes of the multi-dimensional data structure.

In another particular embodiment, a computer-readable storage medium includes instructions that, when executed by a processor, cause the processor to receive an image having a first resolution and to generate a first upsampled image based on an interpolation of the image. The first upsampled image has a second resolution that is greater than the first resolution by an integer multiple. The instructions are also executable to cause the processor to generate a five-dimensional (5-D) data structure corresponding to a 5-D image space based on the first upsampled image. The 5-D image space includes a first dimension along a first positional axis, a second dimension along a second positional axis, a third dimension corresponding to a first color, a fourth dimension corresponding to a second color, and a fifth dimension corresponding to a third color. Each node of the 5-D data structure has a value that is determined based on a weighted sum of values of one or more pixels in the first upsampled image, where each of the one or more pixels corresponds to a pixel in the received image and is located within a 5-D simplex in the 5-D image space that has a vertex defined by the node. For example, each contributing pixel that is located within a particular simplex may contribute to the nodes of the 5-D data structure that make up the vertices of the particular simplex. The instructions are further executable to cause the processor to apply a filter to modify the values of the nodes of the 5-D data structure and to generate a second upsampled image based on the modified values of the nodes in the 5-D data structure. Each pixel of the second upsampled image that does not correspond to a pixel in the received image is determined based on a weighted sum of the modified values of one or more nodes of the 5-D data structure.

Referring toFIG. 1, a block diagram of a particular embodiment of a system100that is operable to upsample an image is illustrated. In particular embodiments, components of the system100may be implemented in hardware and/or as instructions executable by a processor, such as a dedicated graphics processing unit (GPU) or other processor.

The system100includes an interpolator110that receives an original image101as input. In a particular embodiment, the original image101is represented by digital data. The original image101may be a relatively low-resolution image, such as a frame of standard definition (SD) video or 480 p video. The interpolator110may interpolate the original image101to generate a first upsampled image102. For example, the applied interpolation may be bi-cubic interpolation, linear interpolation, and/or Gaussian interpolation. The first upsampled image102may have a resolution that is greater than the resolution of the original image101. In a particular embodiment, the first upsampled image102may have a resolution that is greater than the resolution of the original image101by an integer multiple (i.e., integer scaling factor), as further described with reference toFIG. 2.

The system100may further include a splatter120configured to generate a multi-dimensional data structure (e.g., a multi-dimensional lattice122) based on the first upsampled image102. The lattice122may be a permutohedral lattice corresponding to a multi-dimensional image space. In a particular embodiment, the lattice122may be a five-dimensional (5-D) lattice, including two spatial dimensions along two positional axes (e.g., x and y) and three intensity dimensions (e.g., corresponding to red, green, and blue colors).

The splatter120may not use all pixels from the first upsampled image102when generating the lattice122. Instead, the splatter120may only use pixels in the first upsampled image102that correspond to pixels in the original image101. For example, let the original image101be represented by I, the first upsampled image102be represented by Iu, and the first upsampled image102be larger than the original image101by an integer scaling factor s. Because s is an integer, the (sx,sy) pixels in the first upsampled image102may be substantially identical in r,g,b value to the corresponding (x,y) pixels from the original image101. That is Iu(sx,sy)=I(x,y). The splatter120may only use such pixels located at (sx,sy) in the first upsampled image102, which correspond to pixels (x,y) in the original image101, to generate the lattice122. Each of such pixels in the first upsampled image102may contribute to nodes of the lattice122via barycentric weighting, the nodes defining vertices of a region (e.g., simplex) of the image space that the pixel belongs to, as further described with reference toFIG. 3.

The system100may further include a filter130configured to modify node values determined by the splatter120, thereby transforming the lattice122into a modified (e.g., blurred) lattice132. In a particular embodiment, the filter130may be a 1-2-1 filter, as further described with reference toFIG. 4.

The system100includes a slicer140configured to generate a second upsampled image103based on the modified (e.g., blurred) values of the nodes in the modified lattice132. Operation of the slicer140may be considered as a reverse of the operation of the splatter120. That is, each pixel in the second upsampled image103that does not correspond to a pixel in the original image101may have a pixel value that is determined based on a weighted sum of one or more modified values from the modified lattice132. A particular example of operation of the slicer140is further described with reference toFIG. 5. The resulting second upsampled image103may be a more accurate upsampling of the original image101than the first upsampled image102.

In a particular embodiment, to further improve and refine the second upsampled image103, the system100may include an image deconvolver150for de-blurring. Two-dimensional image deconvolution can be used for image de-blurring, de-noising, and restoration. In a particular embodiment, the image deconvolver150is configured to apply an iterative, non-blind deconvolution technique using a hyper-Laplacian image prior (e.g., distribution function). The image deconvolver150may use an alternating minimization scheme that alternates between two sub-problems. One of the sub-problems may use a standard L2minimization solved in closed form in the Fourier domain. The other problem, which may be a non-convex problem that is separable over pixels, may be solved by polynomial root finding, where roots may be pre-computed and stored in a lookup table. For example, roots may be stored in a lookup table on-board or accessible to a graphics processing unit (GPU). In a particular embodiment, the image deconvolver150may apply a single iteration of the image deconvolution technique to the second upsampled image103to generate a de-blurred upsampled image104.

The system100may thus provide an adapted filtering framework for image upsampling using a multi-dimensional lattice (e.g., permutohedral lattice). Advantageously, the number of nodes in the lattice may grow linearly, not exponentially, as a function of the number of dimensions. Thus, the lattice may be well-suited for real time or near-real time computations in all five dimensions (two dimensions for space and three dimensions for color). The system100ofFIG. 1may be used to upsample low-resolution SD video to generate high-definition (HD) video. In a particular embodiment, the system100may use spatial variance σs=5.0 and range variance σr=0.06 (assuming color values are normalized to [0,1]).

FIGS. 2-5illustrate particular examples of operation of various components of the system100ofFIG. 1. For example,FIG. 2illustrates a particular example of operation of the interpolator110ofFIG. 1, and is generally designated200. As shown inFIG. 2, the interpolator may receive an original image201(e.g., corresponding to the original image101ofFIG. 1) and may produce a first upsampled image202(e.g., corresponding to the first upsampled image102ofFIG. 1). In a particular embodiment, the interpolator is a bi-cubic, linear, or Gaussian interpolator that interpolates the original image201by an integer scaling factor s.

The original image201may include a plurality of pixels. Each pixel211in the original image (illustrated in black inFIG. 2) has coordinates (x,y). During interpolation, the pixels from the original image may be “spread out,” and “gap” pixels (illustrated in gray inFIG. 2) may be computed. For example, as shown inFIG. 2, the first upsampled image202may include a pixel221corresponding to the pixel211from the original image201, as well as gap pixels222.

FIG. 3illustrates a particular embodiment of operation of the splatter120ofFIG. 1and is generally designated300. As described with reference toFIG. 1, the splatter120may generate a multi-dimensional (e.g., 5-D) lattice data structure based on an interpolated image. For ease of illustration,FIG. 3depicts the lattice structure including white nodes as an overlay on top of the interpolated image. Dark gray pixels in the interpolated image correspond to pixels in an original input image, and light gray pixels correspond to gap pixels. This illustration convention is also used inFIGS. 4-5.

During splatting, each pixel in the interpolated image (e.g., the first upsampled image102ofFIG. 1or the first upsampled image202ofFIG. 2) that corresponds to a pixel in the original image (e.g., the original image101ofFIG. 1or the original image201ofFIG. 2) may contribute its value via barycentric weights to the nodes of the lattice that define vertices of the multi-dimensional simplex that the pixel belongs to. For example, inFIG. 3, nodes301-303define vertices of a first simplex (shown as a triangle for ease of illustration) that includes the pixel310, and nodes303-305define vertices of a second simplex that includes the pixel320. Based on its “distance” (as measured in 5-D) from the nodes301-303, the pixel310makes contributions barycentrically weighted at ⅙, ⅙, and ⅓ to the nodes301,302, and303, respectively. The pixel320makes a contribution weighted at ⅓ to each of the nodes303,304, and305. The sum of contribution weights for contributing pixels310,320may be equal to 1.

It will be appreciated that by considering distance in spatial as well as color dimensions, splatting as illustrated inFIG. 3may adapt to local image features such as edges. For example, even though two pixels at (x,y) coordinates (0,0) and (0,1) may be adjacent in the spatial domain, if the pixels define a black-white edge, the pixels would not be adjacent in the intensity domain, because the 5-D representations of the pixels may be (0,0,0,0,0) and (0,0,255,255,255), respectively. Thus, the two pixels may not be located near each other in the 5-D lattice and may not contribute to the same nodes of the 5-D lattice, which may reduce blurring across the black-white edge in an upsampled image.

FIG. 4illustrates a particular embodiment of operation of the filter130ofFIG. 1and is generally designated400. In the particular embodiment ofFIG. 4, the filter is a 1-2-1 filter that is applied in each dimension of the multi-dimensional (e.g., 5-D) lattice.

Filtering may modify (also known as “blurring”) the values of nodes in the lattice. In accordance with a 1-2-1 filter, each neighboring node of a particular node may make a contribution weighted at 1 to the particular node, and the particular node may make a contribution weighted at 2 to itself For example, inFIG. 4, each of the nodes401-406make equally weighted contributions to the node410, and the node410contributes twice as much to itself The 1-2-1 filter illustrated inFIG. 4may thus modify the values of the nodes in the lattice, which may lead to a more accurate upsampled image when the nodes are used to compute gap pixels, as described with reference toFIG. 5.

FIG. 5illustrates a particular embodiment of operation of the slicer140ofFIG. 1and is generally designated500. During slicing, which may be considered the reverse of splatting, the nodes of the 5-D lattice are used to compute gap pixels (light gray inFIG. 5) in a second upsampled image (e.g., the second upsampled image103ofFIG. 1). Each gap pixel is determined based on a weighted sum of the modified (e.g., blurred) values of the nodes that “surround” the pixel. For example, as shown inFIG. 5, the value of the pixel510may be determined based on contributions from the nodes501,502, and503of the lattice, weighted at 1/10, ½, and ⅖, respectively. Thus, slicing as illustrated inFIG. 5may “query” the nodes of the lattice to determine the value of the gap pixels in an upsampled image.

FIG. 6is a diagram to illustrate particular examples of input and output images of the system100ofFIG. 1. An original image601(e.g., corresponding to the original image101ofFIG. 1or the original image201ofFIG. 2) represents a frame of a video showing fish swimming underwater. A first upsampled image602(e.g., corresponding to the first upsampled image102ofFIG. 1or the first upsampled image202ofFIG. 2) is generated via bi-cubic interpolation of the original image601.

A second upsampled image603(e.g., corresponding to the second upsampled image103ofFIG. 1) is generated by performing splatting, filtering, and slicing (e.g., as described with reference toFIGS. 3-5) on the first upsampled image602. It will be appreciated that the second upsampled image603appears to be less blurry and contain more detail than the first upsampled image602.

Further, as described with reference to the image deconvolver150ofFIG. 1, the second upsampled image603may optionally be subject to a single iteration of an image deconvolution technique, thereby generating a de-blurred upsampled image604(e.g., corresponding to the de-blurred upsampled image104ofFIG. 1). It will be appreciated that the de-blurred upsampled image604appears to have sharper edges than the first upsampled image602and the second upsampled image603.

FIG. 7is a flowchart to illustrate a particular embodiment of a method700of upsampling an image. In an illustrative embodiment, the method700may be performed by the system100ofFIG. 1and may be illustrated with reference toFIGS. 2-5.

The method700may include receiving an image having a first resolution, at702, and generating a first upsampled image based on the image, at704. The first upsampled image may be generated by bi-cubic, linear, or Gaussian interpolation of the received image. The first upsampled image may have a second resolution that is greater than the first resolution by an integer multiple. For example, inFIG. 1, the interpolator110may receive the original image101and may generate the first upsampled image102, where the first upsampled image102is larger than (i.e., has a higher resolution than) the original image101by an integer multiple. In an illustrative embodiment, the interpolator110may perform interpolation by “spreading out” original pixels and computing gap pixels, as described with reference toFIG. 2.

Advancing to706, a multi-dimensional data structure corresponding to a multi-dimensional image space may be generated based on the first upsampled image. In a particular embodiment, the data structure may be a 5-D lattice having x (x-axis position), y (y-axis position), r (red), g (green), and b (blue) dimensions. Each node of the multi-dimensional data structure may have a value that is determined based on a weighted sum of values of one or more pixels in the first upsampled image, where each of the one or more pixels corresponds to a pixel in the received image and is located within a region of the multi-dimensional image space that has a vertex defined by the node. For example, inFIG. 1, the splatter120may generate the lattice122based on the first upsampled image102. In an illustrative embodiment, pixels in the first upsampled image corresponding to pixels in the original image may contribute to “surrounding” nodes of the lattice via barycentric weighting, as described with reference toFIG. 3.

Proceeding to708, a filter may be applied to modify the values of the nodes in the multi-dimensional data structure. For example, inFIG. 1, the filter130may modify the values of the nodes in the lattice122, thereby generating the modified lattice132. In an illustrative embodiment, the filter is a 1-2-1 filter, as described with reference toFIG. 4.

Continuing to710, a second upsampled image may be generated based on the modified values of the nodes in the multi-dimensional data structure. Each pixel of the second upsampled image that does not correspond to a pixel in the received image is determined based on a weighted sum of one or more modified values of one or more nodes of the multi-dimensional data structure. For example, inFIG. 1, the slicer140may generate the second upsampled image103based on the modified values of the nodes in the modified lattice132. In an illustrative embodiment, slicing may be performed via a reverse splatting operation in which gap pixels of the second upsampled image receive weighted contributions from “surrounding” nodes of the lattice, as described with reference toFIG. 5.

Advancing to712, the second upsampled image may be de-blurred by executing a single iteration of an image deconvolution technique. For example, inFIG. 1, the image deconvolver150may de-blur the second upsampled image103via a single iteration of an image deconvolution technique, thereby generating the de-blurred upsampled image104.

The method700may thus upsample a low-resolution image into a high-resolution image. In a particular embodiment, the low resolution image is a frame of SD video and the high-resolution image is a frame of HD video that may be output to a display device such as an HD television or monitor. In such an embodiment, the method700may be repeated for remaining video frames, at714. For example, the received image may correspond to a first video frame and the method700may include receiving a second image having the first resolution, where the second image corresponds to a second video frame. A third upsampled image having the second resolution may be generated based on the second image, and a second multi-dimensional data structure may be generated based on the third upsampled image. The filter may be applied to blur values of nodes in the second multi-dimensional data structure and a fourth upsampled image may be generated based at least in part on the blurred values of the nodes in the second multi-dimensional data structure. The method700may repeat for each successive frame of input video.

Referring toFIG. 8, an illustrative embodiment of a general computer system is shown and is designated800. For example, the computer system800may include, implement, or be implemented by one or more components of the system100ofFIG. 1. The computer system800includes or has access to a set of instructions that can be executed to cause the computer system800to perform any one or more of the methods and computer-based and/or processor-based functions disclosed herein. The computer system800, or any portion thereof, may operate as a standalone device or may be connected, e.g., using a network, to other computer systems or peripheral devices.

In a networked deployment, the computer system800may operate in the capacity of a set-top box device, a personal computing device, a mobile computing device, or some other computing device. The computer system800can also be implemented as or incorporated into various devices, such as a personal computer (PC), a tablet PC, a personal digital assistant (PDA), a mobile device, a palmtop computer, a laptop computer, a desktop computer, a communications device, a web appliance, a television or other display device, or any other machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine. In a particular embodiment, the computer system800can be implemented using electronic devices that provide voice, video, or data communication. Further, while a single computer system800is illustrated, the term “system” shall also be taken to include any collection of systems or sub-systems that individually or jointly execute a set, or multiple sets, of instructions to perform one or more computer functions.

As illustrated inFIG. 8, the computer system800may include a processor802, e.g., a central processing unit (CPU). The computer system800may also include a graphics processing unit (GPU)803(e.g., a dedicated graphics processor). In a particular embodiment, the GPU803may include hardware corresponding to components of the system100ofFIG. 1and/or execute instructions corresponding to components of the system100ofFIG. 1. The GPU803may also be operable to perform processes and methods disclosed herein, such as the method700ofFIG. 7. In a particular embodiment, the GPU803may execute instructions via compute unified device architecture (CUDA), a parallel graphics computing platform that enables programming instructions in languages such as C, C++, and FORTRAN to be issued to a GPU. In one implementation, the GPU803may achieve real time or near-real time upsampling performance at rates of approximately 25 frames per second on 480 p input video.

Moreover, the computer system800can include a main memory804and a static memory806that can communicate with each other via a bus808. As shown, the computer system800may further include or be coupled to a video display unit810, such as a liquid crystal display (LCD), an organic light emitting diode (OLED), a flat panel display, a solid-state display, or a projection display. For example, the video display unit810may be an HD television or monitor. Additionally, the computer system800may include an input device812, such as a keyboard, a remote control device, and a cursor control device814, such as a mouse. In a particular embodiment, the cursor control device814may be incorporated into a remote control device such as a television or set-top box remote control device. The computer system800can also include a disk drive unit816, a signal generation device818, such as a speaker or remote control device, and a network interface device820. The network interface device820may be coupled to other devices (not shown) via a network826.

In a particular embodiment, as depicted inFIG. 8, the disk drive unit816may include a computer-readable non-transitory medium822in which one or more sets of instructions824, e.g. software, can be embedded. Further, the instructions824may embody one or more of the methods or logic as described herein. In a particular embodiment, the instructions824may reside completely, or at least partially, within the main memory804, the static memory806, and/or within the processor802and/or the GPU803during execution by the computer system800. The main memory804, the processor802, and the GPU803also may include (e.g., on-board) computer-readable non-transitory media.

The present disclosure contemplates a computer-readable non-transitory medium that includes instructions824so that a device connected to a network826can communicate voice, video, or data over the network826. Further, the instructions824may be transmitted or received over the network826via the network interface device820(e.g., via uploading and/or downloading of an image upsampling application or program).

While the computer-readable non-transitory medium is shown to be a single medium, the term “computer-readable medium” includes a single medium or multiple media, such as a centralized or distributed database, and/or associated caches and servers that store one or more sets of instructions. The term “non-transitory computer-readable medium” shall also include any medium that is capable of storing a set of instructions for execution by a processor or that cause a computer system to perform any one or more of the methods or operations disclosed herein.

In a particular non-limiting, exemplary embodiment, the computer-readable non-transitory medium can include a solid-state memory such as a memory card or other package that houses one or more non-volatile read-only memories. Further, the computer-readable non-transitory medium can be a random access memory or other volatile re-writable memory. Additionally, the computer-readable non-transitory medium can include a magneto-optical or optical medium, such as a disk or tapes. Accordingly, the disclosure is considered to include any one or more of a computer-readable non-transitory storage medium and successor media, in which data or instructions may be stored.

It should also be noted that software that implements the disclosed methods may optionally be stored on a tangible storage medium, such as: a magnetic medium, such as a disk or tape; a magneto-optical or optical medium, such as a disk; or a solid state medium, such as a memory card or other package that houses one or more read-only (non-volatile) memories, random access memories, or other re-writable (volatile) memories.