Patent Description:
Flaws in sensors used to acquire source video content associated with a media title typical result in defective pixels being generated in the source video content. For instance, if a charge-coupled device or complementary metal-oxide-semiconductor image sensor included in a video camera were unable to properly sense intensities, then the source video content could include a dead pixel or a stuck pixel. As used herein, a "dead pixel" is associated with a black color; whereas, a "stuck pixel" is associated with an invariant color. Visual anomalies resulting from such defective pixels (i.e., dead or stuck pixels) being included in source video content are oftentimes exacerbated by downstream transcoding operations that are used to translate the source video content into different versions of target video content that are tailored for a variety of different display devices. These visual anomalies can negatively impact the overall viewing experience. More specifically, during transcoding, each defective pixel can corrupt neighboring pixels, thereby creating multi-pixel regions of visual anomalies in the target video content. During playback of that target video content, the attention of a viewer may be diverted to those multi-pixel regions of visual anomalies, thereby degrading the viewing experience for that viewer.

In an effort to avoid scenarios like those described above, many video content distributors attempt to detect defective pixels included in source video content via post-production analyses. The video content distributors then perform remedial actions to mitigate or fix the defective pixels prior to generating the target video content. For example, a video content distributor could modify the intensities of a defective pixel based on the intensities of neighboring pixels and/or notify the provider of the source video content that the associated video camera is defective.

In one conventional approach for detecting defective pixels, a quality control person plays back source video content multiple times and attempts to manually identify any defective pixels included in the source video content. In a typical setup, the quality control person views the source video content from beginning to end repeatedly on a large, high-resolution display in a very dark room. During the repeated viewings, the quality control person scrutinizes the display in an attempt to manually identify any visual anomalies.

One drawback of the above approach to detecting defective pixels is that viewing and analyzing source video content multiple times over is time-consuming and labor intensive. For example, if the source video content makes up a typical feature film "A" having a playback duration of <NUM> minutes, and the quality control person were to view the entire film "A" six times, then the quality control person would spend <NUM> hours viewing the source video content Further, manually identifying defective pixels is extremely difficult and error-prone. For example, if the quality control person were to view the film "A" on a <NUM> television, and the film "A" has <NUM>,<NUM> frames, then the quality control person would have to scrutinize over eight million pixels during each of <NUM>,<NUM> different frames. As a result, the quality control person likely would overlook at least some defective pixels.

Certain conventional quality control tools exist that may be used to reduce the manual effort required to detect defective pixels. However, these types of quality control tools are error-prone and usually require manual intervention in order to finalize the results. In particular, a typical quality control tool usually erroneously identifies many non-defective pixels as defective. A quality control person then needs to manually review each of the identified "defective" pixels to determine which of the pixels are actually defective. Even with manual intervention, a finalized list of defective pixels may omit some defective pixels and include some non-defective pixels.

<CIT> discloses an image processing device including a defective pixel estimation unit that estimates a defect of a pixel of interest in an image captured by a solid-state imaging device that contains a two-dimensional array of color pixels, and high-sensitivity pixels having higher sensitivity than sensitivities of the color pixels. The defect of the pixel of interest is estimated on the basis of a correlation between pixel values of the high-sensitivity pixels, and pixel values of the color pixels. The image processing device further includes a defective pixel correction unit that corrects the pixel of interest when it is estimated that the pixel of interest is a defective pixel.

As the foregoing illustrates, what is needed in the art are more effective techniques for detecting defective pixels in source video content.

Particular and preferred aspects of the present invention are set out in the appended claims.

At least one technical advantage of the disclosed techniques relative to the prior art is that, with the disclosed techniques, both the time and the amount of manual effort required to detect defective pixels in video content is reduced. In particular, the disclosed techniques automatically detect defective pixels in a high-throughput fashion using a distributed execution framework. In addition, the disclosed techniques implement high-level statistics-based spatial filtering and temporal consistency filtering to increase the accuracy of detection relative to prior art solutions. These technical advantages provide one or more technological advancements over the prior art.

So that the manner in which the above recited features of the various embodiments can be understood in detail, a more particular description of the inventive concepts, briefly summarized above, may be had by reference to various embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments of the inventive concepts and are therefore not to be considered limiting of scope in any way, and that there are other equally effective embodiments.

In the following description, numerous specific details are set forth to provide a more thorough understanding of the various embodiments. However, it will be apparent to one of skilled in the art that the inventive concepts may be practiced without one or more of these specific details.

The overall visual experience that a video streaming service provides to customers depends on the degree to which source video content enables accurate playback of an associated media title. However, flaws in sensors used to acquire the source video content can result in defective pixels being generated in the source video content. The defective pixels are oftentimes associated with visual anomalies that can negatively impact the overall viewing experience. For example, during playback of a media title, a dead pixel in the source video content could appear as a black speck in a blue lake. Therefore, being able to detect and mitigate defective pixels in source video content is an important factor in the ability of the video streaming service to provide high-quality viewing experiences to customers.

Prior art techniques for detecting defective pixels are labor-intensive, prohibitively time-consuming, and/or not necessarily reliable. For instance, in one prior art technique, a quality control person views a media title from beginning to end repeatedly under controlled viewing conditions. During the repeated viewings, the quality control person scrutinizes the display in an attempt to manually identify any visual anomalies. If the video streaming service were to provide access to <NUM>,<NUM> media titles having an average playback duration of <NUM> minutes, and quality control people were to view each media title <NUM> times, then the total number of person-hours required to screen the media titles for defective pixels would be <NUM>,<NUM> hours - over <NUM> person-months. Further, manually identifying defective pixels within the millions of pixels that would typically be displayed at any given time during each viewing is prohibitively difficult. Consequently, the quality control people likely would overlook at least some defective pixels. Other prior-art techniques involve using quality control tools to automatically detect defective pixels. However, these type of quality control tools are error-prone and usually require manual intervention to determine which of the automatically detected "defective" pixels are actually defective.

With the disclosed techniques, however, a video streaming service can automatically and reliably generate a list of defective pixels included in source video content, where the list is ranked by candidate frequencies across the source video content. The "candidate frequency" for a given pixel correlates to a total number of frames in which the pixel exhibits the types of visual defects typically associated with defective pixels. The video streaming service can then modify the defective pixels to more closely match neighboring pixels and/or inform the provider of the source video content that the video camera that acquired the source video content is defective.

In various embodiments, a defective pixel detection application uses distributed execution techniques, outlier detection techniques, and filtering techniques to efficiently and reliably detect defective pixels in source video content. First, the defective pixel detection application partitions the source video content into multiple, equal-sized clips. For each clip, the defective pixel detection application uses cloud resources to concurrently generate a different candidate frequency table. To generate the candidate frequency table for a given clip, the defective pixel detection application identifies potentially defective pixels for each frame. More specifically, for each frame, the defective pixel detection application performs locally adaptive contrast enhancement operations to improve the contrast of any defective pixels. The defective pixel application then executes outlier detection algorithms on the resulting enhanced frame to generate an initial candidate list of potentially defective pixels.

For each enhanced frame, the defective pixel application executes high-level, statistics-based spatial filtering operations on the associated initial candidate list to generate an associated refined candidate list. During the spatial filtering operations, if the defective pixel detection application determines that a candidate pixel is associated with abrupt changes in intensity gradient that are characteristic of defective pixels, then the defective pixel detection application adds the candidate pixel to the refined candidate list. Otherwise, the defective pixel detection application omits the candidate pixel from the refined candidate list. The defective pixel detection application then computes a candidate frequency across the clip for each candidate pixel that appears in one or more of the refined candidate lists. If the candidate frequency associated with a particular candidate pixel exceeds a frequency threshold, then the defective pixel detection application adds the candidate pixel to the candidate frequency table. Otherwise, the defective pixel detection application omits the candidate pixel from the candidate frequency table. Finally, the defective pixel detection application consolidates the candidate frequency tables associated with the different clips to generate a defective pixel list ordered by decreasing overall candidate frequency across the video content.

At least one technical advantage of the disclosed techniques relative to prior art solutions is that, with the disclosed techniques, the defective pixel detection application more efficiently and reliably identifies defective pixels in the source video content. In particular, detecting defective pixels does not involve the labor-intensive and time-consuming activities associated with prior art techniques for detecting defective pixels. Further, by performing both spatial and temporal filtering operations based on an initial list of outliers, the disclosed techniques increase the accuracy of defective pixel detection relative to prior art solutions. These technical advantages provide one or more technological advancements over the prior art.

<FIG> is a conceptual illustration of a system <NUM> configured to implement one or more aspects of the present invention. As shown, the system <NUM> includes, without limitation, a secure storage <NUM> and any number of compute instances <NUM>. For explanatory purposes, multiple instances of like objects are denoted with reference numbers identifying the object and parenthetical numbers identifying the instance where needed. In various embodiments, any number of the components of the system <NUM> may be distributed across multiple geographic locations or implemented in one or more cloud computing environments (i.e., encapsulated shared resources, software, data, etc.) in any combination.

The secure storage <NUM> includes any amount and type of memory resources that are capable of securely storing data. For instance, in some embodiments, the secure storage <NUM> is managed by a secure storage subsystem and stores data in an encrypted form. In various embodiments, the secure storage <NUM> may be replaced by any type of storage that is implemented in any technically feasible fashion. For instance, the secure storage <NUM> could be replaced by any amount and type of cloud-based memory resources that store any amount and type of data in any form. In alternative embodiments, the system <NUM> may include any amount and type of secure storage <NUM>.

As shown for the compute instance <NUM>(<NUM>), each compute instance <NUM> includes, without limitation, a processor <NUM> and a memory <NUM>. In alternative embodiments, each compute instance <NUM> may include any number of processors <NUM> and any number of memories <NUM>. The processor <NUM> may be any instruction execution system, apparatus, or device capable of executing instructions. For example, the processor <NUM> could comprise a central processing unit (CPU), a graphics processing unit (GPU), a controller, a microcontroller, a state machine, or any combination thereof. The memory <NUM> stores content, such as software applications and data, for use by the processor <NUM> of the compute instance <NUM>.

The memory <NUM> may be one or more of a readily available memory, such as random access memory (RAM), read only memory (ROM), floppy disk, hard disk, or any other form of digital storage, local or remote. In some embodiments, a storage (not shown) may supplement or replace the memory <NUM>. The storage may include any number and type of external memories that are accessible to the processor <NUM>. For example, and without limitation, the storage may include a Secure Digital Card, an external Flash memory, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

Each compute instance <NUM> is configured to implement one or more applications or subsystems of applications. For explanatory purposes only, each application is depicted as residing in the memory <NUM> of a single compute instance <NUM> and executing on a processor <NUM> of the single compute instance <NUM>. However, as persons skilled in the art will recognize, the functionality of each application may be distributed across any number of other applications that reside in the memories <NUM> of any number of compute instances <NUM> and execute on the processors <NUM> of any number of compute instances <NUM> in any combination. Further, the functionality of any number of applications or subsystems may be consolidated into a single application or subsystem.

In particular, the compute instance <NUM>(<NUM>) enables the system <NUM> to detect defective pixels in video content <NUM> in a high-throughput fashion. The video content <NUM> may be in any video format and includes, without limitation, any number of frames. Each frame includes, without limitation, any number of pixels. Each pixel is associated with a physical location relative to the other pixels and is characterized by an intensity that can differ between frames. The intensity includes, without limitation, three channel values, where each channel value is associated with a different channel of an RGB color space. In alternative embodiments, each pixel may be associated with any number of channels and include any number of channel values. Further, the number of channel values may vary based on the channel. In the same or other alternative embodiments, the video content <NUM> may be represented in any color space and the techniques described herein are modified accordingly. Examples of other color spaces include, without limitation, YCbCr, Y'CbCr, YUV, YIQ, YPbPr, and ICtCp, to name a few.

In general, the video content <NUM> is associated with a media title (not shown). Some examples of media titles include, without limitation, a feature-length movie, an episode of a television show, and a television advertisement. As referred to herein, a "defective pixel" in the video content <NUM> is caused by a flawed sensor included in a video camera during the process of acquiring the video content <NUM>. A flawed sensor is unable to properly sense intensities. For example, a flawed sensor that is unable to sense any intensities results in a dead pixel being included in the video content <NUM>, whereas a flawed sensor that is unable to sense blue or green intensities results in a red "stuck pixel" being included in the video content <NUM>.

The video content <NUM> may be source video content that is acquired directly by video cameras or target video content derived from source video content. Oftentimes, target video content is generated from source video content during downstream transcoding operations that are used to translate the source video content into different versions of target video content that are tailored for a variety of different display devices. Notably, visual anomalies resulting from defective pixels being included in source video content are oftentimes exacerbated in a domino fashion by the transcoding operations More specifically, during transcoding, each defective pixel can corrupt neighboring pixels, creating multi-pixel regions of visual anomalies in the target video content. During playback of that target video content, the attention of a viewer may be diverted to those multi-pixel regions of visual anomalies, thereby degrading the viewing experience for that viewer.

In an effort to avoid scenarios like those described above, many video content distributors attempt to detect defective pixels included in the video content <NUM> to guide remedial actions. For example, a video streaming service could attempt to detect defective pixels included in the source video content for a media title. The video streaming service could then modify the intensities of a defective pixel based on the intensities of neighboring pixels and/or notify the provider of the source video content that the associated video camera is defective. Subsequently, the video streaming service could perform transcoding operations to generate target video content in a variety of different formats targeting different display devices. By correcting the pixels that are identified as defective prior to performing the transcoding operations, the video streaming service would mitigate the domino effect described above.

Conventional approaches to detecting defective pixels in video content are labor-intensive, prohibitively time-consuming, and/or not necessarily reliable. For instance, in some conventional approaches to detecting defective pixels, a quality control person scrutinizes a high definition display for visual anomalies while viewing source video content from beginning to end repeatedly under controlled viewing conditions. Because manually monitoring pixels over the length of a media title is prohibitively difficult, the quality control person would likely overlook at least some defective pixels. Other conventional approaches to detecting defective pixels involve using quality control tools to detect defective pixels. One drawback of these type of quality control tools is that the quality control tools typically misidentify many non-defective pixels as defective and, consequently, require manual intervention to determine which of the identified "defective" pixels are actually defective.

To address the above problems, the system <NUM> includes, without limitation, a defective pixel detection application <NUM> that automatically and reliably detects defective pixels in the media content <NUM> based on intensity patterns that typically characterize defective pixels. The defective pixel detection application <NUM> resides in the memory <NUM> of the compute instance <NUM>(<NUM>) and executes on any number of processors <NUM> included in any number of the compute instances <NUM> in a distributed execution fashion. In alternative embodiments, the defective pixel detection application <NUM> may execute in any technically feasible fashion on any number of processors <NUM> (including one).

As shown, the defective pixel detection application <NUM> includes, without limitation, a split function <NUM>, a map function <NUM>, and a collection function <NUM>. The split function <NUM>, the map function <NUM>, and the collection function <NUM> execute via a split-map-collect workflow provided by a workflow subsystem <NUM>. The workflow subsystem <NUM> is a distributed execution framework that enables concurrent processing of the video content <NUM>. The workflow subsystem <NUM> resides in the memory <NUM> of the compute instance <NUM>(<NUM>) and executes on the processor <NUM> of the compute instance <NUM>(<NUM>). In alternative embodiments, the defective pixel detection application <NUM> may include any number and type of functions and may implement any number and type of distributed execution and/or concurrent processing techniques in any technically feasible fashion. In various embodiments, the system <NUM> may omit the workflow subsystem <NUM>.

For explanatory purposes only, a single instance of the split function <NUM> is depicted as executing on the compute instance <NUM>(<NUM>), N instances of the map function <NUM> are depicted as executing on N compute instances <NUM>(<NUM>)-<NUM>(N+<NUM>), and a single instance of the collect function <NUM> is depicted as executing on the compute instance <NUM>(<NUM>). In alternative embodiments, the instance of the split function <NUM>, the instances of the map function <NUM>, and the instance of the collect function <NUM> may execute on any number of compute instances <NUM> in any combination and in any technically feasible fashion.

Upon receiving the video content <NUM>, the workflow system <NUM> generates a job <NUM>(<NUM>) that executes the split function <NUM> on the video content <NUM>. The split function <NUM> partitions the video content <NUM> into clips <NUM>(<NUM>)-<NUM>(N). The split function <NUM> may partition the video content <NUM> into any number of the clips <NUM> in any technically feasible fashion. For example, the split function <NUM> could partition <NUM>,<NUM> frames included in the video content <NUM> based on a clip size of <NUM> frames to generate the clips <NUM>(<NUM>)-<NUM>(<NUM>,<NUM>), where each of the clips <NUM> includes a different set of <NUM> contiguous frames.

After the job <NUM>(<NUM>) finishes executing, the workflow subsystem <NUM> generates the jobs <NUM>(<NUM>)-<NUM>(N), where N is the number of the clips <NUM>. The job <NUM>(x+<NUM>) executes an instance of the map function <NUM>(x) based on the clip <NUM>(x) to generate a candidate frequency table <NUM>(x). The map functions <NUM>(<NUM>)-<NUM>(N) may generate the candidate frequency tables <NUM>(<NUM>)-<NUM>(N) concurrently, sequentially, or in any combination thereof based on the available resources. To generate the candidate frequency table <NUM>(x), the map function <NUM>(x) first performs per-frame operations (not shown in <FIG>) independently for each of the frames included in the clip <NUM>(x).

For a given frame, the map function <NUM>(x) performs locally adaptive contrast enhancement to amplify or suppress discontinuities in intensities of anomalous pixels to generate an enhanced frame. Subsequently, the map function <NUM>(x) generates an initial candidate list based on local intensity gradients associated with the enhanced frame. The initial candidate list includes, without limitation, any number of candidates. As referred to herein, a "candidate" specifies a potentially defective pixel. Advantageously, the number of candidates included in the initial candidate list is typically orders of magnitude less than the total number of pixels associated with the video content <NUM>.

Subsequently, the map function <NUM>(x) performs spatial filtering operations on the initial candidate list to generate a refined candidate list. More precisely, for each of the initial candidates, the map function <NUM> computes a statistical distance between a distribution of first degree intensity gradients and a distribution of second degree intensity gradients. If the statistical difference exceeds a distance threshold, then the map function <NUM> determines that the initial candidate is associated with patterns characteristic of defective pixels and adds the initial candidate to the refined candidate list. Otherwise, the map function <NUM> omits the initial candidate from the refined candidate list.

After the map function <NUM>(x) computes the different refined candidate lists associated with the different frames included in the clip <NUM>(x), the map function <NUM>(x) generates the candidate frequency table <NUM>(x) based on a temporal consistency criterion (not shown in <FIG>). The candidate frequency table <NUM>(x) includes, without limitation, any amount and type of information associated with any number of per-clip candidates. The map function <NUM>(x) may acquire and implement the temporal consistency criterion in any technically feasible fashion.

For instance, in some embodiments, the temporal consistency criterion is a minimum number of frames F that is specified via a graphical user interface (GUI). For each candidate included in any of the refined candidate lists, the map function <NUM>(x) computes the candidate frequency across the clip <NUM>(x). As referred to herein, the candidate frequency across the clip <NUM>(x) is the total number of refined candidate lists associated with the clip <NUM>(x) in which the candidate appears. If the candidate frequency across the clip <NUM>(x) is greater than F, then the map function <NUM>(x) adds the candidate to the candidate frequency table <NUM>(x). Otherwise, the map function <NUM>(x) omits the candidate from the candidate frequency table <NUM>(x). In this fashion, the map function <NUM>(x) filters out the candidates that are associated with temporally spurious visual anomalies. The map function <NUM> is described in greater detail below in conjunction with <FIG> and <FIG>.

After the jobs <NUM>(<NUM>)-<NUM>(N+<NUM>) finish executing, the workflow subsystem <NUM> generates the job <NUM>(N+<NUM>) that executes the collect function <NUM> based on the candidate frequency tables <NUM>(<NUM>)-<NUM>(N). More precisely, the collect function <NUM> evaluates the candidate frequency tables <NUM>(<NUM>)-<NUM>(N) with respect to a defect criterion <NUM> to generate a defective pixel list <NUM>. The defective pixel list <NUM> includes, without limitation, any amount and type of information associated with defective pixels. The collect function <NUM> may acquire and implement the defect criterion <NUM> in any technically feasible fashion. Further, the collect function <NUM> may perform any number of evaluation operations based on the candidate frequency tables <NUM>(<NUM>)-<NUM>(N) to determine the defective pixels. Some examples of evaluation operations include, without limitation, comparison operations and ranking operations.

For instance, in some embodiments, the defect criterion <NUM> is a maximum number "D" of pixels to include in the defective pixel list <NUM>. For each candidate, the collect function <NUM> computes an overall candidate frequency across the video content <NUM> based on the candidate frequency tables <NUM>(<NUM>)-<NUM>(N). The collect function <NUM> then ranks the candidates in an order of decreasing overall candidate frequency. Finally, the collect function <NUM> generates the defective pixel list <NUM> that includes, without limitation, the D candidates having the highest candidate frequencies. If less than D candidates have an overall candidate frequency greater than <NUM>, then the defective pixel list <NUM> includes less than D pixels.

In other embodiments, the defect criterion <NUM> is a frequency threshold specifying a minimum number "F" of frames across the video content <NUM>. F is the minimum number of frames in which a candidate occurs before the candidate is to be categorized as a defective pixel. For each candidate, the collect function <NUM> computes the overall candidate frequency across the video content <NUM> and compares the overall candidate frequency to the frequency threshold. If the overall candidate frequency exceeds the frequency threshold, then the collect function <NUM> adds the candidate to the defective pixel list <NUM> as a defective pixel. Otherwise, the collect function omits the candidate from the defective pixel list <NUM>. In the same or other embodiments, the collect function <NUM> may rank the defective pixels included in the defective pixel list <NUM> based on the overall candidate frequencies and/or include the overall candidate frequencies in the defective pixel list <NUM>.

Note that the techniques described herein are illustrative rather than restrictive, and may be altered without departing from the scope of the invention as defined in the claims. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope of the described embodiments and techniques. As a general matter, the techniques outlined herein are applicable to detecting defective pixels in video content based on a statistical distance between two distributions of pixel intensity gradients.

For instance, as persons skilled in the art will recognize, the intensity of a given pixel may vary across the frames of the video content <NUM>. In particular, during transcoding operations, defective pixels may alter the intensities of neighboring pixels in a manner that varies over time. For this reason, in alternative embodiments, the map function <NUM> may generate the candidate frequency table <NUM> at a color-level of granularity instead of a pixel-level of granularity. In the same or other embodiments, the collect function <NUM> may generate the defective pixel list <NUM> at the color-level of granularity instead of a pixel-level of granularity. As referred to herein, at a "pixel-level" granularity, two items are distinguished based on an (x, y) coordinate associated with the video content <NUM>. By contrast, at a "color-level," two items are distinguished based on a combination of an intensity range and an (x, y) coordinate associated with the video content <NUM>.

For example, the map function <NUM> could determine that the pixel-level candidate P at the (x, y) coordinates of (<NUM>, <NUM>) is associated with a primarily red intensity range from frame <NUM> to frame <NUM> and associated with a primarily green intensity range from frame <NUM> to frame <NUM>. The map function <NUM> would add two color-level candidates associated with the pixel-level candidate P to the candidate frequency table <NUM>(x). The first color-level candidate would be associated with the red intensity range and the (x, y) coordinates of (<NUM>, <NUM>). The second candidate would be associated with the green intensity range and the (x, y) coordinates of (<NUM>, <NUM>). Subsequently, the collect function <NUM> could determine that both the first color-level candidate and the second color-level candidate satisfy the defect criterion <NUM>. The collect function <NUM> would then add both the first color-level candidate and the second color-level candidate to the defective pixel list <NUM>. Advantageously, providing additional information regarding the intensity ranges for which a pixel is defective facilitates subsequent remedial actions.

<FIG> is a more detailed illustration of the map function <NUM>(<NUM>) of <FIG>, according to various embodiments of the present invention. As shown, the map function <NUM>(<NUM>) generates the candidate frequency table <NUM>(<NUM>) based on the clip <NUM>(<NUM>). The clip <NUM>(<NUM>) includes, without limitation, any number of frames <NUM>(<NUM>)-<NUM>(M). The map function <NUM>(<NUM>) includes, without limitation, M different sets of per-frame operations <NUM>(<NUM>)-<NUM>(M) and a set of temporal consistency operations <NUM>(<NUM>). The map function <NUM>(<NUM>) may perform the per-frame operations <NUM>(<NUM>)-<NUM>(M) concurrently, sequentially, or in any combination thereof using any number and type of resources. For instance, in some embodiments, the map function <NUM>(<NUM>) performs the per-frame operations <NUM> via cloud-based resources. The map function <NUM>(<NUM>) performs the temporal consistency operations <NUM>(<NUM>) after the sets of per-frame operations <NUM>(<NUM>)-<NUM>(M) have finished executing.

As shown, each set of per-frame operations <NUM>(x) includes, without limitation, contrast enhancement operations <NUM>(x), local gradient operations <NUM>(x), and spatial filtering operations <NUM>(x). The contrast enhancement operations <NUM>(x) generate an enhanced frame <NUM>(x) based on the frame <NUM>(x). To perform the contrast enhancement operations <NUM>(x), the map function <NUM>(<NUM>) applies a patch-based adaptive histogram equalization algorithm to the frame <NUM>(x). The patch-based adaptive histogram equalization algorithm operates on different small regions of the frame <NUM>(x) or "patches" to generate output patches included in the enhanced frame <NUM>(x). The histogram of each output patch approximately matches a histogram specified by a distribution parameter that is fine-tuned to selectively amplify spatially local signals emanating from defective pixels. In this fashion, the contrast enhancement operations <NUM>(x) enhance the intensities of defective pixels relative to the neighboring pixels.

In general, the contrast enhancement operations <NUM>(x) enhance the discriminative properties of the frame <NUM>(x) with respect to defective pixels. Consequently, the accuracy of detection operations performed on the enhanced frame <NUM>(x) is increased relative to the accuracy of detection operations performed on the frame <NUM>(x). In alternative embodiments, the map function <NUM> may perform any number and type of technically feasible operations to enhance the discriminative properties of the frames <NUM> with respect to defective pixels.

The local gradient operations <NUM>(x) generate an initial candidate list <NUM>(x) based on the enhanced frame <NUM>(x). The initial candidate list <NUM>(x) includes, without limitation, any amount and type of information associated with any number of candidates, where each candidate is associated with a potentially defective pixel. For instance, in some embodiments, the initial candidate list <NUM>(x) includes, without limitation, an ordered list of (x, y) coordinates in the enhanced frame <NUM>(x). To perform the local gradient operations <NUM>(x), the map function <NUM>(<NUM>) applies Hessian and Harris corner detection algorithms to identify outliers included in the enhanced frame <NUM>(x). The map function <NUM>(<NUM>) then adds the outliers to the initial candidate list <NUM>(i). Advantageously, the number of candidates included in the initial candidate list <NUM>(x) is typically orders of magnitude less than the number of pixels included in the enhanced frame <NUM>(x). In alternative embodiments, the map function <NUM> may perform any number and type of technically feasible operations to generate the initial candidate lists <NUM>.

The spatial filtering operations <NUM>(x) generate a refined candidate list <NUM>(x) based on the initial candidate list <NUM>(x) and the enhanced frame <NUM>(x). The refined candidate list <NUM>(x) includes, without limitation, any amount and type of information associated with any number of candidates. To generate the refined candidate list <NUM>(x), the map function <NUM>(<NUM>) performs high-level statistics based spatial filtering on the initial candidate list <NUM>(x) to identify which candidates may actually be defective based on intensity patterns typically associated with defective pixels.

More precisely, for each candidate included in the initial candidate list <NUM>(x), the map function <NUM>(<NUM>) computes a first degree gradient matrix and a second degree gradient matrix associated with, respectively, a <NUM>-by-<NUM> neighborhood and a <NUM>-by-<NUM> neighborhood. The map function <NUM>(<NUM>) then computes a Mahalanobis distance between the first degree gradient matrix and the second degree gradient matrix. If the Mahalanobis distance is greater than a distance threshold, then the map function <NUM>(<NUM>) adds the candidate to the refined candidate list <NUM>(x). Otherwise, the map function <NUM>(<NUM>) omits the candidate from the refined candidate list <NUM>(x). The spatial filtering operations <NUM> are described in greater detail below in conjunction with <FIG>. In alternative embodiments, the map function <NUM> may compute any type of statistical distance between any two distributions of gradients to generate the refined candidate list <NUM>(x).

After the map function <NUM>(<NUM>) generates the refined candidate lists <NUM>(<NUM>)-<NUM>), the map function <NUM>(<NUM>) performs the temporal consistency operations <NUM>(<NUM>). The temporal consistency operations <NUM>(<NUM>) generate the candidate frequency table <NUM>(<NUM>) based on the refined candidate lists <NUM>(<NUM>)-<NUM>), the enhanced frames <NUM>(<NUM>)-<NUM>(M), and a temporal consistency criterion <NUM>. The candidate frequency table <NUM>(<NUM>) includes, without limitation, any amount and type of information associated with any number of candidates associated with the clip <NUM>(<NUM>). The map function <NUM> may acquire and implement the temporal consistency criterion <NUM> in any technically feasible fashion.

For instance, in some embodiments, the temporal consistency criterion <NUM> is a temporal threshold that the map function <NUM>(<NUM>) acquires via a GUI. The temporal threshold specifies a minimum number of F frames in which a candidate is identified before the candidate is to be classified as a defective pixel. For each candidate that is included in at least one of the refined candidate lists <NUM>(<NUM>)-<NUM>(M), the map function <NUM>(<NUM>) computes the candidate frequency across the clip <NUM>(<NUM>). The candidate frequency across the clip <NUM>(<NUM>) is the total number of the refined candidate lists <NUM>(<NUM>)-<NUM>(M) in which the candidate appears. If the candidate frequency across the clip <NUM>(x) is greater than F, then the map function <NUM>(x) adds the candidate to the candidate frequency table <NUM>(x). Otherwise, the map function <NUM>(x) omits the candidate from the candidate frequency table <NUM>(x). In this fashion, the map function <NUM>(x) filters out the candidates that are associated with temporally spurious visual anomalies.

In various implements, the map function <NUM>(<NUM>) computes and/or supplements the candidate frequency table <NUM>(<NUM>) with any amount and type of additional information relevant to detecting and mitigating defective pixels. For instance, in some embodiments, as part of performing the temporal consistency operations <NUM>, the map function <NUM>(<NUM>) computes clip-level statistics for each defective pixel included in at least one of the refined candidate lists <NUM>(<NUM>)-<NUM>(M). Examples of clip-level statistics include, without limitation, the mean intensity of the candidate and the mean intensity of a <NUM>-by-<NUM> neighborhood surrounding the candidate. The map function <NUM>(<NUM>) may then add the clip-level statistics to the candidate frequency table <NUM>(<NUM>) and/or perform additional operations based on the clip-level statistics. Subsequently, the collect function <NUM> may perform any number and type of operations and/or supplement the defective pixel list <NUM> based on the clip-level statistics.

In some alternative embodiments, the refined candidate lists <NUM>(<NUM>)-<NUM>(M) include, without limitation, pixel-level candidates More specifically, each refined candidate list <NUM>(i) specifies an ordered list of (x, y) coordinates within the frame <NUM>(i). As described previously herein, the intensity of the pixel at a given (x, y) coordinates includes three channel values (R, G, and B). For each pixel-level candidate, the map function <NUM> computes any number of color-level candidate frequencies based on the refined candidate lists <NUM>(<NUM>)-<NUM>) and the ranges of the three channel values. More specifically, if the color of a given pixel-level candidate varies significantly across the frames <NUM> included in the clip <NUM>(<NUM>), then the map function <NUM>(<NUM>) computes multiple different color-level candidate frequencies. As referred to herein, a "color-level candidate" is potentially defective and is associated with (x, y) coordinates as well as an intensity range.

The map function <NUM>(<NUM>) then performs filtering operations based on the color-level candidate frequencies and the temporal consistency criterion <NUM> to generate the candidate frequency table <NUM>(<NUM>). For each unfiltered color-level candidate, the map function <NUM>(<NUM>) adds candidate data associated with the color-level candidate to the candidate frequency table <NUM>(<NUM>). The candidate data for a given color-level candidate includes, without limitation, the color-level candidate frequency, the (x, y) coordinates, and the intensity range. Subsequently, the collect function <NUM> generates the defective pixel list <NUM> at the color-level of granularity. More precisely, the collect function <NUM> computes overall color-level candidate frequencies and filters the color-level candidates based on the overall color-level candidate frequencies and the defect criterion <NUM>. The collect function <NUM> then ranks the unfiltered color-level candidates based on the overall color-level candidate frequencies. In this fashion, the collect function <NUM> indicates which of the channels (R, G, B) associated with each defective pixel are defective. Advantageously, indicating the channels associated with defective pixels streamlines remedial activities.

<FIG> is a more detailed illustration of the spatial filtering operations <NUM>(<NUM>) of <FIG>, according to various embodiments of the present invention. The map function <NUM>(<NUM>) performs the spatial filtering operations <NUM>(<NUM>) to generate the refined candidate list <NUM>(<NUM>) based on the enhanced frame <NUM>(<NUM>) and the initial candidate list <NUM>(<NUM>). Both the frame <NUM>(<NUM>) and the enhanced frame <NUM>(<NUM>) have a height of A pixels, a width of B pixels, and <NUM> different channel values (red, green, and blue) for each pixel.

For explanatory purposes only, <FIG> describes the spatial filtering operations <NUM>(<NUM>) at the pixel level of granularity. As shown, the initial candidate list <NUM>(<NUM>) includes, without limitation, <NUM> candidates <NUM>(<NUM>)-<NUM>(<NUM>) at the pixel level of granularity. The refined candidate list <NUM>(<NUM>) includes, without limitation, the <NUM> candidates <NUM>(<NUM>), <NUM>(<NUM>), <NUM>(<NUM>), <NUM>(<NUM>), <NUM>(<NUM>), and <NUM>(<NUM>) at the pixel level of granularity. In alternative embodiments, the map function <NUM>(<NUM>) may perform the spatial filtering operations <NUM>(<NUM>) at any level of granularity and may perform different operations at different levels of granularity. Further, the initial candidate list <NUM>(<NUM>) may include any number of candidates <NUM> at any level of granularity and the refined candidate list <NUM>(<NUM>) may include any number of candidates <NUM> at any level of granularity. In such embodiments, as outlined previously herein in conjunction with <FIG> and <FIG>, the techniques described herein may be modified accordingly.

As shown, the map function <NUM>(<NUM>) generates eight difference matrices <NUM>(<NUM>)-<NUM>(<NUM>) based on the enhanced frame <NUM>(<NUM>). Each of the difference matrices <NUM> has the same size as the enhanced frame <NUM>(<NUM>) but is associated with a different direction. The difference matrices <NUM>(<NUM>)-<NUM>(<NUM>) are associated with, respectively, top-left, left, right, top, bottom, bottom-right, top-right, and bottom-left directions. To compute the difference matrices <NUM>, the map function <NUM>(<NUM>) pads the enhanced frame <NUM>(<NUM>) to generate a padded matrix P[<NUM>:PA-<NUM>, <NUM>:PB-<NUM>, <NUM>:<NUM>]. (not shown in <FIG>) having height PA = A+<NUM>, width PB = B+<NUM>, and <NUM> different channel values for each pixel The map function <NUM>(<NUM>) may pad the enhanced frame <NUM>(<NUM>) in any technically feasible fashion.

For instance, in some embodiments, the map function <NUM>(<NUM>) adds two rows above, two rows below, two rows to the right of, and two rows to the left of the enhanced frame <NUM>(<NUM>). The two topmost rows included in the padded frame are replicas of the top row included in the enhanced frame <NUM>(<NUM>). The two bottom-most two rows included in the padded frame are replicas of the bottom row included in the enhanced frame <NUM>(<NUM>). The two rightmost columns included in the padded frame are replicas of the rightmost column included in the enhanced frame <NUM>(<NUM>). The two leftmost columns included in the padded frame are replicas of the leftmost column included in the enhanced frame <NUM>(<NUM>). To compute the additional corner entries included in the padded frame based, the map function <NUM>(<NUM>) performs averaging operations on any number of the neighboring entries in the padded frame.

To compute the difference matrices <NUM>(d), the map function <NUM> first computes a frame matrix based on the padded matrix. The frame matrix is the sub-matrix P[<NUM>:PA-<NUM>, <NUM>:PB-<NUM>, <NUM>:<NUM>], where the ranges <NUM>:PA-<NUM>, <NUM>:PB-<NUM>, and <NUM>:<NUM> are inclusive. As persons skilled in the art will recognize, the frame matrix is a matrix representation of the frame <NUM>(<NUM>). Subsequently, to compute the difference matrix <NUM>(d), the map function <NUM>(<NUM>) determines an AxBx3 "offset" sub-matrix included in the padded matrix based on the direction associated with the difference matrix <NUM>(d). Each offset sub-matrix is also referred to herein as an offset matrix. The map function <NUM>(<NUM>) then subtracts the offset sub-matrix from the frame matrix to generate the difference matrix <NUM>(d).

For instance, in some embodiments, to compute the difference matrix <NUM>(<NUM>) ctl associated with the direction of top-left, the map function <NUM>(<NUM>) sets the offset sub-matrix equal to P[<NUM>:PA-<NUM>, <NUM>:PB-<NUM>, <NUM>:<NUM>]. To compute the difference matrix <NUM>(<NUM>) cl associated with the direction of left, the map function <NUM>(<NUM>) sets the offset sub-matrix equal to P[<NUM>:PA-<NUM>, <NUM>:PB-<NUM>, <NUM>:<NUM>]. To compute the difference matrix <NUM>(<NUM>) cr associated with the direction of right, the map function <NUM>(<NUM>) sets the offset sub-matrix equal to P[<NUM>:PA-<NUM>, <NUM>:PB, <NUM>:<NUM>]. To compute the difference matrix <NUM>(<NUM>) ct associated with the direction of top, the map function <NUM>(<NUM>) sets the offset sub-matrix equal to P[<NUM>:PA-<NUM>, <NUM>:PB-<NUM>, <NUM>:<NUM>]. To compute the difference matrix <NUM>(<NUM>) cb associated with the direction of bottom, the map function <NUM>(<NUM>) sets the offset sub-matrix equal to P[<NUM>:PA, <NUM>:PB-<NUM>, <NUM>:<NUM>]. To compute the difference matrix <NUM>(<NUM>) cbr associated with the direction of bottom-right, the map function <NUM>(<NUM>) sets the offset sub-matrix equal to P[<NUM>:PA, <NUM>:PB, <NUM>:<NUM>]. To compute the difference matrix <NUM>(<NUM>) ctr associated with the direction of top-right, the map function <NUM>(<NUM>) sets the offset sub-matrix equal to P[<NUM>:PA-<NUM>, <NUM>:PB, <NUM>:<NUM>]. And, to compute the difference matrix <NUM>(<NUM>) cbl associated with the direction of bottom-left, the map function <NUM>(<NUM>) sets the offset sub-matrix equal to P[<NUM>:PA, <NUM>:PB-<NUM>, <NUM>:<NUM>]. Advantageously, as persons skilled in the art will recognize, computing the difference matrices <NUM> efficiently computes the intensity gradients along the different directions.

Subsequently, for each of the candidates <NUM>(i) included in the initial candidate list <NUM>(<NUM>), the map function <NUM>(<NUM>) computes a first (<NUM>st) degree gradient matrix (S) <NUM>(i) and a second (<NUM>nd) degree gradient matrix (T) <NUM>(i) based on the difference matrices <NUM>. The first degree gradient matrices <NUM> and the second degree gradient matrices <NUM> are also referred to herein as, respectively, the first order derivative matrices and the second order derivative matrices. In general, the entries in the first degree gradient matrix <NUM>(i) specify the first degree local gradients in a 3x3 spatial neighborhood around the candidate <NUM>(i). The first degree gradient matrix <NUM>(i) has dimensions of 8x3, where the columns signify the gradients along the three channels R, G, and B. By contrast, the entries in the second degree gradient matrix <NUM>(i) specify the second degree local gradients in a 5x5 spatial neighborhood around the candidate <NUM>(i). The second degree gradient matrix <NUM>(i) has dimensions of 20x3, where the columns signify the gradients along the three channels R, G, and B.

The map function <NUM>(<NUM>) may compute the first degree gradient matrix <NUM>(i) and the second degree gradient matrix <NUM>(i) in any technically feasible fashion. For instance, in some embodiments, the map function <NUM>(<NUM>) generates the first degree gradient matrix (S) <NUM>(i) for the candidate <NUM>(i) at the coordinates (x, y) based on the following equation (<NUM>): <MAT>.

And the map function <NUM>(<NUM>) generates the second degree gradient matrix (T) <NUM>(i) for the candidate <NUM>(i) at the coordinates (x, y) based on the following equation (<NUM>): <MAT>.

The first degree gradient matrix <NUM>(i) and the second degree gradient matrix <NUM>(i) provide insight into whether the candidate <NUM>(i) is associated with visually anomalous patterns that typically characterize defective pixels. An example of such a pattern is an abrupt change in intensity gradient that is typically observed in association with a dead pixel. In operation, the map function <NUM>(<NUM>) computes a statistical difference between the first degree gradient matrix <NUM>(i) and the second degree gradient matrix <NUM>(i). The map function <NUM>(<NUM>) then compares the statistical distance to a distance threshold <NUM> to determine whether to continue to re-classify the candidate <NUM>(i) as non-defective. The map function <NUM> may compute any statistical difference and apply any distance-related filtering criterion instead of the distance threshold <NUM> in any technically feasible fashion.

For instance, in some embodiments, the map function <NUM>(<NUM>) computes a Mahalanobis distance <NUM>(i) based on the first degree gradient matrix <NUM>(i) and the second degree gradient matrix <NUM>(i). First, the map function <NUM>(<NUM>) computes a 3x3 covariance matrix (C) based on the second degree gradient matrix <NUM>(i). The covariance matrix captures the variance across multiple channels (e.g., R, G, and B). Subsequently, the map function <NUM>(<NUM>) computes a 3x1 mean difference vector associated with the candidate <NUM>(i) based on the following equation (<NUM>): <MAT>.

The map function <NUM>(<NUM>) then computes the Mahalanobis distance (d) <NUM>(i) based on the following equation (<NUM>): <MAT>.

The map function <NUM>(<NUM>) then compares the Mahalanobis distance <NUM>(i) to the distance threshold <NUM>. If the Mahalanobis distance <NUM>(i) exceeds the distance threshold <NUM>, then the map function <NUM>(<NUM>) adds the candidate <NUM>(i) to the refined candidate list <NUM>(<NUM>). Otherwise, the map function <NUM>(<NUM>) omits the candidate <NUM>(i) from the refined candidate list <NUM>(<NUM>).

In the example depicted in <FIG>, the Mahalanobis distances <NUM>(<NUM>), <NUM>(<NUM>), <NUM>(<NUM>), <NUM>(<NUM>), <NUM>(<NUM>), and <NUM>(<NUM>) exceed the empirically determined distance threshold <NUM> of <NUM>. Consequently, the map function <NUM>(<NUM>) generates the refined candidate list <NUM>(<NUM>) that includes the candidates <NUM>(<NUM>), <NUM>(<NUM>), <NUM>(<NUM>), <NUM>(<NUM>), <NUM>(<NUM>), and <NUM>(<NUM>). The map function <NUM> may determine the distance threshold <NUM> in any technically feasible fashion.

Many modifications and variations on the functionality provided by the defective pixel detection application <NUM>, including the map function <NUM>, will be apparent to those of ordinary skill in the art without departing from the scope of the described embodiments. For instance, in alternative embodiments, the map function <NUM> may compute any type of statistical difference between any number of distributions of pixel intensity gradients to determine whether to re-classify a candidate as a non-defective pixel.

<FIG> is a flow diagram of method steps for detecting defective pixels in video content, according to various embodiments of the present invention. Although the method steps are described with reference to the systems of <FIG>, persons skilled in the art will understand that any system configured to implement the method steps, in any order, falls within the scope of the present invention.

As shown, a method <NUM> begins at step <NUM>, where the split function <NUM> partitions the video content <NUM> into the clips <NUM>(<NUM>)-<NUM>(N). At step <NUM>, for each of the clips <NUM>(<NUM>)-<NUM>(N), the map function <NUM> performs the contrast enhancement operations <NUM> on the frames <NUM>(<NUM>)-<NUM>(M) included in the clip <NUM> to generate the enhanced frames <NUM>(<NUM>)-<NUM>(M) associated with the clip <NUM>. At step <NUM>, for each of the clips <NUM>(<NUM>)-<NUM>(N), the map function <NUM> performs the local gradient operations <NUM> on the enhanced frames <NUM>(<NUM>)-<NUM>(M) associated with the clip <NUM> to generate the initial candidate lists <NUM>(<NUM>)-<NUM>(M) associated with the clip <NUM>. Accordingly, as part of step <NUM>, the map function <NUM> generates a total of NxM initial candidate lists <NUM>, where each initial candidate list <NUM> is associated with a different enhanced frame <NUM>. At step <NUM>, for each of the clips <NUM>(<NUM>)-<NUM>(N), the map function <NUM> generates eight different difference matrices <NUM>(<NUM>)-<NUM>(<NUM>) for each of the associated enhanced frames <NUM>(<NUM>)-<NUM>(M). In this fashion, the map function <NUM> generates a total of NxMx8 difference matrices <NUM>, where each set of <NUM> difference matrices <NUM>(<NUM>)-<NUM>(<NUM>) is associated with a different enhanced frame <NUM> and a different initial candidate list <NUM>.

At step <NUM>, for each candidate <NUM> included in each initial candidate list <NUM>, the map function <NUM> generates a different first degree gradient matrix <NUM> and a different second degree gradient matrix <NUM> based on the associated difference matrices <NUM>(<NUM>)-<NUM>(<NUM>). At step <NUM>, for each candidate <NUM> included in each initial candidate list <NUM>, the map function <NUM> computes a different Mahalanobis distance <NUM> based on the associated first degree gradient matrix <NUM> and the associated second degree gradient matrix <NUM>. At step <NUM>, the map function <NUM> filters each of the initial candidate lists <NUM> based on the associated Mahalanobis distances <NUM> and the distance threshold <NUM> to generate the corresponding refined candidate list <NUM>.

At step <NUM>, for each clip <NUM>(i), the map function <NUM> performs the temporal consistency operations <NUM>(i) based on the refined candidate lists <NUM>(<NUM>)-<NUM>(M) associated with the clip <NUM>(i) to generate the candidate frequency table <NUM>(i). At step <NUM>, the collect function <NUM> generates the defective pixel list <NUM> based on the candidate frequency tables <NUM>(<NUM>)-<NUM>(N). At step <NUM>, the collect function <NUM> transmits the defective pixel list <NUM> to any number and type of applications to facilitate remedial actions associated with the video content <NUM> and/or the image sensor(s) that acquired the defective pixels. The method <NUM> then terminates.

In sum, the disclosed techniques may be used to automatically and efficiently detect defective pixels in video content. In operation, a workflow subsystem provides a distributed execution framework for a defective pixel detection application. The defective pixel detection application includes, without limitation, a split function, a map function, and a collect function. In operation, the split function partitions the video content into clips of frames. For each of the clips, a different instance of the map function performs per-frame operations followed by temporal consistency operations to generate a different candidate frequency table.

More precisely, for a given frame, the map function implements an adaptive histogram equalization algorithm to generate an associated enhanced frame. In general, the adaptive histogram equalization algorithm selectively amplifies spatially local signals emanating from defective pixels. Subsequently, the map function executes Hessian and Harris corner detection algorithms on the enhanced frame to generate an initial candidate list of potentially defective pixels. For each candidate included in the initial candidate list, the map function then generates a first degree gradient matrix associated with a <NUM>-by-<NUM> neighborhood surrounding the candidate. The map function also generates a second degree gradient matrix associated with a <NUM>-by-<NUM> neighborhood surrounding the candidate. The map function then computes a Mahalanobis distance between the first degree gradient matrix and the second degree gradient matrix. If the Mahalanobis distance exceeds an empirically determined distance threshold, then the map function adds the candidate to a refined candidate list associated with the frame.

Subsequently, for each of the clips, the map function generates a different candidate frequency table based on the refined candidate lists for all the frames included in the clip and a temporal consistency criterion that specifies a minimum number of frames (F). More specifically, for each candidate, the map function sets the candidate frequency across the clip equal to the number of refined candidate lists in which the candidate appears. If the candidate frequency across the clip is greater than F, then the map function adds the candidate and the associated candidate frequency across the clip to the candidate frequency table. Otherwise, the map function omits the candidate from the candidate frequency table.

Finally, for each candidate included in the candidate frequency tables, the collect function computes an overall candidate frequency across the video content. The collect function then ranks the candidates based on the overall candidate frequencies and filters out any number of the ranked candidates based on a defect criterion. The collect function then generates a ranked defective pixel list that includes the remaining candidates and the associated overall candidate frequencies.

Advantageously, by automatically identifying defective pixels, the defective pixel detection application reduces both the time and amount of manual effort required to detect defective pixels in video content relative to prior art solutions. In particular, because the defective pixel detection application processes clips in a concurrent fashion, the defective pixel detection application efficiently processes the video content, irrespective of the total number of frames. Further, because the defective pixel detection application performs statistics-based spatial filtering as well as temporal consistency filtering on the initial candidate lists, the accuracy with which the defective pixel detection algorithm classifies pixels is increased relative to prior art solutions. These technical advantages provide one or more technological advancements over the prior art.

Aspects of the present embodiments may be embodied as a system, method or computer program product. Accordingly, aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "module" or "system. " In addition, any hardware and/or software technique, process, function, component, engine, module, or system described in the present disclosure may be implemented as a circuit or set of circuits. Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.

Claim 1:
A computer-implemented method (<NUM>), comprising:
computing (<NUM>) a first matrix of intensity gradients based on a first frame of a video and a first degree local gradient in a three-by-three neighborhood of pixels around a first pixel;
computing (<NUM>) a second matrix of intensity gradients based on the first frame and a second degree local gradient in a five-by-five neighborhood of pixels around the first pixel;
computing (<NUM>) a Mahalanobis distance between the first matrix of intensity gradients and the second matrix of intensity gradients; and
determining (<NUM>) that the first pixel is defective when the Mahalanobis distance crosses a threshold,
wherein computing the Mahalanobis distance comprises:
calculating a vector based on a difference between a mean associated with the first matrix of intensity gradients and a mean associated with the second matrix of intensity gradients;
computing a three-by-three covariance matrix based on the second matrix of intensity gradients; and
computing the Mahalanobis distance based on the vector and the covariance matrix.