Image processing apparatus and image processing method

An image processing apparatus that handles RAW images efficiently performs editing for reproducing RAW images.The apparatus compresses each of RAW images and reduced RAW images which are reduced from the RAW images, and records each in a recording medium. When editing, the image processing apparatus communicates with an external apparatus and transmits the reduced RAW images stored in the recording medium to the external apparatus, and causes the external apparatus to perform editing processing of the reduced images. Further, the image processing apparatus uses editing information representing the content of editing processing from the external apparatus to execute editing of the RAW image corresponding to the reduced RAW image on which the editing processing has been performed, by referencing the editing information.

TECHNICAL FIELD

The present invention relates to an image processing apparatus and an image processing method, and more particularly relates to technology of handling RAW images of moving images or still images.

BACKGROUND ART

When performing shooting operations, conventional imaging apparatuses subject raw image information imaged by an imaging sensor (RAW images) to de-Bayering processing (demosaicing processing) so as to convert the raw image information into signals made up of luminance and color difference. The signals are then subjected to developing processing such as noise removal, optical distortion correction, image optimization, and so forth, The imaging apparatus then generally performs compression encoding on the developed luminance signals and color difference signals, and then records in a recording medium.

On the other hand, there are imaging apparatuses capable of recording RAW images. While the amount of data necessary to record RAW image is great, this is preferred by many advanced users. The reason is that advantages there are such as correction and deterioration of the original image being minimal, and that post-shooting editing can be performed.

PTL 1 discloses an imaging apparatus that records RAW images. Disclosed in PTL 1 is a configuration where developing parameters are recorded along with a RAW image, and when reproducing, the RAW image is developed and reproduced using these developing parameters.

As of recent, imaging sensors in imaging apparatuses have advanced to where the number of pixels per image is much greater. Further, the number of images which can be taken by continuous shooting per second is on the rise. This has led to a compounded increase in the amount of processing for each of the processing making up the developing processing, such as de-Bayering processing on RAW images, noise removal, optical distortion correction, and so forth. This has come to necessitate large-scale circuits and increased electric power consumption in the imaging apparatuses, in order to performing real-time developing processing in parallel with shooting. Even then, there may be cases where the imaging apparatus cannot exhibit high-level shooting performance, due to circuits being occupied for developing processing, and constraints related to electric power consumption.

On the other hand, the amount of processing related to developing at the time of shooting might be reduced by a configuration where RAW images are recorded without being developed, such as in PTL 1, but promptly reproducing and displaying the images becomes difficult since the images are recorded in a pre-development state. Further, the fact that RAW images have peculiarities unique to this data format, and that the format may differ from one manufacturer to another, can result in RAW images taken with one device not being able to be reproduced (developed) on another device. Accordingly, conventional RAW image recording formats have in cases been disadvantageous with regard to ease of use by the user.

There has been a problem as described above, that in order for an imaging apparatus to realize high-level shooting performance and also be capable of fast image output of reproduced images, either expensive circuits need to be installed for high-output driving, or the RAW images need to be able to be recorded and reproduced in a fast and convenient manner. Particularly, increased costs are a detriment to the user, so it is important that the imaging apparatus be able to record RAW images in an easy-to-handle manner.

CITATION LIST

Patent Literature

SUMMARY OF INVENTION

Provided is an image processing apparatus and image processing method enabling an apparatus that records and reproduces RAW images to efficiently perform editing for reproducing RAW images.

An image processing apparatus according to the present invention includes: an imaging unit configured to use an imaging sensor to acquire a RAW image representing a pre-development image from imaging signals of a subject image; a reduction unit configured to reduce the RAW image to generate a reduced RAW image; a recording unit configured to record each data of the RAW image and the reduced RAW image in a recording medium; a communication unit configured to communicate with an external apparatus; and a control unit configured to transmit data of the reduced RAW image recorded in the recording medium to the external apparatus via the communication unit, and to cause the external apparatus to perform editing processing of the reduced RAW image, wherein the control unit receives editing information representing the content of editing processing from the external apparatus via the communication unit, and uses this editing information to execute editing of the RAW image corresponding to the reduced RAW image on which the editing processing has been performed, by referencing the editing information.

DESCRIPTION OF EMBODIMENTS

First Embodiment

Embodiments of the present invention will be described in detail with reference to the drawings.FIG. 1is a block diagram illustrating a configuration example of an image processing apparatus according to a first embodiment of the present invention. An imaging apparatus100will be exemplarily illustrated as the image processing apparatus according to the present embodiment inFIG. 1. The imaging apparatus100not only records image information obtained by imaging a subject in a recording medium, but also has functions of reproducing image information from a recording medium, performing developing processing, and displaying, and functions of exchanging image information with an external apparatus, server (cloud), or the like, and performing editing. Accordingly, the image processing apparatus according to the present embodiment is not restricted to being expressed as being an imaging apparatus, and may also referred to as being a recording apparatus, reproducing apparatus, recording/reproducing apparatus, communication apparatus, editing apparatus, image processing system, editing system, and so forth.

A control unit161inFIG. 1includes a central processing unit (CPU) and memory (omitted from illustration) storing a control program which the CPU executes, thereby controlling the overall processing of the imaging apparatus100. An operating unit162includes input devices such as keys, buttons, a touch panel, and so forth, for the user to give instructions to the imaging apparatus100. Operation signals from the operating unit162are detected by the control unit161, and the components are controlled by the control unit161so that actions are executed corresponding to the operations. A display unit123includes a liquid crystal display (LCD) or the like to display images that have been shot or reproduced, menu screens, various types of information, and so forth, on the imaging apparatus100.

In the present embodiment, the terms “shoot” and “shooting” mean the actions of imaging a subject and displaying the image obtained by the imaging on the display unit123, and also further recording the image in a predetermined recording medium as a file.

Upon starting of shooting action being instructed by the operating unit162, an optical image of a subject to be imaged is input via an optical unit101including a lens unit, and is imaged on an imaging sensor unit102. When shooting, the actions of the optical unit101and the imaging sensor unit102are controlled by a camera control unit104, based on evaluation value calculation results such as aperture, focus, shaking, and so forth, that are acquired by an evaluation value calculation unit105, and subject information such as facial recognition results extracted by a recognition unit131.

The imaging sensor unit102converts light (subject image) that has passed through a mosaic color filter of red, green, and blue (RGB) placed at each pixel, into electric signals. The resolution of the imaging sensor is equivalent to, for example, 4K (8 million pixels or more) or 8K (33 million pixels or more).FIG. 11is a diagram illustrating an example of a color filter disposed at the imaging sensor unit102, showing a pixel array of an image which the imaging apparatus100handles. Red (R), green (G), and blue (B) are arrayed in mosaic fashion at each pixel, with one set of one red pixel, one blue pixel, and two green pixels being orderly arrayed for every four pixels (two pixels by two pixels), as illustrated inFIG. 11. Electric signals converted by the imaging sensor unit102have the red (R), green (G), and blue (B) components. Note that green (G) can be handled as two type of green (G0, G1) components at different positions. This sort of pixel array is generally referred to as a Bayer array. The imaging sensor may be a charge-coupled device (CCD) image sensor, complementary metal oxide semiconductor (CMOS) image sensor, or the like. While an RGB color Bayer array has been illustrated, other arrays may also be optionally used for the colors and array for the color filter.

Electric signals converted by the imaging sensor unit102are handled as imaging signals of individual pixels. A sensor signal processing unit103subjects the pixels included in the imaging signals to restoration processing. This restoration processing involves processing of handling values of missing or unreliable pixels at the imaging sensor unit102by performing interpolation using nearby pixel values for such pixels to be restored, and subtracting a predetermined offset value. This may be altered so that part or all of this restoration processing is not performed here but later when developing. In the present embodiment, all images which have not been subjected to substantive developing are handled as RAW images, indicating pre-development images, regardless of whether or not restoration processing has been performed.

That is to say, image information output from the sensor signal processing unit103is referred to as RAW image information (hereinafter, “RAW image”), meaning a raw (pre-development) image in the present embodiment. The RAW image is supplied to the RAW compression unit113and compressed for efficient recording. The size of the RAW image is also reduced by a RAW reduction unit109for efficient display and reproduction. The RAW reduction unit109resizes the input RAW images to high-definition (HD) size (equivalent to around 2 million pixels), for example. Hereinafter, a RAW image which has been reduced by the RAW reduction unit109will be referred to as a reduced. RAW image.

The reduced RAW image is supplied to a live view developing unit111within a developing unit110, and subjected to developing processing to be used for display when shooting (live view). The reduced RAW image is also supplied to a RAW compression unit113for use in simple reproduction or as a proxy when editing. The reduced RAW image is recorded after being compressed by the RAW compression unit113for efficient recording, in the same way as with a RAW image. Further, the reduced RAW image is also supplied to the evaluation value calculation unit105.

The developing unit110will be described in detail now. The developing unit110has the live view developing unit111(first developing unit) that performs developing for the live view when shooting, a high-image, quality developing unit112(second developing unit) that performs high-quality developing when not shooting, and a switch unit121that selects the output thereof. The live view developing unit111has the capability to perform real-time developing of the reduced RAW image in parallel with shooting, while the high-image-quality developing unit112has the capability to develop the RAW image before reduction, which is a larger image than a reduced RAW image, in high definition in non-real-time. In the present embodiment, live view developing may also be referred to as simple developing or display developing, and high-quality developing may also he referred to as main developing or reproduction developing, performed at a requested timing.

The high-image-quality developing unit112performs de-Bayering processing (demosaicing processing) on input RAW images or reduced RAW images, where color interpolation processing is performed to convert into luminance and color difference (or primary color) signals, removes noise included in the signals, corrects optical distortion, and optimizes the image, performs so-called developing processing. Further, the live view developing unit111performs de-Bayering processing (demosaicing processing), i.e., color interpolation processing, on reduced RAW images, to convert into luminance and color difference (or primary color) signals, removes noise included in the signals, corrects optical distortion, and optimizes the image, i.e., performs so-called developing processing.

The high-image-quality developing unit112performs each process in higher precision than the live view developing unit111, Due to the high precision, a higher quality developed image is obtained as compared to the live view developing unit111, but the processing load is greater. Accordingly, the high-image-quality developing unit112according to the present embodiment is of a configuration where the developing processing can be performed when reproducing, or when idle after shooting. The circuit scale and the maximum (peak) electric power consumption can be suppressed by performing high-image-quality developing after shooting or when reproducing, and not while shooting. On the other hand, the live view developing unit111is configured such that the amount of processing involved in developing is less than in the high-image-quality developing so that developing processing can be performed at high speed while shooting, although the image quality is lower than that of the high-image-quality developing unit112. The processing load of the live view developing unit111is low, and accordingly real-time developing of each frame can be performed in parallel with shooting actions when performing live view shooting of moving images or still images.

The switch unit121is switched by the control unit161, following control in accordance with operations instructed by the user from the operating unit162or an operating mode currently being executed. An arrangement may be made where just the one of the live view developing unit111and high-image-quality developing unit112that is to output signals performs developing operations in conjunction with the switching of the switch unit121, while the operations of the other are stopped. Although the live view developing unit111and the high-image-quality developing unit112are illustrated as being separately-existing configurations in the developing unit110according to the present embodiment, a single developing unit may selectively perform live view developing and high-image-quality developing by switching an operation mode.

The image information subjected to developing processing by the developing unit110is further subjected to predetermined display processing by a display processing unit122, and thereafter displayed at a display unit123. The image information subjected to developing processing can also be output to an external display device connected by a video output terminal124. The video output terminal124includes general-purpose interfaces such as High-Definition Multimedia Interface (HDMI, a registered trademark) and serial digital interface (SDI).

When shooting, image information subjected to live view developing by the developing unit110is supplied to the evaluation value calculation unit105. The evaluation value calculation unit105calculates evaluation values from the reduced RAW image or developing-processed image information, such as focus state, exposure state, and so forth. The calculated evaluation values are output to the camera control unit104. Display information indicating the evaluation results is output to the display processing unit122. Further, the image information subjected to the live view developing is also supplied to the recognition unit131. The recognition unit131has functions of detecting and recognizing subject information, such as faces, people, and so forth, in the image information. For example, the recognition unit131detects whether or not there are faces in the screen of the image information, and if there are faces, outputs information indicating the position of the faces to the camera control unit104, and further recognizes particular people based on feature information such as faces and so forth. Display information indicating the detection and recognition results is output to the display processing unit122.

Image information subjected to high-image-quality developing by the developing unit110is supplied to a still image compression unit141or a moving image compression unit142. In a case of compressing the image information as a still image, the still image compression unit141is used. In a case of compressing the image information as a moving image, the moving image compression unit142is used. The still image compression unit141and moving image compression unit142perform high-efficiency encoding (compression coding) of respective image information thereby generating image information where the amount of information has been compressed, and converts into a high-image-quality-developed file (still image file or moving image file), Examples of standard coding techniques that can be used include JPEG and the like for still image compression, and MPEG-2, H.264, H.265, and the like for moving image compression.

The RAW compression unit113performs high-efficiency coding of the RAW image data from the sensor signal processing unit103and reduced RAW image data from the RAW reduction unit109, using technology such as wavelet transform, quantization, entropy coding (differential coding, etc.), and so forth. The RAW compression unit113generates a files of RAW images (RAW files) and reduced RAW images (reduced RAW files) which have been compressed by high-efficiency coding. The RAW files and reduced RAW files are first stored in a buffer115(storage medium). The RAW files and reduced RAW files may be left in the buffer115and called up again, or may be stored in the buffer115and then transferred to a separate recording medium and recorded (deleted from the buffer115), as described later.

The RAW files and reduced RAW files, and the high-image-quality developed files (still image files and moving image files) are recorded in a recording medium152by a recording/reproducing unit151. Examples of the recording medium152include a built-in large-capacity semiconductor memory or hard disk, a detachable memory card, or the like. The recording/reproducing unit151may also read out various types of files from the recording medium152in accordance with user operations.

The recording/reproducing unit151can exchange various types of files and related information with an external storage server connected over a network via a communication unit153, or with a mobile information terminal or personal computer (PC) or the like. The communication unit153is configured so as to he able to access external apparatuses over the Internet by wireless communication or cabled communication, or by direct communication between devices using a communication terminal154. Accordingly, the recording/reproducing unit151can record the RAW files and reduced RAW files, and the high-image-quality developed files (still image files and moving image files) in external apparatuses, and give instructions to generate editing information based on the recorded information.

When performing reproducing operations of the imaging apparatus100, the recording/reproducing unit151obtains a desired file from the recording medium152, or from an external apparatus via the communication unit153, and reproduces the file. If the file to be reproduced is a RAW file or reduced RAW file, the recording/reproducing unit151stores the obtained RAW file in the buffer115. If the file to be reproduced is a still image file, the recording/reproducing unit151supplies the obtained file to a still image decompression unit143, If the file to be reproduced is a moving image file, the recording/reproducing unit151supplies the obtained file to a moving image decompression unit144.

A RAW decompression unit114reads out a RAW file or reduced RAW file stored in the buffer115, performs conversion opposite to that performed during compression, and thus decompresses the compressed file. The RAW image or reduced RAW image decompressed by the RAW decompression unit114is supplied to the high-image-quality developing unit112, and is subjected to the high-image-quality developing processing as described above.

The still image decompression unit143decodes and decompresses still image files that have been input, and supplies these to the display processing unit122as reproduced images of still images. The moving image decompression unit144decodes and decompresses moving image files that have been input, and supplies these to the display processing unit122as reproduced images of moving images.

Next, operating modes of the imaging apparatus100according to the present embodiment will be describe with reference to the drawings.FIG. 2is a state (mode) transition diagram according to the present embodiment. Such mode transition is executed in accordance with user operation instructions from the operating unit162, or by determination by the control unit161. Accordingly, transition may be performed manually by operations, or transition may take place automatic. As can be seen inFIG. 2, the imaging apparatus100operations by switching through an idle state (200) to the five modes of a still image shooting mode (201), a still image reproducing mode (202), a moving image shooting mode (203), a moving image reproducing mode (204), and an editing mode (205), as appropriate.

Next, the operations relating to the still image shooting mode and moving image shooting mode of the imaging apparatus100will be described.FIG. 3illustrates a flowchart relating to shooting processing according to the present embodiment. The flowchart inFIG. 3illustrates processing procedures executed by the control unit161controlling each processing block. These processing procedures are realized by loading a program stored in memory (read only memory (ROM)) of the control unit161to random access memory (RAM), and the CPU executing the program.

Upon shooting processing of a still image or moving image starting inFIG. 3, in S301the control unit161determines whether or not to stop shooting. If determination is made to stop the shooting processing, transition is made to the idle state, otherwise, the flow advances to S302. An arrangement may be made where the state transitions to the idle state even in the shooting mode, of there is no operating input accepted for a predated amount of time, or there is time till the next shooting.

In S302, the camera control unit104controls the operations of the optical unit101and imaging sensor unit102so that shooting is performed under suitable conditions. For example, lenses included in the optical unit101are moved according to zoom or focusing instructions given by the user, a readout region of the imaging sensor unit102is set following instructions of the number of shooting pixels, and so forth. Also, control such as focus adjustment and tracking of a particular subject is performed based on evaluation value information and subject information supplied from the later-described evaluation value calculation unit105and recognition unit131. Further, in S302, electric signals obtained by conversion at the imaging sensor unit102are subjected to signal processing at the sensor signal processing unit103for restoration of pixels. That is to say, the sensor signal processing unit103subjects missing or unreliable pixels to interpolation using nearby pixel values for such pixels to be restored, subtracting a predetermined offset value, or the like, in the present embodiment, image information which is output from the sensor signal processing unit103after the processing in S302has ended is called a RAW image, meaning a raw (pre-development) image.

In S303, the RAW reduction unit109generates a reduced RAW image from the above-described RAW image. In S304, the live view developing unit111performs developing processing (live view developing) of the reduced RAW image. The control unit161switches the switch unit121within the developing unit110to select output of the image information subjected to developing processing by the live view developing unit111.

The live view developing unit111subject the reduced RAW image to de-Bayering processing (demosaicing processing), i.e., color interpolation processing, so as to convert into signals made up of luminance and color difference (or primary colors). The signals are then subjected to developing processing such as noise removal, optical distortion correction, image optimization, and so forth. The live view developing unit111realizes high speed of developing and simple processing, by eliminating or restricting the range of the noise removal and optical distortion correction, The developing processing (simple developing) which the live view developing unit111performs will be described. Due to the live view developing unit111handling reduced RAW images, and restricting part of the developing processing functions, the imaging apparatus100can realize shooting with performance such as 60 frames of 2 million pixels per second, for example, using a small circuit scale and low power consumption.

The image information subjected to developing processing at the live view developing unit111is supplied to the evaluation value calculation unit105. In S305, the evaluation value calculation unit105computes evaluation values such as focus state, exposure state, and so forth, from the luminance values, contrast values, and so forth, included in the image information. Note that the evaluation value calculation unit105may obtain a reduced RAW image before live view developing, and calculate some sort of evaluation value from the reduced RAW image as well.

The image information subjected to developing processing at the live view developing unit111is also supplied to the recognition unit131. In S306, the recognition unit131performs detection of a subject (such as a face) from the image information, and recognizes the subject information. For example, the recognition unit131detects whether or not there are faces in the screen of the image information, and if there are faces, recognizes position of the faces and particular people, and outputs the results as information.

The image information subjected to developing processing at the live view developing unit111is also supplied to the display processing unit122. In S307, the display processing unit122forms a display image from the acquired image information, and outputs to the display unit123or an external display device for display. A display image on the display unit123is used as a live view display for aiding the user in suitably framing the subject, i.e., a live view image for shooting. Note that the display image from the display processing unit122may be displayed on another external display device such as a television or the like, via the video output terminal124. Further, the display processing unit122may use the evaluation value information and subject information output from the evaluation value calculation unit105and recognition unit131to mark a focused region on the displayed image, displaying a frame at the position where a face has been recognized, and so forth, for example.

In S308, the control unit161determines whether or not there has been a shooting instruction from the user (a recording instruction in the case of a moving image), and if there has been such an instruction, the flow advances to S310. If there is no instruction in S308, the flow returns to S301, and repeats shooting (recording) preparation operations and live view display.

In S310, the RAW compression unit113performs high-efficiency coding (reduced RAW compression) on the reduced RAW image corresponding to the image to be shot (in the case of a moving image, multiple consecutive frames) in response to the aforementioned shooting instruction, and generates a reduced RAW file. Further, in S311, in response to the above-described shooting instructions, the RAW compression unit113performs high-efficiency coding (RAW compression) on the RAW image corresponding to the image to be shot (in the case of a moving image, multiple consecutive frames), and generates a RAW file. The high-efficiency coding which the RAW compression unit113performs here is according to known technology such as wavelet transform, entropy encoding, and so forth, but may be lossy coding or lossless coding. In the present embodiment, a RAW file is generated which can be restored as a high-image-quality file where the quality is the original RAW image is not markedly lost even if the RAW is compressed.

In S312, the reduced RAW file is recorded in the buffer115. Further, in S313, the RAW file is recorded in the buffer115. In the case of either file, once the file is recorded in the buffer115, the timing of recording in a recording medium downstream by the recording/reproducing unit151may be either at that point or later. Once the reduced RAW file and the RAW file are recorded to at least the buffer115, the flow returns to S301.

Accordingly, the imaging apparatus100according to the present embodiment generates RAW files in response to shooting instructions of still images or moving images (recording instructions) at the time of shooting. Otherwise when shooting, a shooting standby state is effected where an image obtained by live view developing is displayed. A RAW file is a high-image-quality file so that the image information supplied from the sensor signal processing unit103is not markedly lost, but developing processing is not needed to generate this file. Accordingly, RAW files can be recorded when shooting with a greater number of image pixels and faster consecutive shooting speed, using a small-scale circuit with low electric power consumption.

Next, the structure of various types of files according to the present embodiment will be described.FIGS. 4A through 4Cillustrate configuration examples of files recorded in the present embodiment, withFIG. 4Aillustrating a reduced RAW file,FIG. 4Ba RAW file, andFIG. 4Ca high-image-quality developed file.

The reduced RAW file400illustrated inFIG. 4Ais recorded in a predetermined recording area of the recording medium152, for example, by the recording/reproducing unit151. The reduced RAW file400includes a header portion401, a metadata portion402, and a compressed data portion403. The header portion401contains identification code indicating that this file is of a reduced RAW file format, and so forth. The compressed data portion403contains compressed data of a reduced RAW image that has been subjected to high-efficiency coding. In a case of a reduced RAW file of a moving image, compressed audio data is also included.

The metadata portion402includes identification information404such as file name of corresponding RAW file, and so forth, generated at the same time as this reduced RAW file, In a case where there is a high-image-quality developed file obtained by the corresponding RAW file having been subjected to high-image-quality developing, this identification information407is stored. Also, if this reduced RAW image has already been developed, information405of the developing status thereof is included. Further, shooting metadata406including evaluation values and subject information calculated and detected at the evaluation value calculation unit105and recognition unit131at the time of shooting, and information from the optical unit101and imaging sensor unit102at the time of shooting (e.g., lens type identification information, sensor type identification information, etc.), is included. Although omitted from illustration, identification code of a recording medium where a RAW file generated at the same time is recorded, path information of a folder where recorded, a thumbnail of the image, and so forth, may further be included.

The RAW file410illustrated inFIG. 4Bis recorded in a predetermined recording area of the recording medium152, for example, by the recording/reproducing unit151. The RAW file410includes a header portion411, a metadata portion412, and a compressed data portion413. The header portion411contains identification code indicating that this file is of a RAW file format, and so forth. The compressed data portion413contains compressed data of a RAW image that has been subjected to high-efficiency coding. In a case of a RAW file of a moving image, compressed audio data is also included.

The metadata portion412includes identification information414such as file name of corresponding reduced. RAW file, and so forth, generated at the same time as this RAW file. In a case where there is a high-image-quality developed file obtained by this RAW image having been subjected to high-image-quality developing, this identification information417is stored. Also, information415of the developing status thereof in the high-image-quality developing is included. Further, shooting metadata416including evaluation values and subject information calculated and detected at the evaluation value calculation unit105and recognition unit131at the time of shooting, and information from the optical unit101and imaging sensor unit102at the time of shooting (e.g., lens type identification information, sensor type identification information, etc.), is included. Although omitted from illustration, identification code of a recording medium where a reduced RAW file generated at the same time is recorded, path information of a folder where recorded, a thumbnail of the image, and so forth, may further be included. Alternatively, the actual data of the reduced RAW file generated at the same time as this RAW file itself may be made into metadata and sorted in the metadata portion412. Further, the actual data of high-image-quality developed file corresponding to this RAW file may itself be made into metadata and stored in the metadata portion412.

The high-image-quality developed file420illustrated inFIG. 4Cis recorded in a predetermined recording area of the recording medium152, for example, by the recording/reproducing unit151. The high-image-quality developed file420includes a header portion421, a metadata portion422, and a compressed data portion423. The header portion421contains identification code indicating that this file is of a high-image-quality developed file format, and so forth. The compressed data portion423contains compressed data of still image portions and moving image portions of a high-image-quality developed file. In a case of a moving image, compressed audio data is also included.

The metadata portion422includes identification information424such as the file name of a reduced RAW file corresponding to this high-image-quality developed file, and so forth, Also, identification information427of the file name of a RAW file corresponding to the high-image-quality developed file is included. Also, information425of the developing status of this high-image-quality developed file in the high-image-quality developing is included. Further, shooting metadata426including evaluation values and subject information calculated and detected at the evaluation value calculation unit105and recognition unit131at the time of shooting, and information from the optical unit101and imaging sensor unit102at the time of shooting (e.g., lens type identification information, sensor type identification information, etc.), is included. Although omitted from illustration, identification code of a recording medium where the corresponding RAW file and reduced RAW file is recorded, path information of a folder where recorded, a thumbnail of the image, and so forth, may further he included.

The above-described file structures according to the present embodiment are only exemplary, and may have structures in common with standards such as Design rule for Camera File system (DCF), Exchangeable Image File format (EXIF), Advanced Video Codec High Definition (AVCHD), or Material eXchange Format (MXF).

An example of high-image-quality developing processing of the imaging apparatus100will be described.FIG. 5is a flowchart relating to developing processing according to the present embodiment. The flowchart inFIG. 5illustrates the processing procedures which the control unit161carries out by controlling the processing blocks, and is realized by loading a program stored in memory (ROM) of the control unit161to memory (RAM), and the CPU executing the program.

InFIG. 5, the control unit161determines whether or not to perform “catch-up developing” according to user settings when in an idle state (S501), and if determination is made not to perform catch-up developing, the flow ends (returns). If determination is made to perform catch-up developing, the flow advances to S520.

The “catch-up developing” according to the present embodiment means that after the shooting operation has ended, a RAW file recorded in the buffer115or recording medium152is read out, and the RAW image is subjected to high-image-quality developing, thereby generating a high-image-quality developed file. This catch-up developing is developing processing performed when the device is idle or in the background of other processing. The name comes from developing processing seeming to chase a RAW file recorded earlier, as if it were trying to catch up. While both still images and moving images may be included in RAW files which are the objet of catch-up developing according to the present embodiment, an example of still images will be described below.

As described above, a recorded RAW file is a high-image-quality file where the image information supplied from the sensor signal processing unit103is not markedly lost, but reproduction display or printing cannot be performed immediately since the RAW file is data before developing, and a request for reproduction display or printing necessities time for RAW developing. Also, a RAW file is not a format in widespread use like PEG or the like, so reproducing environments which can handle RAW tiles are restricted. Accordingly, the catch-up developing according to the present embodiment is a useful function. Upon catch-up developing being performed in the present embodiment, a RAW file that has already been recorded is read out, subjected to high-image-quality developing processing by the high-image-quality developing unit112, and the generated high-image-quality developed still image file is recorded in the recording medium152or the like. This sort of catch-up developing is performed in states where the processing load of the apparatus is relatively low in standby for user operations (when idle), such as in between shootings, when in standby in reproducing mode, in sleep state, or the like. The catch-up developing may be initiated manually, but preferably is designed so that the control unit161automatically executes in the background in a certain state. Due to this configuration, even in a case where there is a request at a later time for reproduction of high-image-quality images, such as displaying to confirm details of an image or printing or the like, there is no delay in the developing processing (reproduction output) each time, and a general operating environment the same as with conventional still image files can be used.

In S520inFIG. 5, the control unit161determines whether the catch-up developing for the RAW file of interest is already processes or unprocessed. Examples of conceivable ways of determining include whether or not a high-image-quality developed file has been created, from identification information in the RAW file410, or the like. Alternatively, the developing status information415in the RAW file410may be referenced to make this determination in the same way. Alternatively again, a table file indicating the state of execution of developing processing of a series of RAW files may be prepared separately and used for determination.

If the control unit161determines that catch-up developing has been competed for all RAW files of interest, the flow ends (returns) there, and the imaging apparatus100transitions to the idle state. If there are any RAW files regarding which the catch-up developing is unprocessed, the flow advances to S521. If a RAW file regarding which catch-up developing is unprocessed has been buffered in the buffer115in S521, the flow advances to S523. If not, the RAW file is read out from the recording medium152or the like in S522, and temporarily held at the buffer115.

The data of the buffer115is updated so that the images which are newer in the order of shooting are held with higher priority. That is to say, images shot in the past are deleted from the buffer in order. Thus, the most recently shot images are always held in the buffer, so S522can be skipped and processing performed at high speed. Further, an arrangement where catch-up developing is performed starting from an image shot immediately prior and going back in time enables images held in the buffer to complete processing with higher priority, so the processing is more efficient.

In S523, the RAW decompression unit114decompresses the RAW file read out from the buffer115or recording medium152and buffered, thereby restoring the RAW image. In S524, the high-image-quality developing unit112executes high-image-quality developing processing on the restored RAW image, and outputs the high-image-quality developed image to the display processing unit122and still image compression unit141via the switch unit121, At this time, a display image may be displayed on the display unit123if the imaging apparatus100is in a state where images developed later can be displayed.

The high-image-quality developing unit112performs de-Bayering processing (demosaicing processing) on the RAW images, i.e., performs color interpolation processing to convert into signals made up of luminance and color difference (or primary color) signals, removes noise included in the signals, corrects optical distortion, and optimizes the image, i.e., performs so-called developing processing. The size (number of pixels) of the developed image generated at the high-image-quality developing unit112is the full-size read out from the imaging sensor unit102, or a size set by the user, so the image quality is much higher than the live view developing image that handles around 2 million pixels. The high-image-quality developing unit112performs each process in higher precision than the live view developing unit111, so a higher quality developed image is obtained as compared to the live view developing unit111, but the processing load is greater. The high-image-quality developing unit112according to the present embodiment has a configuration with increase in circuit scale and electric power consumption suppressed, by avoiding real-time developing processing in parallel with shooting, and enabling developing to be performed taking time.

The image information subjected to developing processing at the high-image-quality developing unit112is supplied to the still image compression unit141or the moving image compression unit142, in the case of a still image, the still image compression unit141handles compression. In S525, the still image compression unit141performs high-efficiency coding processing (still image compression) on the acquired high-image-quality developed image, thus generating a high-image-quality developed file (still image file). Note that the still image compression unit141performs compression processing by a known format such as JPEG or the like. In S526, the recording/reproducing unit151records the high-image-quality developed file in the recording medium152or the like.

If the control unit161determines in S527that the imaging apparatus100is no longer in die idle state, partway through the flow, transition is made to interruption processing. Otherwise, the flow returns to S520. In the processing of S520and thereafter, if there is a RAW image regarding which catch-up developing is unprocessed, the same processing can be repeatedly executed for each image. On the other hand, in a case of interrupting the catch-up processing in S528, information at the point of the control unit161interrupting (RAW file to be interrupted, identification information regarding whether developing completed or not completed, etc.) is stored in memory or the recording medium152as recovery information (interruption processing). At the time of performing catch-up developing, the control unit161references this recovery information so as to resume from the RAW file that was interrupted. After the interruption processing ends, the imaging apparatus100returns to the idle state.

The high-image-quality developed file recorded in S526is of a file structure such as illustrated inFIG. 4C. The information such as the file name of the original RAW file for this high-image-quality developed file is written to the metadata portion422. Information425is also described regarding the fact that this high-image-quality developed file has been subjected to high-image-quality developing by the high-image-quality developing unit112, and the developing status indicating the content of this developing. Shooting metadata426including evaluation values and subject information calculated and detected by the evaluation value calculation unit105and recognition unit131, and information at the time of shooting from the optical unit101and imaging sensor unit102, extracted from the metadata of the original RAW file, is also copied. Further, the recording/reproducing unit151updates information for each metadata portion of the original RAW file for the catch-up developing, and the reduced RAW file thereof as well, to the newest information relating to the generated high-image-quality developed file.

An arrangement where the recording/reproducing unit151records the new high-image-quality developed file recorded in S526after the high-image-quality developing, using a file name similar or correlated with the original RAW file, facilitates identification. For example, a file name where only part of the file name has been changed (e.g., suffix or ending characters) while the rest of the file name is the same, is desirable.

Thus, the imaging apparatus100according to the present embodiment executes catch-up developing in states where the processing load of the apparatus is relatively low in standby for user operations (when idle), such as in between shootings, when in standby in reproducing mode, in sleep state, or the like. High-image-quality developed files are sequentially created from the RAW files. Thus, even when receiving a request for reproduction of a high-image-quality image, such as for confirmation display of detailed portions of the image or for printing, there is no delay due to developing processing (reproduction output) each time this occurs, and the files can be used in a general operating environment in the same way as conventional still image files.

Next, operations relating to the still image reproducing mode and moving image reproducing mode of the imaging apparatus100will be described.FIG. 6is a flowchart relating to reproducing processing of the present embodiment. The flowchart inFIG. 6illustrates the processing procedures which the control unit161carries out by controlling the processing blocks, and is realized by loading a program stored in memory (ROM) of the control unit161to memory (RAM), and the CPU executing the program.

In S601inFIG. 6, the control unit161determines whether or not to stop reproduction processing. In a case of stopping reproduction processing, the imaging apparatus100returns to the idle state. Otherwise, the flow advances to S602.

In S602, the control unit161reads out a reduced RAW image file which is the object of reproduction processing, and determines whether or not to reproduce the reduced RAW image. RAW files are high-resolution, and accordingly take time to develop as described above. Further, there is a possibility that a high-image-quality developed file has not net been generated, so when reproducing, the control unit161reproduces RAW images with higher priority than others. A reduced RAW image has around2million pixels, so high-speed processing can be performed in live view developing in the same way, sudden requests for reproducing can be immediately handled, and a reproduced image can be quickly output in quick response. Note however, that a reduced RAW image has limited image quality. While this is effective for general confirmation of the image, it may be insufficient for usages such as confirming details of the image or printing. Accordingly, The imaging apparatus100also executes reproduction processing of other images depending on usage, as described below.

In a case of reproducing a reduced. RAW image, in S620the control unit161reproduces the reduced RAW file to be reproduced, from the buffer115or recording medium152or the like. In S621, the RAW decompression unit114decompresses the compressed reduced RAW image obtained from the reproduced reduced RAW file, Further, in S622the high-image-quality developing unit112develops the decompressed reduced RAW image to generate a reproduction image, which is supplied to the display processing unit122, While developing of the reduced RAW image has been described as being performed by the high-image-quality developing unit112in S622, this may be performed by the live view developing unit111instead.

In a case of not reproducing the reduced RAW image, in S603the control unit161determines whether or not to reproduce a high-image-quality developed image. A condition for reproducing a high-image-quality developed image is that the above-described catch-up developing has been performed, or that high-image-quality developing has already been completed by reproduction of the RAW file in response to a user request. In a case of reproducing a high-image-quality developed image, in S630the control unit161reproduces the high-image-quality developed file to be reproduced from the recording medium152or the like. In S631, the still image decompression unit143or the moving image decompression unit144decompresses the compressed high-image-quality developed image obtained from the reproduced high-image-quality developed file to generate a reproduction image (still image or moving image), which is supplied to the display processing unit122.

In a case where no reduced RAW tile is reproduced and no high-image-quality developed image is reproduced, the RAW file is reproduced. Examples of usage environments of reproduction of a RAW file in the present embodiment will be described with reference toFIGS. 7A, 7B, and 7C.FIGS. 7A, 7B, and 7Care diagrams illustrating examples of display processing of the present embodiment.FIGS. 7A, 7B, and 7Care examples of image displays at different timings.

Display example700inFIG. 7Ais an example of reduced display of six images, denoted by reference numeral701, on the display unit123in a reduced manner. This display state is a list display. Display example710inFIG. 7Bis an example of display of one image711on the entire display unit123. This display state is a normal display. Display example720inFIG. 7Cis an example of display of an enlarged portion of one image721on the entire display unit123. This display state is an enlarged display. The enlarged display assumes usage where part of a subject image is displayed in an enlarged manner such as in the display example720, to confirm details of an image that has been shot, partially extract (trimming), or check whether in focus or not, for example.

Resolution is sufficient for display of the reproduced image obtained from the reduced RAW image in the state of display examples700or710. However, resolution of a reduced RAW image is insufficient in a case of enlarged display such as in display example720(leads to poor resolution), so a RAW file that has high resolution is prefer reproduced and displayed. In a case of performing enlarged display of the reproduced image, the flow transitions to RAW file reproduction in S604and thereafter.

In a case of reproducing a RAW image, in S604the control unit161determines whether or not the RAW file to be reproduced has been buffered in the buffer115. In a case where the RAW file has been buffered, the flow advances to S606, and if not buffered, to S605. In S605, the control unit161reads out the RAW file to be reproduced from the recording medium152or the like, and buffers in the buffer115.

The data of the buffer115is updated so that the images which are newer in the order of shooting in the shooting mode are held with higher priority. That is to say, images shot in the past are deleted from the buffer in order. Thus, the most recently shot images are always held in the buffer, so S605can be skipped and processing performed at high speed.

In S606, the RAW decompression unit114obtains the buffered RAW file, and decompresses the compressed RAW image obtained from this RAW file. Further, in S607the high-image-quality developing unit112performs high-image-quality developing of the decompressed RAW image to generate a reproduction image, which is supplied to the display processing unit122. Note that the imaging apparatus100can also newly create a high-image-quality developed file corresponding to the RAW file to be reproduced by the high-image-quality developing in S607.

In S608, the display processing unit122outputs one of the reproduction images according to the object of reproduction to the display unit123for display. The display form is as illustrated inFIGS. 7A through 7C. The display processing unit122can also output the display image from the video output terminal124to an external apparatus. Upon the displaying in S608ending, the flow returns to S601.

It can be assumed that the RAW tile reproduction from S604and thereafter inFIG. 6may occur at a timing where catch-up developing has not the been performed. This conversely means that even in a case of enlarged display that requires high image quality, the high-image-quality developed image can be provided without newly developing the RAW file as long as the high-image-quality developed file has been created. Catch-up developing is performed in the present embodiment in states where the processing load of the apparatus is relatively low in standby for user operations (when idle), such as in between shootings, when in standby in reproducing mode, in sleep state, or the like. Also, a high-image-quality developed image can be created when reproducing a RAW file under user instructions, high-image-quality developed files are sequentially created from RAW tiles in this way. The more high-image-quality developing is performed beforehand, the lower the frequency of high-image-quality developing occurring at the point of requesting an enlarged display, so the faster high-image-quality images can be output for enlarged displays, and the better the operability is anticipated to be.

Description has been made above that, in a case where the RAW file is buffered in the buffer115, S605can be skipped, so the image can be displayed faster. Accordingly, when performing display of the display examples700and710inFIGS. 7A and 7B, the RAW files corresponding to the images701and711are prefer read out from the recording medium beforehand and loaded to the buffer115beforehand, so that RAW files are held in the buffer115as much as possible. An arrangement where the recording/reproducing unit151reads out the corresponding RAW file from the recording medium152or the like and buffers before an instruction to enlarge enables display to be made at an even faster response speed when an instruction for an enlarged display such as in the display example720is made.

Next, operations relating to the editing mode of the imaging apparatus100will be described,FIG. 8is a block diagram illustrating a configuration example of an editing apparatus (external apparatus) that performs editing processing.FIGS. 9A and 9Bare flowcharts relating to editing processing and editing execution of the present embodiment. The flowcharts inFIGS. 9A and 9Billustrate the processing procedures which the control unit161carries out by controlling the processing blocks, and is realized by loading a program stored in memory (ROM) of the control unit161to memory (RAM), and the CPU executing the program.

An editing apparatus800inFIG. 8is an external apparatus from the imaging apparatus100, more specifically, an external storage or server, or a mobile information terminal (tablet, smartphone, etc.), personal computer (PC), or the like. The editing apparatus800can exchange various types of files and related information with the imaging apparatus100. A communication unit802has a configuration which enables it to access the imaging apparatus100via the Internet or by direct communication between apparatuses, by wireless or cabled communication using a communication terminal801.

At the time of performing editing processing or beforehand, the editing apparatus800acquires a reduced RAW file from the external imaging apparatus100that is connected over a network via the communication unit802. The received reduced RAW file is saved in a recording medium803via an internal bus811. The internal bus811is connected with the parts within the editing apparatus800, and serves as a data bus and system bus.

The reduced RAW file is decompressed developed, and image information obtained thereby is used in the editing processing. A RAW decompression unit804reads out a desired reduced. RAW file saved in the recording medium803, and decompresses the compressed reduced RAW image. A RAW developing unit805performs developing processing of the decompressed reduced RAW image. The image obtained by the developing processing from the reduced RAW image is displayed on a display unit809.

A control unit808includes a CPU, and unshown memory storing application programs and control programs which the CPU executes. The control unit808controls the editing processing, which is executed accepting editing instructions from a user monitoring an image displayed on the display unit809, under control of an editing application program executed by the control unit808. Editing instructions are input from the user by a user interface unit807. The user interface unit807uses an operating unit realized by a touch panel, mouse, keyboard, dedicated or general-purpose console, or the like for example, and an editing application that provides display information.

The displayed reduced RAW image is subjected to trimming, compositing, and so forth, according to editing instructions from the user interface unit807. If the file is a moving image, instructions that are applied may also include optional scene selection by cut-in/cut-out being specified, effects being applied and so forth. An editing information generating unit806generates editing information according to the contents of editing. The editing information is the contents of editing which the image obtained from the reduced RAW image has been subjected to, in the form of data information. The editing information has described therein whether or not editing has been applied to each frame of still images or moving images, and the contents of editing. The editing information may further include actual data of the reduced RAW image which has been subjected to the editing processing. This editing information is recorded in the recording medium803, and also transmitted to the imaging apparatus100in response to requests from the imaging apparatus100.

Next, the flows of editing processing and editing execution by the imaging apparatus100will he described with reference toFIGS. 9A and 9B.FIG. 9Ais an editing processing flow, andFIG. 9Bis an editing execution flow. Upon editing processing starting, in S901ofFIG. 9Athe control unit161determines whether or not to stop editing processing. In a case of stopping editing processing, the imaging apparatus100returns to the idle state. In a case of continuing editing processing, the flow advances to S902.

In S902, the control unit161transmits the reduced RAW file corresponding to the

RAW file to be edited, to the editing apparatus800which is an external apparatus, as editing data. In S903, the control unit161issues an editing command to the editing apparatus800. The editing command is transmitted to the control unit808, and serves as an editing information generating instruction to the editing apparatus800. After S903, the flow returns to S901.

Upon receiving the aforementioned editing command, the editing apparatus800performs the editing processing described above, thereby generating editing information. The control unit161can start the editing execution flow inFIG. 9Bunder user instructions, under the condition that generating of the editing information has ended at the editing apparatus800.

Upon editing execution being started, in S911inFIG. 9Bthe control unit161determines whether or not to stop editing processing. In a case of stopping editing processing, the imaging apparatus100returns to the idle state. In a case of continuing editing execution, the flow advances to S912.

In S912, the control unit161receives from the editing apparatus800the editing information corresponding to the RAW file to be edited, using the communication unit153. In S913, the control unit161references this editing information to apply editing or reproduction according to the content described in the received editing information as to the RAW file or reduced RAW file within the imaging apparatus100. That is to say, the contents of editing on the reduced RAW image performed at the editing apparatus800are reproduced on the original RAW image or reduced RAW image within the imaging apparatus100. After S913, the flow returns to S911. The RAW image subjected to editing execution in this way may be saved as a file where the original file has been updated, or the original image and editing information may each be saved separately and the editing information being reflected when reproducing the image, i.e., edited, each time the image is reproduced.

In this way, the reduced RAW image is used as editing data or as a proxy for the RAW image, whereby the process of editing be streamlined and capabilities increased by this configuration of editing using an external apparatus. Moreover, the overall load on the system regarding editing processing can be reduced.

Second Embodiment

FIG. 10is a block diagram illustrating a configuration example of an image processing apparatus according to a second embodiment of the present invention.FIG. 10illustrates an imaging apparatus1000as the image processing apparatus according to the present embodiment. The imaging apparatus1000not only records image information obtained by imaging a subject in a recording medium, but also has functions of reproducing image information from a recording medium, performing developing processing, and displaying, and functions of exchanging image information with an external apparatus, server (cloud), or the like, and performing editing. Accordingly, the image processing apparatus according to the present embodiment is not restricted to being expressed as being an imaging apparatus, and may also referred to as being a recording apparatus, reproducing apparatus, recording/reproducing apparatus, communication apparatus, editing apparatus, image processing system, editing system, and so forth.

Configurations of the imaging apparatus1000according to the present embodiment which are the same as those of the imaging apparatus100in the first embodiment described above will be denoted with the same reference numerals, and description thereof will be omitted.

InFIG. 10, the imaging apparatus1000has an imaging module170serving as an imaging unit, and an image processing module180serving as an image processing unit, the imaging module170and the image processing module180being connected by a bus107. The RAW data and the like obtained by imaging within the imaging module170is supplied to the image processing module180via the bus107. The imaging module170includes the imaging sensor unit102, sensor signal processing unit103, camera control unit104, and encoding unit106. The image processing module180includes a decoding unit108, RAW reduction unit109, developing unit110, live view developing unit111, high-image-quality developing unit112, switch unit121, RAW compression unit113, RAW decompression unit114, and buffer115. The image processing module180further includes the evaluation value calculation unit105, recognition unit131, display processing unit122, still image compression unit141, moving image compression unit142, still image decompression unit143, moving image decompression unit144, recording/reproducing unit151, and control unit161. The imaging apparatus1000also has the optical unit101, display unit123, video output terminal124, communication unit153, communication terminal154, and built-in or detachable recording medium152.

Configurations of the imaging apparatus1000according to the present embodiment which differ from those of the imaging apparatus100according to the first embodiment described above, will be described in detail. The imaging module170internally has the encoding unit106. This exists to compress RAW data, to reduce transmission of the massive RAW data from overwhelming the communication band over the bus107. Examples of encoding processing (compression processing) which can be applied at the encoding unit106include compression by Differential Pulse Code Modulation (DPCM) and Golomb coding. This method reduces the amount of information of pixels in pixel data subjected to DPCM processing by Golomb coding of difference values among the pixel data thereof. Alternatively, compression may be performed where unnecessary high-band components of the pixel data are deleted using one-dimensional discrete cosine transform (DCT). The rate of compression may be fixed when carrying out either method, or May be designed to be adjustable according to user instructions or the shooting mode.

Pixel data obtained by the imaging sensor unit102converting the subject image when shooting is subjected to pixel restoration processing by the sensor signal processing unit103. This restoration processing involves processing of handling missing or unreliable pixels at the imaging sensor unit102by performing interpolation using nearby pixel values for such pixels to be restored, and subtracting a predetermined offset value. This may be altered so that part or all of this restoration processing is not performed here but later when developing.

The encoding unit106performs encoding processing of the pixel data supplied from the sensor signal processing unit103according to an aforementioned method. The pixel data in the state of having been compressed in the encoding processing is transmitted to the image processing module180via the bus107. The compressed pixel data is decoded by the decoding unit108disposed at the input portion of the image processing module180. The decoding unit108subjects inverse transform from the encoding unit106upstream, to decode the compressed pixel data.

The pixel data that has been compressed by the encoding unit106and further decoded by the decoding unit108is referred to as a RAW image, meaning a raw (pre-development) image in the present embodiment. A RAW image that has been subjected to compression can still be handled as a high-quality RAW image, in the same way as in the first embodiment.

Thereafter, the RAW image is supplied to the RAW compression unit113and compressed again, for efficient recording. The RAW image is reduced in size by the RAW reduction unit109to effectively perform display and reproduction. The RAW reduction unit109resizes the input RAW images to HD size (equivalent to around 2 million pixels), for example, thereby generating a reduced RAW image. Processing of the RAW image and reduced RAW image thereafter is the same as in the first embodiment.

Note that the decoding unit108and RAW decompression unit114in the image processing module180, which both perform decompression/decoding of compressed RAW images may he realized by a circuit where part or all of the processing circuits of the two are shared or integrated.

The shooting processing, developing processing, reproduction processing, editing processing, and editing executing processing performed at the imaging apparatus1000according to the present embodiment, and the configurations of the files created thereby, are the same as in the first embodiment.

Thus according to the present embodiment, the same functions as the first embodiment can be realized, while improving the transmission efficiency of RAW data within the imaging apparatus1000.

While the first and second embodiments have been described above, the present invention is not restricted to these embodiments, rather, various modifications may be made by circuit arrangements as suitable without departing from the technical idea of the present invention.

Other Embodiments

This application claims the benefit of Japanese Patent Application No. 2014-186870, filed Sep. 12, 2014, which is hereby incorporated by reference herein in its entirety.