CONTROL APPARATUS, LENS APPARATUS, IMAGE PICKUP APPARATUS, CAMERA SYSTEM, CONTROL METHOD, AND STORAGE MEDIUM

A control apparatus for use in a camera system that includes a first correction member provided on one of an image pickup apparatus and a lens apparatus detachable from the image pickup apparatus, and a second correction member provided on the other of the image pickup apparatus and the lens apparatus includes a memory storing instructions, and a processor configured to execute the instructions to control image stabilization using first control configured to provide image stabilization based on a first correction ratio between image stabilization using the first correction member and image stabilization using the second correction member and to electronically correct a trapezoidal distortion in an image blur, or second control configured to provide image stabilization based on a second correction ratio between image stabilization using the first correction member and image stabilization using the second correction member without electronically correcting the trapezoidal distortion in an image blur.

BACKGROUND

Technical Field

One of the aspects of the embodiments relates to a control apparatus, a lens apparatus, an image pickup apparatus, a camera system, a control method, and a storage medium.

Description of Related Art

Camera systems have conventionally been proposed that perform image stabilization using both image stabilization (OIS) that moves an image stabilizing lens provided in an interchangeable lens and image stabilization (IIS) that moves an image sensor provided in a camera (body). Japanese Patent No. 6410431 discloses a configuration for setting an image stabilizing ratio between OIS and IIS so as to effectively utilize image stabilizing ranges of both OIS and IIS.

Depending on the optical system in the interchangeable lens that adopts the central projection method, an image point moving amount that occurs on an object image during camera shake may differ between a central portion and a peripheral portion of the image. In particular, as the optical system becomes a wide angle, the image point moving amount in the peripheral portion becomes larger than that in the central portion. Therefore, after the image stabilization is performed, the peripheral portion may remain blurred more than the central portion. Japanese Patent Laid-Open No. 2018-173632 discloses a configuration for setting an image stabilizing ratio in consideration of a difference between an image blur amount in the central portion and an image blur amount at an arbitrary image point position, which is generated by the central projection method.

Japanese Patent No. 6103877 discloses a tilt correction configuration, which is one of the so-called electronic image stabilizations, for canceling, through image transformation, image blur, in which the image is distorted into a trapezoidal shape due to vibration applied to the image pickup apparatus.

In the configurations of Japanese Patent No. 6410431 and Japanese Patent Laid-Open No. 2018-173632, the image stabilizing degree in a peripheral portion of an image is different depending on a difference in the setting of an image stabilizing ratio. In a case where tilt correction is additionally performed, proper tilt correction may not be able to be performed unless the operation is changed in accordance with the method of setting the image stabilizing ratio.

SUMMARY

A control apparatus according to one aspect of the embodiment for use in a camera system that includes a first correction member provided on one of an image pickup apparatus and a lens apparatus attachable to and detachable from the image pickup apparatus, and a second correction member provided on the other of the image pickup apparatus and the lens apparatus includes a memory storing instructions, and a processor configured to execute the instructions to control image stabilization using first control configured to provide image stabilization based on a first correction ratio between image stabilization using the first correction member and image stabilization using the second correction member and to electronically correct a trapezoidal distortion in an image blur, or second control configured to provide image stabilization based on a second correction ratio between image stabilization using the first correction member and image stabilization using the second correction member without electronically correcting the trapezoidal distortion in an image blur. A lens apparatus, an image pickup apparatus, and a camera system each having the above control apparatus also constitute another aspect of the embodiment. A control method corresponding to the above control apparatus also constitutes another aspect of the embodiment.

DESCRIPTION OF THE EMBODIMENTS

In the following, the term “unit” may refer to a software context, a hardware context, or a combination of software and hardware contexts. In the software context, the term “unit” refers to a functionality, an application, a software module, a function, a routine, a set of instructions, or a program that can be executed by a programmable processor such as a microprocessor, a central processing unit (CPU), or a specially designed programmable device or controller. A memory contains instructions or programs that, when executed by the CPU, cause the CPU to perform operations corresponding to units or functions. In the hardware context, the term “unit” refers to a hardware element, a circuit, an assembly, a physical structure, a system, a module, or a subsystem. Depending on the specific embodiment, the term “unit” may include mechanical, optical, or electrical components, or any combination of them. The term “unit” may include active (e.g., transistors) or passive (e.g., capacitor) components. The term “unit” may include semiconductor devices having a substrate and other layers of materials having various concentrations of conductivity. It may include a CPU or a programmable processor that can execute a program stored in a memory to perform specified functions. The term “unit” may include logic elements (e.g., AND, OR) implemented by transistor circuits or any other switching circuits. In the combination of software and hardware contexts, the term “unit” or “circuit” refers to any combination of the software and hardware contexts as described above. In addition, the term “element,” “assembly,” “component,” or “device” may also refer to “circuit” with or without integration with packaging materials.

Referring now to the accompanying drawings, a detailed description will be given of embodiments according to the disclosure. Corresponding elements in respective figures will be designated by the same reference numerals, and a duplicate description thereof will be omitted.

First Embodiment

FIG.1is a block diagram of a camera system according to one embodiment. The camera system includes a camera body (image pickup apparatus)100and an interchangeable lens101detachably and communicatively attached to the camera body100.

The camera body100includes a camera MPU102, an operation unit103, an image sensor104, a camera-side contact terminal105, a camera-side gyro sensor106, an image sensor actuator107, an image sensor position sensor (IS position sensor)108, an acceleration sensor109, and a rear display116.

The camera MPU102is a controller that governs overall control of the camera system, and controls various operations such as auto-exposure (AE), autofocus (AF), and imaging according to input from the operation unit103. The camera MPU102communicates various commands and information with the lens MPU110provided in the interchangeable lens101through the camera-side contact terminal105and a lens-side contact terminal112provided to the interchangeable lens101. The camera-side contact terminal105and the lens-side contact terminal112include power terminals for supplying power from the camera body100to the interchangeable lens101.

The operation unit103includes a mode dial for setting various imaging modes, a release button for instructing an imaging preparation operation and imaging start, and the like. In a case where the release button is half-pressed, a first switch (SW1) is turned on, and in a case where the release button is fully pressed, a second switch (SW2) is turned on. In a case where SW1is turned on, AE and AF are performed as an imaging preparation operation, and in a case where SW2is turned on, the AE setting is finalized, the AF is stopped, and the like, and an instruction to start imaging (exposure) is issued (SW2-1is turned on). A predetermined time after the instruction, actual exposure is started (SW2-2is turned on). SW2-1and SW2-2are turned off when the set exposure time has passed and the imaging is finished. The turning on and off of SW1, SW2-1, and SW2-2are notified from the camera MPU102to the lens MPU110through communication.

The image sensor104includes a photoelectric conversion element such as a CCD sensor or a CMOS sensor, and photoelectrically converts an object image formed by an imaging optical system, which will be described below, to generate an imaging signal. The camera MPU102generates a video (or image) signal using the imaging signal from the image sensor104.

The camera-side gyro sensor106detects angular shake (camera shake) applied to the camera body100due to camera shake or the like, and outputs a camera shake detection signal as an angular velocity signal. The camera MPU102drives the image sensor actuator107based on the camera shake detection signal and the image stabilizing ratio for ITS, which will be described below, and thereby moves the image sensor104in a direction including a component of a direction orthogonal to the optical axis of the imaging optical system. Thereby, image blur caused by camera shake can be reduced (corrected). At this time, the camera MPU102performs feedback control of the image sensor actuator107so that the position of the image sensor104detected by the image sensor position sensor108(a moving amount from the position on the optical axis, which is a moving center) approaches the target position. Due to the above configuration, image stabilization (referred to as IIS hereinafter) that moves the image sensor104is performed. The camera MPU102performs IIS and so-called electronic image stabilization, which electronically transforms an image using a signal from the camera-side gyro sensor106, and provides translational transformation (deformation), rotational transformation, and trapezoidal transformation (tilt correction).

The acceleration sensor109detects the orientation of the camera body100and shake (shift shake) that is difficult to detect with the camera-side gyro sensor106.

The rear display116displays an image corresponding to the image signal generated by the camera MPU102using the imaging signal from the image sensor104. Prior to imaging, the user can observe a displayed image as a finder image (live-view image). After imaging, a still image or moving image for recording generated by imaging can be displayed on the rear display116. “imaging” in this embodiment means imaging for recording.

The interchangeable lens101includes the imaging optical system including an image stabilizing lens114, a lens MPU110, a lens-side gyro sensor111, the lens-side contact terminal112, a lens actuator113, and a lens position sensor115.

The lens-side gyro sensor111detects angular shake (lens shake) applied to the interchangeable lens101and outputs a lens shake detection signal as an angular velocity signal. The lens MPU110drives the lens actuator113based on the lens shake detection signal and an image stabilizing ratio for OIS, which will be described below, so that the image stabilizing lens114is moved in a direction including a component in a direction orthogonal to the optical axis of the imaging optical system. Thereby, image blur caused by lens shake can be reduced (corrected). At this time, the lens MPU110performs feedback control of the lens actuator113so that the position of the image stabilizing lens114detected by the lens position sensor115(moving amount from the position on the optical axis, which is a moving center) approaches the target position. Due to the above configuration, image stabilization (hereinafter referred to as OIS) that moves the image stabilizing lens114is performed.

The lens MPU110includes a correction unit (first correction unit)110aand a control unit110bin this embodiment. The correction unit110aperforms image stabilization with the image sensor104using at least one of the image sensor104and the image stabilizing lens114based on a first image stabilizing ratio (first correction ratio) between OIS and IIS or a second image stabilizing ratio (second correction ratio) between OIS and ITS. The first image stabilizing ratio is determined based on a movable amount of the image sensor104and a movable amount of the image stabilizing lens114. The second image stabilizing ratio is determined based on information on an image moving amount between a central portion and a peripheral portion of an image in a case where OIS is performed and information on an image moving amount between a central portion and a peripheral portion in a case where IIS is performed. The control unit110bexecutes the first control or the second control. The first control performs image stabilization based on the first image stabilizing ratio by the correction unit110a, and an image transformation processing unit (second correction unit)217performs image stabilization, in particular, corrects blur that is distorted in a trapezoidal shape, trapezoidal distortion. The second control performs image stabilization based on the second image stabilizing ratio by the correction unit110awithout correcting trapezoidal distortion by the image transformation processing unit217.

Referring now toFIG.2, a description will be given of the configuration of the image stabilizing unit in the camera system.FIG.2is a block diagram of the image stabilizing unit. The image stabilizing unit includes an image stabilizing unit201that is part of the camera MPU102and an image stabilizing unit209that is part of the lens MPU110.

The image stabilizing unit201includes a camera gyro offset remover202, a camera-side angle converter203, a camera information memory204, a lens communication transmitter205, a lens communication receiver206, a camera-side cooperative driving control unit207, and an image sensor driving control unit208. The camera gyro offset remover202removes an offset component from the angular velocity signal detected by the camera-side gyro sensor106. The camera-side angle converter203converts the angular velocity signal output from the camera gyro offset remover202into an angle signal. The camera information memory204stores information such as a stroke (movable amount) of the image sensor104, which is an IIS correction member, and a sensor size. A lens communication transmitter205transmits information stored in the camera information memory204to the camera communication receiver214in the image stabilizing unit209. The lens communication receiver206receives information transmitted from the camera communication transmitter213in the image stabilizing unit209. The camera-side cooperative driving control unit207determines an image blur amount to be corrected by IIS based on the information stored in the camera information memory204and the information received by the lens communication receiver206. The image sensor driving control unit208generates a driving control signal for the image sensor104using the angle signal output from the camera-side angle converter203and the image blur amount output from the camera-side cooperative driving control unit207. The output from the camera gyro offset remover202is sent to an image transformation processing unit217which is a separate processing block in the camera MPU102, and the image transformation processing unit217executes image transformation processing for an image using the output from the camera gyro offset remover202.

The image stabilizing unit209includes a lens gyro offset remover210, a lens-side angle converter211, a lens information memory212, a camera communication transmitter213, a camera communication receiver214, a lens-side cooperative driving control unit215, and an image stabilizing lens driving control unit216. The lens gyro offset remover210removes an offset component from the angular velocity signal detected by the lens-side gyro sensor111. The lens-side angle converter211converts the angular velocity signal output from the lens gyro offset remover210into an angle signal. The camera communication transmitter213transmits information stored in the lens information memory212to the lens communication receiver206in the image stabilizing unit201. The camera communication receiver214receives information transmitted from the lens communication transmitter205in the image stabilizing unit201.

The lens information memory212stores information about an image stabilizing ratio and a stroke of the image stabilizing lens114, which is an OIS correction member. The lens information memory212also stores information about an image stabilization angle in a case where the image sensor104moves by a predetermined amount, that is, the camera image stabilization sensitivity.

The lens information memory212stores lens peripheral image blur residue amount information, which is an image stabilization residue (remaining) amount in a peripheral portion of an image in a case where the image stabilizing lens114performs image stabilization in a central portion of an image by a predetermined angle. The lens peripheral image blur residue amount may be information about an image point moving amount at a central portion of an image and an image point moving amount at a predetermined image height in a case where the image stabilizing lens114is moved by a predetermined amount, or may be obtained from information about an image stabilization angle in a case where the image stabilizing lens114is moved by a predetermined amount. The lens peripheral image blur residue amount may be a function that indicates how the image stabilization residue amount in a peripheral portion of an image changes for each image height in a case where image stabilization is performed for the central portion of the image by a predetermined angle.

The lens information memory212stores a camera peripheral image blur residue amount, which is an image stabilization residue amount in a peripheral portion of an image in a case where the image sensor104performs image stabilization for the central portion of the image by a predetermined angle. The camera peripheral image blur residue amount may be obtained from information about the image point moving amount in the central portion of the image and the image point moving amount at a predetermined image height in a case where the camera body100is rotated by a predetermined amount. The camera peripheral image blur residue amount may be a function that indicates how the image stabilization residue amount in the peripheral portion of the image changes for each image height in a case where image stabilization is performed for the central portion of the image by a predetermined angle.

The information stored in the lens information memory212may be information that is switched by moving the zoom lens or focus lens included in the imaging optical system.

The lens-side cooperative driving control unit215performs cooperative driving control of OIS and IIS using the information stored in the lens information memory212and the information received by the camera communication receiver214. At this time, the lens-side cooperative driving control unit215sets an image stabilizing ratio for determining an image blur amount to be corrected by each image stabilizing unit. The lens-side cooperative driving control unit215determines whether to switch the image stabilizing ratio and lens internal information about sensitivity necessary for IIS using the information stored in the lens information memory212and the information received by the camera communication receiver214. The image stabilizing lens driving control unit216generates a driving control signal for the image stabilizing lens114using the angle signal output from the lens-side angle converter211.

Referring now toFIG.3, a description will be given of a method of setting the first image stabilizing ratio of OIS and IIS. The image stabilizing ratio can be set from a plurality of methods depending on its purpose.FIG.3illustrates a method of setting the first image stabilizing ratio using a stroke ratio of OIS and IIS correction members. A range301is a mechanically movable range of the image sensor104, which is the IIS correction member. A range302is an image circle of the interchangeable lens101and a movable range of the image stabilizing lens114, which is the OIS correction member. A range303is a stroke in the Y-axis direction (OIS stroke) of the image stabilizing lens114. The image sensor104is to move within the ranges301and302. A range304is a stroke in the Y-axis direction (IIS stroke) of the image sensor104. In image stabilization for the entire camera system using two correction members, setting the first image stabilizing ratio illustrated below can effectively utilize each stroke of the OIS and IIS correction members.

A description will now be given of a method of setting the second image stabilizing ratio for suppressing an image blur residue amount in a peripheral portion of an image. Since distortion remains in a general optical system, a difference in an image point moving amount occurs between a central portion and a peripheral portion of an image due to changes in a distortion amount in a case where IIS is performed. Also in OIS, eccentric distortion occurs due to the eccentricity of the lens, and the fluctuation of the eccentric distortion causes a difference in the image point moving amount between the central portion and the peripheral portion of the image. Therefore, in IIS, the influence of the projection method and the influence of distortion cause a difference in the image point moving amount between the central portion and the peripheral portion of the image, and in OIS, the influence of the projection method and the influence of eccentric distortion cause a difference in the image point moving amount between the central portion and the peripheral portion of the image. In other words, since the cause of the difference in the image point moving amount between the central portion and the peripheral portion of the image is different between OIS and IIS, the ratio of the image point moving amount between the central portion and the peripheral portion of the image is different between OIS and IIS.

FIG.4illustrates an image blur residual amount for each image height in a case where a predetermined image stabilizing amount is performed by OIS and IIS and an image blur residual amount in the central portion of the image is set to 0. As illustrated inFIG.4, in a case where a ratio of an image point moving amount is different between a central portion and a peripheral portion of an image, image stabilization may be performed by using one of the correction members preferentially so as to reduce an image blur residual amount.

Accordingly, as illustrated inFIG.5, a method of setting the second image stabilizing ratio is conceivable in a case where the entire camera system has an image stabilization amount “a.” In a case where the absolute value of the image stabilization amount “a” is smaller than a predetermined threshold TH1, the image stabilizing ratio for OIS is set to A (=1), and the image stabilizing ratio for IIS is set to 1−A (=0). In a case where the absolute value of the image stabilization amount a is equal to or larger than the predetermined threshold TH1and smaller than a predetermined threshold TH2, the image stabilizing ratio for OIS is set to B, the image stabilizing ratio for IIS is set to 1−B, and the image stabilizing ratios of OIS and IIS are gradually adjusted. In a case where the absolute value of the image stabilization amount “a” is equal to or larger than the predetermined threshold TH2, the image stabilizing ratio for OIS is set to C (=0) and the image stabilizing ratio for IIS is set to 1−C (=1). Thus, the image stabilizing ratios can be set so as to reduce the image blur residue amount in the peripheral portion in the image by preferentially using a smaller image point moving amount of the peripheral portion to the central portion of the image more effectively than the method of setting the image stabilizing ratios based on the stroke ratio. However, the method illustrated inFIG.5may cause image blur residual amounts in the peripheral portion of the image to be different among regions corresponding to the image stabilizing ratios A, B, and C and increase the calculation load more than the method of setting the image stabilizing ratios based on the stroke ratio.

Electronic image stabilization will now be described. Electronic image stabilization is generally performed by transforming an image using geometric transformation such as projective transformation. For example, as illustrated in FIG.6, in a case where orientation A for capturing a certain object changes to orientation B, point P1on image I1in orientation A is converted into point P2on image I2in orientation B using determinant H. As illustrated inFIG.7, the types of projection transformation include translation, rotation, tilt, and the like, and each transformation can be made by setting each parameter of the determinant. Here, the translational and rotational correction amounts can be calculated from the output of the camera-side gyro sensor106and the focal length information of the interchangeable lens101. The tilt correction amount can be calculated from the translational amount and the focal length information. In order to simplify the calculation, equations are often constructed on the premise that each projection transformation correction amount changes in proportion to the image blur applied to the camera system. Therefore, in a case where the second image stabilizing ratio is set to suppress the image blur residue amount in the peripheral portion of the image, the image blur residual amount in the peripheral portion nonlinearly changes according to the image blur amount applied to the camera system. If the electronic image stabilization is performed in such a state, tilt correction, in particular, may not correctly correct trapezoidal distortion.

Referring now toFIG.8, a description will be given of cooperative driving control of OIS and IIS according to this embodiment. This embodiment assumes that the lens MPU110executes the cooperative driving control.FIG.8is a flowchart illustrating cooperative driving control of OIS and IIS according to this embodiment. After the camera system is powered on and the initial driving operation and the like are performed, this flow is started.

In step S801, the lens MPU110acquires information necessary for the cooperative driving control. The information necessary for the cooperative driving control includes the strokes of the OIS and IIS correction members, a camera peripheral image blur residue amount, a lens peripheral image blur residue amount, information on the sensitivity required for image stabilization, information on whether to perform electronic image stabilization by the image transformation processing unit217, and the like.

In step S802, the lens MPU110determines whether or not tilt correction is to be performed in the electronic image stabilization. In a case where it is determined to perform tilt correction, the process of step S803is executed, and in a case where it is determined not to perform tilt correction, the process of step S804is executed.

In step S803, the lens MPU110selects the method of setting the first image stabilizing ratio based on the stroke ratio.

In step S804, the lens MPU110selects the method for setting the second image stabilizing ratio for suppressing the image blur residue amount in the peripheral portion of the image.

In step S805, the lens MPU110sets the image stabilizing ratio for OIS and the image stabilizing ratio for IIS based on the selected image stabilizing ratio setting method.

In step S806, the lens MPU110executes OIS and IIS using the image stabilizing ratio set in step S805.

In performing tilt correction, the configuration of this embodiment selects a method of setting the first image stabilizing ratio based on the stroke ratio. That is, the lens MPU110causes the image transformation processing unit217to perform tilt correction, and executes cooperative driving control of OIS and IIS at the first image stabilizing ratios set by the setting method selected in step S803. Thereby, image stabilization can be performed so as to properly utilize the OIS and IIS strokes, and the image blur residue amount in the peripheral portion of the image can be corrected using the tilt correction. On the other hand, in a case where tilt correction is not performed, the method of setting the second image stabilizing ratio is selected so as to reduce the image blur residual amount in the peripheral portion of the image. That is, the lens MPU110prevents the image transformation processing unit217from performing tilt correction, and executes the cooperative driving control of OIS and IIS at the second image stabilizing ratios set by the setting method selected in step S804. Thereby, the image blur residual amount in the peripheral portion of the image can be reduced.

The cooperative driving control is performed by the lens MPU110in this embodiment, but may be performed by the camera MPU102. In this case, the camera MPU102includes a correction unit and a control unit.

Second Embodiment

A basic configuration of a camera system according to this embodiment is the same as that of the camera system according to the first embodiment. This embodiment is different from the first embodiment in cooperative driving control. This embodiment will discuss only a configuration different from that of the first embodiment and will omit a description of the common configuration.

Referring now toFIG.9, a description will be given of the cooperative driving control of OIS and IIS according to this embodiment. This embodiment assumes that the lens MPU110executes the cooperative driving control.FIG.9is a flowchart illustrating the cooperative driving control of OIS and IIS according to this embodiment. After the camera system is powered on and the initial driving operation and the like are performed, this flow is started.

In step S901, the lens MPU110acquires information necessary for the cooperative driving control. The information necessary for the cooperative driving control includes the movable strokes of the OIS and IIS correction members, a camera peripheral image blur residue amount, a lens peripheral image blur residue amount, information on the sensitivity required for image stabilization, information on whether to perform electronic image stabilization by the image transformation processing unit217, and the like.

In step S902, the lens MPU110determines whether a ratio of one of the camera peripheral image blur residue amount and the lens peripheral image blur residue amount to the other is larger than a predetermined value. In a case where it is determined that the ratio of one of the camera peripheral image blur residue amount and the lens peripheral image blur residue amount to the other is larger than the predetermined value, that is, the execution of image stabilization based on the first image stabilizing ratio is selected, the process of step S903is executed. In a case where it is determined that the ratio of one of the camera peripheral image blur residue amount and the lens peripheral image blur residue amount to the other is smaller than a predetermined value, that is, the execution of image stabilization based on the second image stabilizing ratio is selected, the process of step S905is executed. In a case where the ratio of one of the camera peripheral image blur residue amount and the lens peripheral image blur residue amount to the other is equal to the predetermined value, which step is executed may be arbitrarily set.

In step S903, the lens MPU110selects the method of setting the second image stabilizing ratio for reducing the image blur residue amount in the peripheral portion of the image.

In step S904, the lens MPU110sets to prohibit tilt correction in the electronic image stabilization. Thereby, only translational and rotational transformations can be performed in the electronic image stabilization.

In step S905, the lens MPU110selects the method of setting the first image stabilizing ratio based on the stroke ratio.

In step S906, the lens MPU110sets the image stabilizing ratio for OIS and the image stabilizing ratio for IIS using the selected image stabilizing ratio setting method.

In step S907, the lens MPU110executes OIS and IIS using the image stabilizing ratio calculated in step S805.

In a case where the ratio of one of the camera peripheral image blur residue amount and the lens peripheral image blur residue amount to the other is larger than the predetermined value, the configuration according to this embodiment selects the method of setting the second image stabilizing ratio for suppressing the image blur residue amount in the periphery portion of the image. Therefore, the image blur residue amount in the peripheral portion of the image can be reduced. In this case, a high image stabilizing effect can be obtained by prohibiting the tilt correction in the electronic image stabilization.

The cooperative driving control is performed by the lens MPU110in this embodiment, but may be performed by the camera MPU102. In this case, the camera MPU102includes an acquiring unit, a determining unit, and a setting unit.

Each embodiment can provide a control apparatus that can provide a good image stabilizing effect.

OTHER EMBODIMENTS

This application claims the benefit of Japanese Patent Application No. 2022-145477, filed on Sep. 13, 2022, which is hereby incorporated by reference herein in its entirety.