Image generation apparatus, control method thereof, and recording medium

At the first scale size, a predetermined first arrangement position is determined as the arrangement position of an image to be arranged on a map. At the second scale size larger than the first scale size, the second arrangement position closer to the shooting location of the image than at a position corresponding to the first arrangement position on the map at the second scale size is determined as the arrangement position.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a display technique for an image arranged on a map.

2. Description of the Related Art

Recently, some image capturing apparatuses such as a digital camera include a position detection unit such as a GPS unit. When image data is captured, position information acquired by the position detection unit is set as position information (so-called geotag) of the shooting location in header information of image data or the like. Image data containing information of a shooting location (to be referred to as image data) can be arranged and displayed at a position corresponding to the shooting location on a map using position information of the shooting location. The shooting location of the image data can be intuitively presented to the user.

However, when arranging and displaying image data on a map, if there are a plurality of image data captured at the same shooting location, these image data overlap each other, making the display difficult to see. In some cases, arranged image data occludes a landmark such as the name or shape of a building on a map, and the user cannot grasp shooting location information. To solve this, there is a technique capable of changing, to an arbitrary position set by a user, image data once arranged on a map in accordance with shooting location information held in image data. For example, Japanese Patent Laid-Open No. 2010-66972 discloses a technique capable of changing the position of image data arranged on a map. In this technique, information of the arrangement position of image data on a map is stored as position information on an actual map, sharing the arrangement position of the image data on the map between a plurality of users.

However, in the technique of Japanese Patent Laid-Open No. 2010-66972, information of the arrangement position of image data on a map is shared for only a scale size of the map at which the arrangement position has been changed. When another user views the map at another scale size, the scale size may be changed to one at which the arrangement position has been changed.

The arrangement position of image data on a map is stored as position information on an actual map because the map is scrolled, as described above. If image data is arranged using position information on the actual map upon a change of the scale size, the following case may occur. For example, when the scale size of a map is greatly changed, image data arranged near a shooting location on the map at a scale size before the change is displayed at a position spaced apart from the shooting location upon the change of the scale size. As a result, the user may not be able to find out or view the image data captured at the shooting location for the purpose of viewing upon the change of the scale size of the map.

To prevent this, image data may contain information of arrangement positions on a map for all scale sizes provided by the map. However, for example, the arrangement position of image data may change depending on the scale size, and the user may be confused. When a change of an arrangement position at one scale size is to be reflected on other scale sizes, the arrangement position needs to be recalculated for all scale sizes. Further, storing information of arrangement positions on a map for all the scale sizes of the map in image data increases the image data size and is not practical.

SUMMARY OF THE INVENTION

The present invention has been made to solve the problems of the conventional techniques. The present invention provides a method of arranging an image at an appropriate position on a map in correspondence with a change of the scale size of the map.

The present invention in its first aspect provides an image processing apparatus including a display unit configured to arrange and display an image on a map, comprising: a change unit configured to change a scale size of the map; and a determination unit configured to determine an arrangement position on the map on which the image is arranged, wherein as the arrangement position of the image, the determination unit determines a predetermined first arrangement position at a first scale size, and at a second scale size larger than the first scale size, determines a second arrangement position closer to a shooting location of the image than at a position corresponding to the first arrangement position on the map at the second scale size.

DESCRIPTION OF THE EMBODIMENTS

Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings. The following embodiment will explain an example in which the present invention is applied to, as an example of an image processing apparatus, a PC capable of arranging image data at a corresponding position on a map in accordance with position information of a shooting location held in the image data. However, the present invention is applicable to an arbitrary device capable of arranging image data at a corresponding position on a map in accordance with position information held in the image data.

FIG. 1is a block diagram showing the functional arrangement of a PC100according to the embodiment of the present invention.

A CPU101is a block which controls the operation of each block of the PC100. The CPU101controls the operation of each block of the PC100by, for example, expanding, in a RAM103, the programs of image arrangement position storage processing and image arrangement position correction processing (to be described later) stored in a ROM102, and executing them.

The ROM102is a rewritable non-volatile memory. In addition to the program of image arrangement position correction processing, the ROM102stores parameters and the like necessary for the operation of each block of the PC100.

The RAM103is a volatile memory. The RAM103is used not only as an area for expanding the programs of image arrangement position storage processing and image arrangement position correction processing, but also as a temporary storage area for data and information output by the operation of each block of the PC100.

In the embodiment, each block arranged as a hardware component in the PC100implements processing. However, the practice of the present invention is not limited to this, and processing by each block may be implemented by a program which performs the same processing as that by the block.

A recording medium104is a recording device such as the internal memory of the PC100, or a memory card or HDD detachably connected to the PC100. The recording medium104is used as a database for managing maps and image data. The program of image arrangement position correction processing (to be described later) may be recorded as an application on the recording medium104.

A network I/F105is an interface for connecting the PC100to an external apparatus via a wire or wirelessly. The network I/F105receives image data from, for example, a connected digital camera (not shown). The PC100may be connected to a server on a network such as the Internet via the network I/F105and receive map data.

An operation unit106is a user interface of the PC100that accepts an input from the user, such as a keyboard or mouse. The operation unit106transmits, to the CPU101, a control signal corresponding to an operation by the user.

A display unit107is a display device such as a liquid crystal display connected to the PC100. In the embodiment, the display unit107displays image data containing a map generated by at least image arrangement position correction processing.

(File Structure of Image File)

An image file containing image data to be arranged on a map in the embodiment will be explained with reference toFIG. 2.

Image data203to be arranged on a map is recorded on the recording medium104as an image file having a file structure as shown inFIG. 2. An image file200is formed from a metadata header201, an image header202, and the image data203. The image file200corresponds to one image data.

The metadata header201is a header area where metadata regarding the image file200is stored. In the embodiment, the area of shooting location position information204in the metadata header201contains position information of a shooting location where the image data203in the image file200was captured. More specifically, the metadata header201is defined by the Exif specification or the like. In addition to the shooting location position information204, the metadata header201contains digital camera setting parameters in shooting such as the shooting date & time, focal length, and zoom ratio, and an image search keyword or comment.

The embodiment assumes that the metadata header201stores, as arrangement position information205, information of the arrangement position of the image data203on a map that has been changed by the user, and a scale size on the map upon the change. When the arrangement position information205exists in the metadata header201, the image data203is arranged at a corresponding position on the map not using the shooting location position information204but using the arrangement position information205.

In the following description, the shooting location position information204and arrangement position information205are position information of the digital camera which has captured the image data203that is obtained by a GPS unit, and information of a display position on a map, respectively. These pieces of information are described in the form of the longitude and latitude.

The image header202is a header area where information necessary to decode the image data203is stored. For example, when the image data203is image data encoded in the JPEG encoding format, this area stores a thumbnail image generated by scaling down the image data203.

(Example of Interface of Image Arrangement Position Change Application)

The embodiment will explain an application which arranges image data on a map in accordance with the shooting location position information204held in the image data, and allows the user to change the layout of the image data.

FIG. 3shows the GUI of an image arrangement position change application that is displayed on the display unit107. InFIG. 3, an application window300displays a map display region301, and a selection region302for image data to be arranged.

A plurality of images are displayed in the selection region302, and the thumbnail image of an image checked in the check box is arranged on a map in the map display region301. A position on the map where the image checked in the check box is arranged is determined in accordance with the shooting location position information204or arrangement position information205.

By using “expressions for calculating plane rectangular coordinates x and y and a meridian convergence angle from a latitude and longitude” put on the homepage of Geographical Survey Institute (http://vldb.gsi.go.jp/sokuchi/surveycalc/algorithm/b12xy/b12xy.htm), x- and y-coordinates on the map that correspond to a latitude φ and longitude λ are given by

Δλ=λ−λ0(east is set positive)

Note that the parameters in equations are as follows:

λ0: longitude of the origin of the coordinate systemm0: scale factor at the origin of the coordinate system (m0=0.9999)S0: meridional part from the equator to the latitude φ0of the origin of the coordinate systemS: meridional part from the equator to the latitude φN: curvature radius of the prime verticale′: minor eccentricity

As described above, an image which is arranged on the map in the map display region301and has the shooting location position information204allows the user to change its position on the map by, for example, a drag operation using the mouse. For example, when an image303is displayed and occludes a character string which is contained in the map and represents a place name, the user can move the image303to a position304where the image303does not occlude the place name. At this time, information of the latitude φ and longitude λ corresponding to the arrangement position (plane rectangular coordinates x and y) of the image in the map display region301is stored as the arrangement position information205together with a scale size of the map that is set in the metadata header201of the image file of the image.

By using “expressions for calculating a latitude and longitude and a meridian convergence angle from plane rectangular coordinates x and y” put on the homepage of Geographical Survey Institute (http://vldb.gsi.go.jp/sokuchi/surveycalc/algorithm/xy2b1/xy2b1.htm), a latitude φ and longitude λ corresponding to plane rectangular coordinates x and y on a map are given by

where

Δλ=λ−λ0(east is set positive)

Note that the parameters in equations are as follows:

λ0: longitude of the origin of the coordinate systemm0: scale factor at the origin of the coordinate system (m0=0.9999)φ1: latitude of the foot of a perpendicular dropped from a point in the coordinate system (x, y) to the standard meridianx: x-coordinatey: y-coordinateN1: curvature radius of the prime vertical (argument is (φ1b)e′: minor eccentricity

In the embodiment, information of a latitude and longitude corresponding to the arrangement position of each image on a map is stored as the arrangement position information205in the metadata header201of an image file containing the image. However, the practice of the present invention is not limited to this. For example, information of a latitude and longitude corresponding to the arrangement position of each image on a map may be stored in the ROM102, RAM103, or recording medium104in association with the image.

Considering a scroll operation on a map, information of the arrangement position of each image on the map is preferably stored as a latitude and longitude. However, this information suffices to be information capable of uniquely defining the position on the map at different scale sizes. The arrangement position information205may be formed from information of a latitude and longitude corresponding to the upper left coordinates or origin of the map display region301, the coordinates of the arrangement position of an image in the map display region301, and the scale size of the map. Alternatively, the arrangement position information205may be formed as the coordinates of the geocentric coordinate system (coordinate system defined by the X-, Y-, and Z-axes using the gravity of the earth as an origin).

Image arrangement position storage processing by the PC100having the above arrangement according to the embodiment will be explained in detail with reference to the flowchart ofFIG. 4. Processing corresponding to this flowchart can be implemented by, for example, reading out a corresponding processing program stored in the ROM102or recording medium104, expanding it in the RAM103, and executing it by the CPU101. In the following description, the image arrangement position storage processing starts when, for example, the above-mentioned image arrangement position change application is activated by a user operation.

In step S401, the CPU101determines whether the user has moved a thumbnail image arranged on a map. More specifically, when the input start coordinates of a drag operation received from the operation unit106indicate the position of the thumbnail image arranged in the map display region301and the input end coordinates are different from the input start coordinates, the CPU101determines that the user has moved the thumbnail image. If the CPU101determines that the user has moved the thumbnail image arranged on the map, it shifts the process to step S402. If the CPU101determines that the user has not moved the thumbnail image, it repeats the processing of this step.

In step S402, the CPU101converts, into information of a latitude and longitude using the above-described equations (3) and (4), the coordinates of the thumbnail image arranged on the map after the movement that have been received from the operation unit106.

In step S403, the CPU101stores, as the arrangement position information205in the metadata header201of an image file, the information of the latitude and longitude corresponding to the arrangement position of the thumbnail image that has been obtained by the conversion in step S402, and information of a currently set scale size. Then, the CPU101returns the process to step S401again.

By this processing, the user can move the image arranged on the map to an arbitrary position in accordance with the shooting location position information. Information of the arrangement position on the map after the movement can also be stored and added as information of the latitude and longitude. When it is designated to arrange the image again, the image can be arranged at the same position on the map at the same scale size.

The embodiment assumes that the image arrangement position change application allows sharing image data which is set by the user to be displayed on a map, and information of the arrangement position of the image data. More specifically, the user can use a map viewing application (to be described later) to provide another user with a map on which image data is associated with a position pertaining to a shooting location on the map.

(Example of Interface of Map Viewing Application)

FIG. 5shows the GUI of the map viewing application that is displayed on the display unit107. In the embodiment, the image arrangement position change application for editing the position of image data on a map, and the map viewing application for viewing an edited map will be explained as different applications. However, one application may perform these processes. More specifically, an image generation apparatus on a network may execute the image arrangement position change application, and the PC100may execute the map viewing application using information of the arrangement position of an image that is determined by the image generation apparatus.

An application window500shown inFIG. 5includes a map display region501, map scroll bars502and503, and a scale size change bar504. The map display region501displays a map on which a thumbnail image whose arrangement position has been edited after the thumbnail image is selected to be arranged in the image arrangement position change application is arranged.

A case in which thumbnail images601,602,603, and604are arranged based on the shooting location position information204of the metadata header201of an image file containing these thumbnail images, as shown inFIG. 6A, will be examined. Assume that the user has changed the arrangement positions of the thumbnail images using the image arrangement position change application, as shown inFIG. 6B, because the displayed thumbnail images occlude a tower605serving as a landmark.

The map viewing application presents, to the user, the map on which the arrangement position-changed thumbnail images601to604are arranged. In this case, if the user scales up the map by operating the scale size change bar504, the map is changed as shown inFIG. 6C. This is because, when displaying the thumbnail images on the map, they are arranged on the map using the arrangement position information205serving as information of latitudes and longitudes corresponding to the arrangement positions changed by the image arrangement position change application. As the scale size is increased (scaled up), the distance on the map that is defined by a latitude and longitude serving as shooting location position information, and information of a latitude and longitude corresponding to the changed arrangement position increases.

In this case, the thumbnail image601which should be arranged near the tower605serving as the shooting location is arranged not near the tower605but near a Ferris wheel606, as shown inFIG. 6C. The user neither grasps nor understands the correspondence between the shooting location and the thumbnail image, and is confused.

The embodiment prevents misunderstanding of the correspondence between a shooting location and a thumbnail image owing to the scale size of a map. For example, the thumbnail image601is corrected to an arrangement position as shown inFIG. 6Dby the following image arrangement position correction processing.

Image arrangement position correction processing by the PC100having the above arrangement according to the embodiment will be explained in detail with reference to the flowchart ofFIG. 7. Processing corresponding to this flowchart can be implemented by, for example, reading out a corresponding processing program stored in the ROM102or recording medium104, expanding it in the RAM103, and executing it by the CPU101. In the following description, the image arrangement position correction processing starts when, for example, the above-mentioned map viewing application is activated by a user operation. For descriptive convenience, a case in which one thumbnail image (thumbnail image601) is arranged on the map will be explained. However, it will be easily understood that the practice of the present invention is not limited to this.

In step S701, the CPU101determines whether the user has changed the scale size of the map. More specifically, the CPU101determines whether an operation received from the operation unit106that has detected an operation input from the user is the change operation of the scale size change bar504. If the CPU101determines that the user has changed the scale size of the map, it shifts the process to step S702. If the CPU101determines that the user has not changed the scale size of the map, it repeats the processing of this step.

In step S702, the CPU101calculates the position of the shooting location of the image on the map by referring to the shooting location position information204in the metadata header201of the image file200containing a thumbnail image to be arranged. More specifically, the CPU101calculates the coordinates (x1, y1) of the shooting location on the map by using the above-described equations (1) and (2) based on information of the latitude and longitude of the shooting location that is contained in the shooting location position information204.

In step S703, the CPU101determines whether the shooting location of the thumbnail image to be arranged is contained in the map after the change of the scale size that is displayed in the map display region501. More specifically, the CPU101determines whether the x- and y-coordinates of the shooting location on the map that have been calculated in step S702fall within the x-coordinate range and y-coordinate range representing the display range of the map display region501. If the CPU101determines that the shooting location of the thumbnail image to be arranged is contained in the displayed map after the change of the scale size, it shifts the process to step S704. If the CPU101determines that the shooting location of the thumbnail image to be arranged is not contained in the displayed map after the change of the scale size, it returns the process to step S701without arranging the thumbnail image on the map.

In step S704, the CPU101calculates the coordinates of the arrangement position set on the map for the image by referring to the arrangement position information205in the metadata header201of the image file200containing a thumbnail image to be arranged. More specifically, the CPU101calculates the coordinates (x2, y2) of the arrangement position on the map by using the above-described equations (1) and (2) based on information of the latitude and longitude of the arrangement position that is contained in the arrangement position information205.

As described above, on a map after a change of the scale size, the correspondence between an arrangement position set at a different scale size and the shooting location on the map before change of the scale size, that is, the distance between the shooting location and the arrangement position on the map is changed. For this purpose, in step S705, the CPU101calculates a new arrangement position on the map after the change of the scale size so as to hold the correspondence between the shooting location and the arrangement position set at a different scale size on the map at a scale size before the change.

More specifically, a new arrangement position (x2′, y2′) is given by
x2′=n(x1−x0)+(1−n)(x2−x0)+x0(5)
y2′=n(y1−y0)+(1−n)(y2−y0)+y0(6)
where

α is a constant which satisfies 0<α<1, or satisfies a function: α=ƒ{(x1−x0)2+(y1−y0)2} (0<f<1). Note that the parameters in equations are as follows:m: scale size before a change of the scale sizem′: scale size after a change of the scale sizex0: x-coordinate of the origin of the map in the map display region501y0: y-coordinate of the origin of the map in the map display region501

In step S706, the CPU101determines whether the new arrangement position of the thumbnail image to be arranged is contained in the map after the change of the scale size that is displayed in the map display region501. More specifically, the CPU101determines whether the x- and y-coordinates of the new arrangement position on the map that have been calculated in step S705fall within the x-coordinate range and y-coordinate range representing the display range of the map display region501. If the CPU101determines that the new arrangement position of the thumbnail image to be arranged is contained in the displayed map after the change of the scale size, it shifts the process to step S707. If the CPU101determines that the new arrangement position of the thumbnail image to be arranged is not contained in the displayed map after the change of the scale size, it returns the process to step S701without arranging the thumbnail image on the map.

In step S707, the CPU101causes a GPU (not shown) to generate, for the map rendered in the map display region501, an image in which the thumbnail image is arranged at the new arrangement position on the map that has been calculated in step S705. The CPU101further causes the operation unit106to display the generated image.

As a result, for an image to be arranged on a map, an arrangement position where the correspondence with the shooting location of the image becomes clear can be calculated in accordance with the scale size. The map on which the image is arranged at the calculated arrangement position is presented to the user, solving the aforementioned problems.

In the above-described image arrangement position correction processing sequence, the new arrangement position of a thumbnail image on a map after a change of the scale size is calculated using equations (5) and (6). However, the practice of the present invention is not limited to this. According to the present invention, when the first arrangement position where an image is arranged on a map at the first scale size is set in advance, the image is arranged at the second arrangement position closer to the shooting location of the image than at a position corresponding to the first arrangement position on the map at the second scale size larger than the first scale size. According to the present invention, the image may be arranged at the third arrangement position farther from the shooting location of the image than at a position corresponding to the first arrangement position on the map at the third scale size smaller than the first scale size, as in the above-described image arrangement position correction processing.

At this time, the second arrangement position preferably exists on a straight line connecting the shooting location and the first arrangement position on the map at the second scale size, but the practice of the present invention is not limited to this. According to the present invention, when the scale size is changed to one different from the first scale size, the thumbnail image is arranged at a position where the correspondence between the shooting location and the thumbnail image on the map at the first scale size can be recognized even on the map at the second scale size. Thus, when a position not present on the straight line connecting the shooting location and the first arrangement position is determined as the second arrangement position, a transition to move the thumbnail image from the first arrangement position to the second arrangement position may be presented so that the user can recognize the correspondence with the thumbnail image on the map at a different scale size.

In the above description, when an arrangement position is not contained in the range of a displayed map, no thumbnail image is arranged. However, the practice of the present invention is not limited to this. For example, when a shooting location is not contained in the range of the displayed map but a new calculated arrangement position is contained in the range of the map, the thumbnail image may be arranged. When the shooting location is contained in the range of the map but a new calculated arrangement position is not contained in the range of the map, the thumbnail image may be displayed outside the map or in a dedicated region to notify the user that the thumbnail image exists outside the range of the map.

When a thumbnail image occludes the icon image (landmark image) of a landmark present at a shooting location upon a change of the scale size of a map, a threshold may be set for the display area of the icon image on the map to set whether to inhibit occlusion by the thumbnail image. For example, when the display area of the icon image of the tower605is equal to or smaller than the threshold, as shown inFIG. 8A, the arrangement position of a thumbnail image is changed outside the region of the icon image even if the arrangement position is on the icon image. To the contrary, when the display area of the icon image of the tower605is larger than the threshold, it is considered that the icon image can be recognized even if the thumbnail images601to604are arranged on the icon image. Hence, the thumbnail images may be arranged on the icon image, as shown inFIG. 8B. Note that the area used for determination may be set not only for the icon image of a landmark but also for a character string image (for example, place name) or map symbol, and the same processing may be executed depending on the area.

When the scale size of a map decreases, the arrangement position of a thumbnail image that is calculated for the map after the change of the scale size may be set at a position other than a dry land on the map. In this case, it may become difficult for the user to specify a shooting location corresponding to the thumbnail image arranged on the map. Thus, for example, the thumbnail image may be arranged at a preset arrangement position without changing the arrangement position, or may not be arranged. This control may be executed when a scale size smaller than a predetermined one is selected for a map or when a scale size smaller than one for which an arrangement position is set in advance is selected.

If an arrangement position at the first scale size is set in advance for an image to be arranged on a map, when scaling down the map, the arrangement position may be changed in the above-described way for a scale size falling within within a predetermined range from the first scale size, and may not be changed for a scale size outside the range. Alternatively, for a scale size outside the range, the image may be arranged at a position corresponding to the shooting location.

For example, when a country name or geographical feature attribute is set at a position on a map, and it is determined that the attribute of a position where a thumbnail image is arranged differs from the attribute of the shooting location, the thumbnail image may be arranged at the position of the shooting location. Alternatively, the thumbnail image may be arranged at a preset arrangement position or may not be arranged.

Also, various modifications below may be properly made. The user may individually select whether to change an arrangement position, instead of changing the arrangement positions of all thumbnail images to be arranged on a map upon a change of the scale size. When thumbnail images overlap each other after a change of the scale size, at least one of the arrangement position, aspect ratio, and thumbnail image size may be controlled so that the thumbnail images do not overlap each other. Alternatively, the user may set an overlapping order in advance when thumbnail images overlap each other.

In the description of the above embodiment, an arrangement position is set for one scale size, and a new arrangement position is calculated for a different scale size based on the set arrangement position. However, arrangement positions may be set for a plurality of scale sizes. More specifically, arrangement positions corresponding to a plurality of scale sizes are stored in the arrangement position information205of an image file. When a scale size for which no arrangement position has been set is set, a new arrangement position may be calculated using, as a reference, an arrangement position corresponding to a scale size closest to the set scale size. Alternatively, a new arrangement position may be calculated using interpolation or extrapolation from arrangement positions set at scale sizes before and after a scale size after the change.

In the above description, the arrangement position information205is stored in the image file of an image to be arranged on a map. However, the arrangement position information205may be contained in a map recorded on the recording medium104, as shown inFIG. 9.FIG. 9shows the file structure of a map file900which is recorded on, for example, the recording medium104and used in the image arrangement position change application and map viewing application. InFIG. 9, information of the arrangement position of an image to be arranged on a map is described not in the form of the latitude and longitude but in the form of coordinates on the map.

The map file900contains map data901, and pieces of image information903to905each corresponding to the image file200to be displayed on the map. The map file900also contains arrangement position setting information902including scale sizes at which arrangement positions have been set for the pieces of image information903to905, and information of the latitude and longitude of the origin on the map. As shown inFIG. 9, each image information includes the metadata header201, image header202, and image data203, similar to the image file. Further, the image information includes arrangement position information906representing coordinate information of each image data on a map at a set scale size. With this structure, the user can easily share the layout of a thumbnail image with another user.

As described above, the image processing apparatus according to the embodiment can provide a method of arranging an image at an appropriate position on a map in correspondence with a change of the scale size of the map. More specifically, the image processing apparatus determines a predetermined first arrangement position at the first scale size as the arrangement position of an image to be arranged on a map. At the second scale size larger than the first scale size, the image processing apparatus determines, as the arrangement position, the second arrangement position closer to the shooting location of the image than at a position corresponding to the first arrangement position on the map at the second scale size.

Even when a scale size different from one for which the arrangement position of an image to be arranged on a map has been set is set, the arrangement position is changed in consideration of the scale size. The user can easily grasp the correspondence between an arranged image and the shooting location of the image.

Other Embodiments

This application claims the benefit of Japanese Patent Application No. 2011-106621, filed May 11, 2011, which is hereby incorporated by reference herein in its entirety.