Patent Description:
A panorama image capture function provided from a smartphone or digital camera of the related art may generate one panorama image (still image) using images obtained through panorama image capture.

<CIT> relates to a panorama generation method and apparatus disclosed in particular relates to the field of photography. The method comprises: determining the work state information of a shooting device in a shooting video file process; determining whether the work state information accords with a preset condition, and extracting the video fragment corresponding to the work state information if the work state information accords with the preset condition; and generating a panoramic photograph through the video fragment. The technical scheme can ensure that video materials according with a panorama mode are generated into panoramic photographs, and provide different visual experiences for users through video files.

<CIT> relates to a portable device capable of generating a panoramic photo/video/audio file. The portable device has a plurality of sensors with overlapping sensing areas and an input device receiving a trigger signal that triggers the sensors to sense data for generating a panoramic photo/video/audio file.

However, if an object moves during panorama image capture, a conventional panorama image may be incorrectly matched or one object may be represented as a plurality of objects, thus hindering the quality of images or user experience.

Aspects of the present disclosure are to address at least the above-mentioned problems and/or disadvantages and to provide at least the advantages described below. Accordingly, an aspect of the present disclosure is to provide a method and an electronic device for providing a dynamic panorama function.

The invention is defined by the independent claims, dependent claims define advantageous embodiments.

According to various embodiments described in the present disclosure, the electronic device may reproduce motion of an object included in panorama image capture by providing a dynamic panorama which may be reproduced as video other than a panorama image (still image) upon panorama image capture.

Further, according to an embodiment, the electronic device may synchronize a panorama image (a static panorama) with a panorama video (a dynamic panorama) by using images and analysis information used to generate a panorama image to generate a dynamic panorama.

Further, according to an embodiment, the electronic device may provide a dynamic panorama synchronized with audio data obtained upon panorama image capture.

In addition, the electronic device may provide a variety of effects directly or indirectly ascertained through the present disclosure.

The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of various embodiments of the present disclosure as defined by the claims.

An electronic device according to various embodiments of the present disclosure may include at least one of a smartphone, a tablet personal computer (PC), and a digital camera supporting a panorama function. An electronic device may be one or more combinations of the above-mentioned devices. An electronic device according to some various embodiments of the present disclosure may be a flexible device. An electronic device according to an embodiment of the present disclosure is not limited to the above-mentioned devices, and may include new electronic devices with the development of new technology.

Hereinafter, an electronic device according to various embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. The term "user" used herein may refer to a person who uses an electronic device or may refer to a device (e.g., an artificial intelligence electronic device) that uses an electronic device.

<FIG> is a block diagram illustrating a configuration of an electronic device according to an embodiment of the present disclosure.

Referring to <FIG>, an electronic device <NUM> may include a processor <NUM>. The processor <NUM> may perform a function of processing a variety of calculations and controlling hardware/software components of the electronic device <NUM>. For example, the processor <NUM> may include a panorama image generator <NUM>, a dynamic panorama generator <NUM>, and a last file generator <NUM>.

In an embodiment, some or all of the panorama image generator <NUM>, the dynamic panorama generator <NUM>, and the last file generator <NUM> may be implemented with a software module. For example, a memory <NUM> may store instructions for implementing the panorama image generator <NUM> and the last file generator <NUM> with software. The processor <NUM> may load a first instruction set stored in the memory <NUM> into a framework to implement the panorama image generator <NUM>. The processor <NUM> may load a second instruction set into the framework to implement the dynamic panorama generator <NUM>. In an embodiment, the dynamic panorama generator <NUM> may be implemented with hardware. The dynamic panorama generator <NUM> implemented with hardware may be referred to as a dynamic panorama generation circuit.

The memory <NUM> may store instructions for implementing various embodiments, an application, and files. For example, the memory <NUM> may store the above-mentioned first instruction set, the above-mentioned second instruction set, a panorama content file according to an embodiment, and an application associated with the panorama content file.

In an embodiment, the camera <NUM> obtains a plurality of images. For example, if an image capture button for obtaining a panorama image is selected on a user interface displayed on a display <NUM> of the electronic device <NUM> or if a physical key or a physical button for starting to capture an image is pushed by a user, the electronic device <NUM> may continuously obtain images via the camera <NUM>. The electronic device <NUM> may obtain a proper number of images for generating a panorama image and a dynamic panorama for each second via the camera <NUM>. For example, the electronic device <NUM> may obtain <NUM> images per second or <NUM> images per second. The electronic device <NUM> may stop obtaining an image in response to an input for stopping the image capture (e.g., reselection of the image capture button).

In an embodiment, a microphone <NUM> may obtain audio data generated around the electronic device <NUM> during image capture in progress. For example, voices of a person or noises around the electronic device <NUM> may be obtained as audio data. Also, in an embodiment, the microphone <NUM> may recognize a voice instruction of the user for starting or stopping image capture. For example, the processor <NUM> may allow the camera <NUM> to obtain a plurality of images until the voice instruction "stop" is recognized in response to the voice instruction "start" in a state where a user interface for panorama image capture is displayed. For another example, the processor <NUM> may allow the camera <NUM> to obtain a plurality of images in response to the voice instruction "start panorama" in a state where the display <NUM> of the electronic device <NUM> is turned off or where an application is being executed.

A motion sensor <NUM> senses motion of the electronic device <NUM> during image capture in progress to obtain information about the motion. The motion sensor <NUM> may correspond to, for example, an acceleration sensor or a gyro sensor, or a combination of the acceleration sensor and the gyro sensor. In general, panorama image capture may be performed while the user moves the electronic device <NUM> such as a smartphone in a direction (e.g., to the right) in a state where image capture is started. Thus, the motion sensor <NUM> may obtain motion information of the electronic device <NUM>, such as a direction and distance where the electronic device <NUM> is moved and an up-and-down shake which occurs while the electronic device <NUM> is moved. Also, if the motion sensor <NUM> senses that the electronic device <NUM> is moved in a direction by a specified distance from a time when panorama image capture is started, the processor <NUM> may stop the panorama image capture. The specified distance may be defined in a different way based on a maximum size of an image file which may be processed by the processor <NUM> or the panorama image generator <NUM>. In addition, if a movement direction of the electronic device <NUM> is changed to an opposite direction during panorama image capture or if it is determined that it is difficult to perform additional image matching since motion of an up-and-down direction of the electronic device <NUM> is increased in width, the processor <NUM> may stop the panorama image capture. The up-and-down direction is assumed if the electronic device <NUM> performs panorama image capture while being moved in a left and right direction. In general, if motion in a second direction perpendicular to a first direction departs from a specified range while the electronic device <NUM> performs panorama image capture while being moved in the first direction, the processor <NUM> may stop the panorama image capture.

The display <NUM> may display a panorama image according to an embodiment and may reproduce a panorama video according to an embodiment. For example, the processor <NUM> may generate a panorama content file including panorama image data corresponding to the panorama image and dynamic panorama data corresponding to the panorama video and may store the generated panorama content file in the memory <NUM>. In an embodiment, if an application associated with the panorama content file, for example, a gallery application, a photo application, or the like is executed, the display <NUM> may output normal images or a panorama image in a proper form (e.g., a thumbnail and the like). If the user selects the panorama image, the display <NUM> may output the panorama image. In this case, the display <NUM> may output the panorama image together with an item (e.g., an icon, a menu, and the like) indicating that there is dynamic panorama data corresponding to the panorama image. If the item is selected, the processor <NUM> may reproduce a panorama video based on the dynamic panorama data on the display <NUM>.

A touch sensor <NUM> may receive a touch input from the user. In an embodiment, the touch sensor <NUM> may be integrated with the display <NUM> to be implemented with, for example, a touch screen panel and the like. In the present disclosure, it is assumed that most user operations are performed by touch inputs. However, a touch input may be replaced with a user input by another input means such as a physical key or a physical button.

<FIG> is a drawing illustrating a process of generating panorama image data according to an embodiment of the present disclosure.

Referring to <FIG>, after panorama image capture is started, a first image <NUM> and a second image <NUM> are sequentially obtained. The first image <NUM> and the second image <NUM> may have the same resolution (e.g., width W × height H). A width value and a height value of resolution may be changed to each other based on an image capture mode (e.g., a landscape mode/portrait mode) of an electronic device <NUM> of <FIG> sensed by a sensor (e.g., a gyro sensor). For example, if the first image <NUM> has <NUM> × <NUM> resolution when a camera <NUM> of <FIG> is in the landscape mode, the first image <NUM> may have <NUM> × <NUM> resolution if the camera <NUM> is in the portrait mode. In various embodiments of <FIG> and <FIG>, an image is obtained in a state where image capture is performed in the landscape mode. However, if an image is captured in the portrait mode, it may be applied to the same embodiment as the various embodiments of <FIG> and <FIG>.

The electronic device <NUM> may shake up and down during panorama image capture. For example, the second image <NUM> may be located at a little lower side than the first image <NUM>. Since a virtual panorama region <NUM> is set by the first image <NUM> initially obtained, a region which departs from the panorama region <NUM> may be excluded from a panorama image. For example, an image <NUM>, a region of which is excluded from a lower end of the second image <NUM>, may be used to generate the panorama image. Also, an image <NUM>, a region of which is excluded from an upper end of the first image <NUM>, may be used to generate the panorama image in response to the region excluded from the second image <NUM> to prevent a blank portion from being generated in the panorama image.

The operation described with reference to <FIG> may be performed by a panorama image generator <NUM> of <FIG>. The panorama image generator <NUM> may perform a task of generating panorama image data by using image sequentially obtained by the camera <NUM> as input images. The task of the generating the panorama image data may correspond to a task (e.g., image matching) of adding a previously obtained image to a current image. The panorama image generator <NUM> performs image matching based on the images obtained by the camera <NUM> and motion information sensed by a motion sensor <NUM> of <FIG>. In detail, the panorama image generator <NUM> may extract a feature point of each of the first image <NUM> and the second image <NUM> by analyzing pixel values of the first image <NUM> and the second image <NUM>. Also, the panorama image generator <NUM> may estimate a location of the electronic device <NUM> based on motion information of the electronic device <NUM>, obtained by the motion sensor <NUM>. If it is determined that the feature point is changed or that the location of the electronic device <NUM> is changed, the panorama image generator <NUM> may match the second image <NUM> to the first image <NUM> based on the determination. The panorama image generator <NUM> may repeatedly perform the above-mentioned matching process during panorama image capture in progress. In other words, the panorama image generator <NUM> continuously matches a currently obtained image to a previously matched image based on image processing and motion information until the panorama image capture is stopped.

If the image matching progresses, the panorama image generator <NUM> may provide information for generating dynamic panorama data to a dynamic panorama generator <NUM>. For example, the images <NUM> and <NUM> and a variety of analysis information used for the image matching may be provided to the dynamic panorama generator <NUM>. A process of generating the dynamic panorama data will be given with reference to <FIG>.

<FIG> is a drawing illustrating a process of generating dynamic panorama image data according to an embodiment of the present disclosure.

Referring to <FIG>, 1st to 9th images <NUM> to <NUM> may be used to generate panorama image data and dynamic panorama image. It is necessary for a larger number of images to generate real panorama image data and/or real dynamic panorama data. However, a description will be given of <NUM> images for convenience of description with reference to <FIG>. A first plurality of images is obtained during panorama image capture, and a second plurality of images to be used to generate panorama image data is selected among the first plurality of images. In the present disclosure, except if specifically noted, a plurality of images may refer to the second plurality of images used to generate panorama image data and dynamic panorama data among the first plurality of images.

A dynamic panorama generator <NUM> of <FIG> may receive images <NUM> to <NUM> used to generate a panorama image or information about the images <NUM> to <NUM> from a panorama image generator <NUM> of <FIG>. Herein, the information about the images <NUM> to <NUM> may include the images <NUM> to <NUM> used to generate the panorama image data among a plurality of images obtained by a camera <NUM> of <FIG> and information in which each of the used images <NUM> to <NUM> is edited (e.g., a region used for image matching, and the like).

The images <NUM> to <NUM> received in the dynamic panorama generator <NUM> may be continuously updated during panorama image capture in progress (or during image matching in progress in the panorama image generator <NUM>). For example, in a state shown in <FIG>, the dynamic panorama generator <NUM> may receive images <NUM> and <NUM>. If proceeding with panorama image capture in this state, for example, if proceeding with image matching to the 6th image <NUM>, the dynamic panorama generator <NUM> may finally update the images <NUM> and <NUM> to the images <NUM> and <NUM> and may further obtain the images <NUM> to <NUM>. Although the 7th to 9th images <NUM>, <NUM>, and <NUM> are obtained in this state, since a height of a panorama image is not changed when the panorama image generator <NUM> performs image matching, the images <NUM> to <NUM> may fail to be updated. Thus, the dynamic panorama generator <NUM> may additionally obtain only the images <NUM> to <NUM> used for image matching in the panorama image generator <NUM>.

In an embodiment, the dynamic panorama generator <NUM> may generate dynamic panorama data using an image (e.g., the image <NUM> and the like) used for image matching in the panorama image generator <NUM>. Images may be resized and/or encoded to reduce capacity of a last result (e.g., a panorama content file). For example, the dynamic panorama data may correspond to a kind of encoded moving image data. The dynamic panorama generator <NUM> may encode each of images used for image matching into a Motion Picture Experts Group audio layer <NUM> (MP4) file format and may provide the encoded result data as dynamic panorama data to a last file generator <NUM> of <FIG>.

In another embodiment, the dynamic panorama generator <NUM> may generate dynamic panorama data using an input image (e.g., the first image <NUM> and the like) used for image matching in the panorama image generator <NUM> and analysis information used for the image matching. The analysis information may be a kind of metadata, and may be stored in a memory <NUM> of <FIG> or may be stored in a partial region of a panorama content file, and may be used when a panorama video is reproduced. The panorama video may refer to a moving image reproduced based on dynamic panorama data.

As such, according to an embodiment, since the dynamic panorama generator <NUM> uses information (e.g., an input image, image analysis information, and motion information of an electronic device <NUM> of <FIG>) used by the panorama image generator <NUM>, an amount of calculation or a calculation processing time may be reduced, compared to the case where each of the panorama image generator <NUM> and the dynamic panorama generator <NUM> independently generate image/video data. Since the dynamic panorama generator <NUM> uses result data calculated by the panorama image generator <NUM>, the panorama image generator <NUM> may referred to as a first result image generator, and the dynamic panorama generator <NUM> may be referred to as a second result image generator. Similarly, the last file generator <NUM> may be referred to as a third result image generator.

The dynamic panorama generator <NUM> may generate the dynamic panorama data using the image using the images <NUM> to <NUM> used for image matching. For example, the dynamic panorama generator <NUM> may generate dynamic panorama data temporally and spatially synchronized with panorama image data based on the images <NUM> to <NUM> and analysis information used to generate the panorama image data. The dynamic panorama data may include frames for reproducing a moving image. For example, the dynamic panorama data may include frames <NUM> to <NUM>. Each frame (e.g., the frame <NUM> or the like) may the same as each image (e.g., the image <NUM> or the like) or may be image data properly encoded to be suitable for reproducing a video.

In an embodiment, the dynamic panorama generator <NUM> may generate dynamic panorama data using the original images <NUM> to <NUM> (or resized original images) used for image matching. For example, the dynamic panorama generator <NUM> may encode original images obtained upon panorama image capture into a moving image and may generate analysis information used for image matching, motion information of the electronic device <NUM>, time information, and the like as metadata. In this case, panorama image data (e.g., a single panorama image in which matching is completed), dynamic panorama data (e.g., encoded moving image data), and metadata may be included in a panorama content file. In another embodiment, the metadata may be stored in a separate file linked to the panorama content file.

<FIG> is a drawing illustrating a process of synchronizing panorama image data with dynamic panorama data according to an embodiment of the present disclosure.

Referring to <FIG>, audio data may correspond to audio data obtained by a microphone <NUM> of <FIG> during panorama image capture. An input image may correspond to images sequentially obtained by a camera <NUM> of <FIG> during panorama image capture. The images obtained by the camera <NUM> may be provided as input images to a panorama image generator <NUM> of <FIG>. The panorama image generator <NUM> may use some of the provided images to generate panorama image data. For example, if it is determined that the electronic device <NUM> is moved at a level or more or that an object captured by the camera <NUM> is changed at a level or more, based on motion of an electronic device <NUM> of <FIG> sensed by a motion sensor <NUM> of <FIG> and a change of a feature point between a previous input image and a current input image, the panorama image generator <NUM> may select a corresponding input image as an image for generating the panorama image data. Referring to <FIG>, an embodiment is exemplified as <NUM> images per second are obtained. However, various embodiments are not limited thereto. For example, the number of images obtained per second may be changed by performance of a camera <NUM> of <FIG>, a setting of the electronic device <NUM>, user settings, or the like.

In an embodiment, the electronic device <NUM> may be moved to be relatively slower than a movement speed optimized for generating a panorama image in a first time interval <NUM> in the entire panorama image capture time. Therefore, the panorama image generator <NUM> may use three of six images obtained in the first time interval <NUM> to generate panorama image data. In other words, the panorama image generator <NUM> may fail to use a second image obtained at a time when it is determined that a motion (e.g., a movement distance) of the electronic device <NUM> is insufficient to perform image matching and may use a third image obtained when sufficient motion occurs to perform image matching, with respect to a first image obtained in the first time interval <NUM>.

Further, in a second time interval <NUM> in the entire panorama image capture time, the electronic device <NUM> may not be actually moved or may be very slowly moved. Thus, the panorama image generator <NUM> may use a first image among <NUM> images obtained in the second time interval <NUM> to generate panorama image data and may fail to use the other <NUM> images to generate the image panorama image data.

In a third time interval <NUM> in the entire panorama image capture time, the electronic device <NUM> may be moved at a proper speed. Thus, the panorama image generator <NUM> may use all of <NUM> images obtained as input images to generate panorama image data.

If using some selected among input images to generate panorama image data, that is, if proceeding with image matching using the selected images, the panorama image generator <NUM> may provide information about the input images used for the image matching and analysis information used for the image matching to a dynamic panorama generator <NUM> of <FIG>. For example, if <NUM> images obtained by a camera <NUM> of <FIG> are provided to as input images to the panorama image generator <NUM> and if twenty-seven of the <NUM> images are used to generate panorama image data, the panorama image generator <NUM> may provide information about the <NUM> images and used analyzed information (e.g., feature point information of each of the <NUM> images, information about a time when each of the <NUM> images is obtained, information in which each of the <NUM> images is cropped, motion information of the electronic device <NUM>, and the like) to the dynamic panorama generator <NUM>. In the above-mentioned example, the panorama image generator <NUM> may perform <NUM> image matching processes (a series of tasks of additionally stitching <NUM> images onto a first image) and may provide image information and analysis information updated for each image matching process to the dynamic panorama image generator <NUM>. Also, as described above, if the panorama image generator <NUM> provides an image edited to generate panorama image data (e.g., an image, part of an upper or lower end of which is cropped) to the dynamic panorama generator <NUM>, editing information may fail to be provided to the dynamic panorama generator <NUM>.

A dynamic panorama rendering period may correspond to a period of rendering images (frames) included in dynamic panorama data when the dynamic panorama data generated by the dynamic panorama generator <NUM> is reproduced. For example, the dynamic panorama data may be rendered to be reproduced based on an image capture time upon panorama image capture and motion of the electronic device <NUM> when reproduced by a processor <NUM> of <FIG>. For example, the dynamic panorama data may be rendered at a different frame rate for each time interval, that is, at a variable frame rate. For example, the dynamic panorama data may be rendered at <NUM> frames per second (<NUM> FPS) in the first time interval <NUM>. The dynamic panorama data may be rendered at <NUM> frames per second (<NUM> FPS) in the second time interval <NUM>. The dynamic panorama data may be rendered at <NUM> frames per second (<NUM> FPS) in the third time interval <NUM>. In this case, a panorama video is reproduced by the processor <NUM>, a playback speed may be the same at one time speed in the entire time interval. However, a user may feel as if the panorama video is reproduced at <NUM> times speed in the first time interval. The user may feel as if the panorama video is reproduced at <NUM> times speed in the second time interval. The numeral values of the frame rate described with reference to <FIG> are exemplified for convenience of description, and may be modified in various manners by those skilled in the art.

For another example, dynamic panorama data may be rendered at a constant frame rate for each time interval, but may be reproduced in a different way for each time interval by the processor <NUM>. In this case, the processor <NUM> may refer to analysis information (e.g., information about a time when an image is obtained) when a panorama video is reproduced. For example, a panorama video generated by rendering a dynamic panorama may be reproduced to be slower two times than a reference speed during the first time interval <NUM> (<NUM> frames per second may be reproduced in an example of <FIG>). The panorama video may be actually stopped or may be very slowly reproduced like a slow motion in the second time interval <NUM>. The panorama video may be reproduced at the reference speed in the third time interval <NUM>.

In the above-mentioned examples, a panorama video may be reproduced with the same or similar feeling to an image displayed on a display <NUM> of the electronic device <NUM> upon real panorama image capture. Also, as a time when a panorama video is reproduced and a time when audio data obtained via a microphone <NUM> of <FIG> is reproduced are synchronized with each other one to one, since an audio is undistorted when the panorama video is reproduced, video and audio upon image capture may be reproduced. The rendering/playback of the panorama video in the above-mentioned manner may be referred to as rendering/playback about audio in the present disclosure.

However, in another embodiment, dynamic panorama data may be rendered at a constant frame rate, and a panorama video may be reproduced at a constant speed. Since images used to generate the dynamic panorama data are images, in which motion of a constant level occurs to a previous image, used for image matching of the panorama image data, a change amount of an image displayed on the display <NUM> of the electronic device <NUM> may be a constant amount. In other words, the electronic device <NUM> may show the entire panorama image at a constant speed from left to right or from right to left. The rendering/playback of the panorama video in the above-mentioned manner may be referred to as rendering/playback about video in the present disclosure.

When dynamic panorama data is rendered about video, as audio data is synchronized with a time when a plurality of images are obtained, a playback speed of audio data may be changed for each time interval. For example, audio data generated in the second time interval <NUM> may have temporally the same length as audio data obtained in the third time interval <NUM>. However, a playback time of a panorama video corresponding to the second time interval <NUM> may be only <NUM>/<NUM> of a playback time of a panorama video corresponding to the third time interval <NUM>. Thus, when a dynamic panorama is reproduced about video, audio data of the second time interval <NUM> may be reproduced at three times speed.

In an embodiment, when dynamic panorama data is rendered or reproduced about video, the processor <NUM> may fail to reproduce audio data not to hinder user experience. In another embodiment, the processor <NUM> may classify audio data into a voice interval and a silence interval, may reproduce a panorama video about audio at the voice interval (i.e., change a playback speed), may reproduce the panorama video about video at the silence interval.

In an embodiment, when dynamic panorama data is rendered about video, the processor <NUM> may reproduce audio data corresponding to a moved object. For example, if capturing a play or musical performance in which a user A, a user B, and a user B appear, using a panorama function, the processor <NUM> may recognize a direction of a speech which occurs from each of the users A to C. For example, when the user A is displayed on the display <NUM> of the electronic device <NUM> during panorama image capture, if the user A and the user B alternately speak their lines, the electronic device <NUM> may distinguish audio data collected from a direction where the user A is located from audio data collected from a direction where the user B is located. For example, if there are a plurality of microphones, the processor <NUM> may map an audio data interval to an image capture direction based on the image capture direction of the electronic device <NUM>, determined using a phase difference between audio waveforms collected from the plurality of microphones and various sensors (e.g., an acceleration sensor, a geomagnetic sensor, a gyro sensor, and the like) of the electronic device <NUM>. When reproducing dynamic panorama data, if a specific direction is shown or if an object (e.g., the user A) corresponding to the specific direction is shown, the processor <NUM> may reproduce audio data corresponding to the corresponding object (i.e., the corresponding direction). If it is started to show an object (i.e., the user B) corresponding to another direction, the processor <NUM> may reproduce audio data corresponding to a direction of the user B.

Hereinafter, a description will be given of a process of generating and storing a panorama content file with reference to <FIG> and <FIG>. The same, similar, or corresponding contents to the above-mentioned contents will be omitted from a description below.

<FIG> is a flowchart illustrating the entire process of generating a panorama content file according to an embodiment of the present disclosure.

Referring to <FIG>, in operation <NUM>, an electronic device <NUM> of <FIG> obtains a plurality of images via a camera <NUM> of <FIG> during panorama image capture in progress. Herein, the plurality of images may correspond to a first plurality of images. Also, the electronic device <NUM> obtains motion information of the electronic device <NUM> via a motion sensor <NUM> of <FIG> while a plurality of images are obtained. In addition, the electronic device <NUM> may further obtain audio data. The plurality of obtained images, the obtained motion information, and the obtained audio data may be matched to each other relative to a time when each of the plurality of images, the motion information, and the audio data is obtained.

In operation <NUM>, the electronic device <NUM> generates panorama image data based on the plurality of obtained images and the obtained motion information. For example, the electronic device <NUM> (e.g., a panorama image generator <NUM> of <FIG>) may generate panorama image data by selecting a second plurality of images to be used for image matching based on motion information and analysis information of each of the first plurality of images among the first plurality of images and performing image matching using the second plurality of images.

In operation <NUM>, the electronic device <NUM> generates dynamic panorama data based on the data (e.g., the second plurality of images) and the analysis information used in operation <NUM>.

In operation <NUM>, the electronic device <NUM> generates a panorama content file including the panorama image data and the dynamic panorama data and stores the generated panorama content file in a storage space (e.g., a memory <NUM>) of the electronic device <NUM>. As described above, the analysis information may be metadata and may be stored to be independent of the panorama content file. If the metadata is stored to be independent of the panorama content file, the electronic device <NUM> may refer to metadata for synchronization when rendering a panorama video using the dynamic panorama data. In the present disclosure, an embodiment of storing the metadata to be independent of the panorama content file may be replaced with an embodiment of not storing the metadata to be independent of the panorama content file. Hereinafter, a description will be given of the embodiment of not storing the metadata to be independent of the panorama content file to exclude unnecessary duplication of the description.

<FIG> is a flowchart illustrating a detailed process of generating a panorama content file according to an embodiment of the present disclosure.

Referring to <FIG>, in operation <NUM>, a panorama image generator <NUM> of <FIG> may receive data for generating panorama image data. For example, a plurality of images obtained by a camera <NUM> of <FIG>, audio data obtained by a microphone <NUM> of <FIG>, motion information of the electronic device <NUM> obtained by a motion sensor <NUM> of <FIG>, time information associated with each of the plurality of images, the audio data and the motion information, and the like may be provided as input data to the panorama image generator <NUM>.

In operation <NUM>, the panorama image generator <NUM> may analyze the input image and the motion information of the electronic device <NUM>. In operation <NUM>, the panorama image generator <NUM> may determine whether to match a current input image to panorama image data previously generated by image matching, based on the analyzed result. For example, if it is determined that motion of the electronic device <NUM> meets a specified condition based on the analyzed result, the panorama image generator <NUM> may determine that image matching will be performed.

If the image matching is performed, the panorama image generator <NUM> may update the panorama image data at operation <NUM>. For example, referring to <FIG>, while matching a 3rd image <NUM> to old panorama image data generated after a 1st image <NUM> and a 2nd image <NUM> are matched to each other, the panorama image generator <NUM> may crop part of an upper end of the 1st image <NUM> and part of an upper end of the 2nd image <NUM> relative to the 3rd image <NUM>.

If updating the panorama image data, the panorama image generator <NUM> may provide the updated information to a dynamic panorama generator <NUM> of <FIG>. The dynamic panorama generator <NUM> may update the old dynamic panorama data at operation <NUM> based on the newly provided information.

The above-mentioned image matching process may be repeatedly performed until the panorama image capture is ended. For example, in operation <NUM>, if it is determined that the image capture is ended, operations <NUM>, <NUM>, and <NUM> of <FIG> may be performed. In detail, if the image capture is ended, that is, if an image matching process of the last input image is completed, the panorama image generator <NUM> may finally generate panorama image data. Also, the dynamic panorama generator <NUM> may generate dynamic panorama data based on the information finally provided from the panorama image generator <NUM>. Also, a last result generator <NUM> of <FIG> may generate a panorama content file based on the panorama image data and the dynamic panorama data. The panorama content file may have a format (e.g., a joint photographic coding experts group (JPEG), a portable network graphics (PNG), and the like) of an image file, but may additionally include data (e.g., MP4 data) corresponding to a moving image file in its additional data region. For example, the last result generator <NUM> may generate an image file in which information (e.g., a marker) indicating that additional data is present other than image data is recorded in a header, a footer, or a tail portion of the image file. This information may include information indicating whether additional data (i.e., a variety of additional (analysis) information used to generate dynamic panorama data and panorama image data, and the like) is present, information about a start location, and information about a stop location.

In the above-mentioned description, at least some of the panorama image generator <NUM>, the dynamic panorama generator <NUM>, and the last result generator <NUM> may be implemented with a software module. In this case, it may be understood that an operation performed by the software module may be performed by the processor <NUM>.

Hereinafter, a description will be given of a method of consuming a panorama content file with reference to <FIG>.

<FIG> is a drawing illustrating a screen of executing a panorama content file according to an embodiment of the present disclosure.

Referring to <FIG>, a panorama content file may be displayed on a first screen <NUM> based on a panorama image. For example, if the panorama content file is executed through an application, a processor <NUM> of <FIG> may display a panorama image based on panorama image data in the panorama content file on a display <NUM> of <FIG>. If the panorama image is displayed on the first screen <NUM>, the processor <NUM> may display control menus <NUM> (e.g., a share menu, an edit menu, a delete menu, and the like) and an item <NUM> corresponding to dynamic panorama data in the panorama content file other than the panorama image.

If the item <NUM> is selected, the processor <NUM> may reproduce the dynamic panorama data on a second screen <NUM>. For example, the processor <NUM> may render a panorama video based on the dynamic panorama data and may reproduce the rendered panorama video on the display <NUM>. The processor <NUM> may render the panorama video in a first time order. Herein, the first time order may correspond to an order of a time when each of images included in the dynamic panorama data is obtained. Also, the processor <NUM> may render a panorama video in a reverse order of the first time order (i.e., a second time order). For example, if a user performs panorama image capture while moving an electronic device <NUM> of <FIG> in a right direction from the left, the panorama video rendered in the first time order may have an effect shown while a panorama image is moved in a left direction. If a panorama video is rendered in the second time order, the panorama video may have an effect shown while a panorama image is moved in the right direction.

In an embodiment, dynamic panorama data may correspond to encoded video data. Thus, the processor <NUM> (or a codec corresponding to video data) may decode video data. If the video data is synchronized with a panorama image, for example, if video data is encoded based on images encoded to generate a panorama image, the processor <NUM> may render decoded frames on the display <NUM>. If video data is encoded based on original images used to generate a panorama image, the processor <NUM> may edit decoded frames based on metadata (e.g., image analysis information, motion information, and time information) and may render the edited frames on a screen.

Since a panorama video is generated based on a plurality of images used to generate a panorama image, although there is a moved object (e.g., a person, an animal, a car, and the like) during panorama image capture, motion of the object may be reproduced without distortion. Thus, although a moved object is distorted and represented on a panorama image, the user may execute a panorama video to check motion of the object.

A panorama image and a panorama video displayed/reproduced based on a panorama content file has the same virtual field of view (FOV). As described with reference to <FIG>, since dynamic panorama data for rendering a panorama video is generated using images and analysis information used to generate panorama image data, when resolution of a panorama image has a first height and a first width, a virtual FOV shown by a panorama video may also have the first height and the first width. In other words, the panorama video may be generated to output the virtual FOV corresponding to the first height and the first width at intervals of an FOV corresponding to the same first height and a second width which is narrower than the first width during a time when reproduced by the processor <NUM>. Herein, since the FOV of the panorama image has a higher value than that of a camera <NUM> of the electronic device <NUM>, the FOV of the panorama image may be a virtual FOV. However, an image (or a frame) displayed on the display <NUM> when a panorama video is reproduced may the same as the FOV of the camera <NUM> or may correspond to a real FOV having a lower range than that of the camera <NUM> due to encoding.

The reproduction of a panorama video may be controlled in various manners. For example, while a panorama video is reproduced on the second screen <NUM>, if a touch input <NUM> on any region occurs, video playback may pause on a third screen <NUM>. If a touch input on a playback icon <NUM> occurs on the third screen <NUM> or if a touch input on any region occurs again, the reproduction of the panorama video may be resumed. If a touch input <NUM> on a save video icon occurs, the panorama video may be saved to memory.

A panorama video rendered in a time order may be reproduced on the second screen <NUM>. When a panorama video is reproduced as if the camera <NUM> is moved from left to right, if a drag input <NUM> of an opposite direction occurs, the processor <NUM> may render the panorama video in a reverse order, that is, a reverse order of a time. In this case, the panorama video may be reproduced on a fourth screen <NUM> like an opposite direction, that is, as if the camera <NUM> is moved from right to left. If a drag input <NUM> of the first direction occurs, the processor <NUM> may pause the video as in the third screen <NUM>.

In another embodiment, a rendering order may be changed by tilting the electronic device <NUM> in a specified direction while a panorama video is reproduced. For example, if a motion sensor <NUM> of <FIG> senses motion <NUM> where the electronic device <NUM> rotates clockwise at an angle or more on a central axis of a vertical direction, the processor <NUM> may render a panorama video in a first order (e.g., a time order). If the motion sensor <NUM> senses motion <NUM> where the electronic device <NUM> rotates counterclockwise, the processor <NUM> may render the panorama video in a second order opposite to the first order.

In addition, an event of changing an order of rendering a panorama video may be defined in various manners. For example, if a last frame of a panorama video rendered in the first order is reproduced, the processor <NUM> may reproduce a panorama video rendered in the second order on the display <NUM>. In this case, the user may have experience as if he or she appreciates a panorama image in one direction (e.g., from left to right) and then appreciates the panorama image in an opposite direction.

<FIG> illustrates an electronic device in a network environment according to an embodiment of the present disclosure.

Referring to <FIG>, an electronic device <NUM> in a network environment <NUM> according to various embodiments of the present disclosure will be described with reference to <FIG>. The electronic device <NUM> may include a bus <NUM>, a processor <NUM>, a memory <NUM>, an input/output interface <NUM>, a display <NUM>, and a communication interface <NUM>. In various embodiments of the present disclosure, at least one of the foregoing elements may be omitted or another element may be added to the electronic device <NUM>.

The bus <NUM> may include a circuit for connecting the above-mentioned elements <NUM> to <NUM> to each other and transferring communications (e.g., control messages and/or data) among the above-mentioned elements.

The processor <NUM> may include at least one of a CPU, an AP, or a communication processor (CP). The processor <NUM> may perform data processing or an operation related to communication and/or control of at least one of the other elements of the electronic device <NUM>.

The memory <NUM> may include a volatile memory and/or a nonvolatile memory. The memory <NUM> may store instructions or data related to at least one of the other elements of the electronic device <NUM>. According to an embodiment of the present disclosure, the memory <NUM> may store software and/or a program <NUM>. The program <NUM> may include, for example, a kernel <NUM>, a middleware <NUM>, an application programming interface (API) <NUM>, and/or an application program (or an application) <NUM>. At least a portion of the kernel <NUM>, the middleware <NUM>, or the API <NUM> may be referred to as an operating system (OS).

The kernel <NUM> may control or manage system resources (e.g., the bus <NUM>, the processor <NUM>, the memory <NUM>, or the like) used to perform operations or functions of other programs (e.g., the middleware <NUM>, the API <NUM>, or the application program <NUM>). Furthermore, the kernel <NUM> may provide an interface for allowing the middleware <NUM>, the API <NUM>, or the application program <NUM> to access individual elements of the electronic device <NUM> in order to control or manage the system resources.

The middleware <NUM> may serve as an intermediary so that the API <NUM> or the application program <NUM> communicates and exchanges data with the kernel <NUM>.

Furthermore, the middleware <NUM> may handle one or more task requests received from the application program <NUM> according to a priority order. For example, the middleware <NUM> may assign at least one application program <NUM> a priority for using the system resources (e.g., the bus <NUM>, the processor <NUM>, the memory <NUM>, or the like) of the electronic device <NUM>. For example, the middleware <NUM> may handle the one or more task requests according to the priority assigned to the at least one application, thereby performing scheduling or load balancing with respect to the one or more task requests.

The API <NUM>, which is an interface for allowing the application <NUM> to control a function provided by the kernel <NUM> or the middleware <NUM>, may include, for example, at least one interface or function (e.g., instructions) for file control, window control, image processing, character control, or the like.

The input/output interface <NUM> may serve to transfer an instruction or data input from a user or another external device to (an)other element(s) of the electronic device <NUM>. Furthermore, the input/output interface <NUM> may output instructions or data received from (an)other element(s) of the electronic device <NUM> to the user or another external device.

The display <NUM> may include, for example, a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic LED (OLED) display, a microelectromechanical systems (MEMS) display, or an electronic paper display. The display <NUM> may present various content (e.g., a text, an image, a video, an icon, a symbol, or the like) to the user. The display <NUM> may include a touch screen, and may receive a touch, gesture, proximity or hovering input from an electronic pen or a part of a body of the user.

The communication interface <NUM> may set communications between the electronic device <NUM> and an external device (e.g., a first external electronic device <NUM>, a second external electronic device <NUM>, or a server <NUM>). For example, the communication interface <NUM> may be connected to a network <NUM> via wireless communications or wired communications so as to communicate with the external device (e.g., the second external electronic device <NUM> or the server <NUM>).

The wireless communications may employ at least one of cellular communication protocols such as long-term evolution (LTE), LTE-advance (LTE-A), code division multiple access (CDMA), wideband CDMA (WCDMA), universal mobile telecommunications system (UMTS), wireless broadband (Wi-Bro), or global system for mobile communications (GSM). The wireless communications may include, for example, short-range communications <NUM>. The short-range communications may include at least one of wireless fidelity (Wi-Fi), Bluetooth, near field communication (NFC), magnetic secure transmission (MST), or global navigation satellite system (GNSS).

The MST may generate a pulse by using electro-magnetic signals according to transmission data, and the pulse may cause magnetic signals. The electronic device <NUM> may transmit the magnetic signals to a point of sales (POS). The POS may detect the magnetic signals using a MST reader and obtain the transmission data by converting the magnetic signals to electronic signals.

The GNSS may include, for example, at least one of global positioning system (GPS), global navigation satellite system (GLONASS), BeiDou navigation satellite system (BeiDou), or Galileo, the European global satellite-based navigation system according to a use area or a bandwidth. Hereinafter, the term "GPS" and the term "GNSS" may be interchangeably used. The wired communications may include at least one of universal serial bus (USB), high definition multimedia interface (HDMI), recommended standard <NUM> (RS-<NUM>), plain old telephone service (POTS), or the like. The network <NUM> may include at least one of telecommunications networks, for example, a computer network (e.g., local area network (LAN) or wide area network (WAN)), the Internet, or a telephone network.

The types of the first external electronic device <NUM> and the second external electronic device <NUM> may be the same as or different from the type of the electronic device <NUM>. According to an embodiment of the present disclosure, the server <NUM> may include a group of one or more servers. A portion or all of operations performed in the electronic device <NUM> may be performed in one or more other electronic devices (e.g., the first electronic device <NUM>, the second external electronic device <NUM>, or the server <NUM>). When the electronic device <NUM> should perform a certain function or service automatically or in response to a request, the electronic device <NUM> may request at least a portion of functions related to the function or service from another device (e.g., the first electronic device <NUM>, the second external electronic device <NUM>, or the server <NUM>) instead of or in addition to performing the function or service for itself. The other electronic device (e.g., the first electronic device <NUM>, the second external electronic device <NUM>, or the server <NUM>) may perform the requested function or additional function, and may transfer a result of the performance to the electronic device <NUM>. The electronic device <NUM> may use a received result itself or additionally process the received result to provide the requested function or service. To this end, for example, a cloud computing technology, a distributed computing technology, or a client-server computing technology may be used.

Referring to <FIG>, the electronic device <NUM> may include, for example, a part or the entirety of the UE <NUM> illustrated in <FIG>. The electronic device <NUM> may include at least one processor (e.g., an AP) <NUM>, a communication module <NUM>, a subscriber identification module (SIM) <NUM>, a memory <NUM>, a sensor module <NUM>, an input device <NUM>, a display module <NUM>, an interface <NUM>, an audio module <NUM>, a camera module <NUM>, a power management module <NUM>, a battery <NUM>, an indicator <NUM>, and a motor <NUM>.

The processor <NUM> may execute, or run, an OS or an application program so as to control a plurality of hardware or software elements connected to the processor <NUM>, process various data, and perform operations. The processor <NUM> may be implemented with, for example, a system on chip (SoC). According to an embodiment of the present disclosure, the processor <NUM> may further include a graphics processing unit (GPU) and/or an image signal processor (ISP). The processor <NUM> may include at least a portion (e.g., a cellular module <NUM>) of the elements illustrated in <FIG>. The processor <NUM> may load, on a volatile memory, an instruction or data received from at least one of the other elements (e.g., a nonvolatile memory) to process the instruction or data, and may store various data in a nonvolatile memory.

The communication module <NUM> may be configured the same as or similar to that of the communication circuit <NUM> of <FIG>. The communication module <NUM> may include, for example, a cellular module <NUM> (e.g., a modem), a Wi-Fi module <NUM>, a Bluetooth (BT) module <NUM>, a GNSS module <NUM> (e.g., a GPS module, a GLONASS module, a BeiDou navigation satellite system module, or a Galileo global navigation satellite system module), a NFC module <NUM>, a MST module <NUM>, and a radio frequency (RF) module <NUM>.

The cellular module <NUM> may provide, for example, a voice call service, a video call service, a text message service, or an Internet service through a communication network. The cellular module <NUM> may identify and authenticate the electronic device <NUM> in the communication network using the SIM <NUM> (e.g., a SIM card). The cellular module <NUM> may perform at least a part of the functions that may be provided by the processor <NUM>. The cellular module <NUM> may include a CP.

Each of the Wi-Fi module <NUM>, the Bluetooth module <NUM>, the GNSS module <NUM>, the NFC module <NUM>, and the MST module <NUM> may include, for example, a processor for processing data transmitted/received through the modules. According to an embodiment of the present disclosure, at least a part (e.g., two or more) of the cellular module <NUM>, the Wi-Fi module <NUM>, the Bluetooth module <NUM>, the GNSS module <NUM>, the NFC module <NUM>, and the MST module <NUM> may be included in a single integrated circuit (IC) or IC package.

The RF module <NUM> may transmit/receive, for example, communication signals (e.g., RF signals). The RF module <NUM> may include, for example, a transceiver, a power amplifier module (PAM), a frequency filter, a low noise amplifier (LNA), an antenna, or the like. According to an embodiment of the present disclosure, at least one of the cellular module <NUM>, the Wi-Fi module <NUM>, the Bluetooth module <NUM>, the GNSS module <NUM>, the NFC module <NUM>, or the MST module <NUM> may transmit/receive RF signals through a separate RF module.

The SIM <NUM> may include, for example, an embedded SIM and/or a card containing the SIM, and may include unique identification information (e.g., an integrated circuit card identifier (ICCID)) or subscriber information (e.g., an international mobile subscriber identity (IMSI)).

The memory <NUM> (e.g., the memory <NUM> of <FIG>) may include, for example, an internal memory <NUM> or an external memory <NUM>. The internal memory <NUM> may include at least one of a volatile memory (e.g., a dynamic random access memory (DRAM), a static random access memory (SRAM), a synchronous DRAM (SDRAM), or the like), a nonvolatile memory (e.g., a one-time programmable read only memory (OTPROM), a programmable ROM (PROM), an erasable and programmable ROM (EPROM), an electrically erasable and programmable ROM (EEPROM), a mask ROM, a flash ROM, a flash memory (e.g., a NAND flash memory, a NOR flash memory, or the like)), a hard drive, or a solid state drive (SSD).

The external memory <NUM> may include a flash drive such as a compact flash (CF) drive, a secure digital (SD) drive, a micro-SD drive, a mini-SD drive, an extreme digital (xD) drive, a multi media card (MMC), a memory stick, or the like. The external memory <NUM> may be operatively and/or physically connected to the electronic device <NUM> through various interfaces.

A security module <NUM>, which is a module including a storage space that is more secure (e.g. has a higher security level) than the memory <NUM>, may be a circuit for providing secure data storage and protected execution circumstances. The security module <NUM> may be implemented with an additional circuit and may include an additional processor. The security module <NUM> may be present in an attachable smart chip or SD card, or may include an embedded secure element (eSE), which is installed in a fixed chip. Additionally, the security module <NUM> may be driven in another OS which is different from the OS of the electronic device <NUM>. For example, the security module <NUM> may operate based on a java card open platform (JCOP) OS.

The sensor module <NUM> may, for example, measure a physical quantity or detect an operation state of the electronic device <NUM> so as to convert measured or detected information into an electrical signal. The sensor module <NUM> may include, for example, at least one of a gesture sensor 940A, a gyro sensor 940B, a barometric pressure sensor 940C, a magnetic sensor 940D, an acceleration sensor 940E, a grip sensor 940F, a proximity sensor <NUM>, a color sensor <NUM> (e.g., a red/green/blue (RGB) sensor), a biometric sensor 940I, a temperature/humidity sensor 940J, an illumination sensor <NUM>, or an ultraviolet (UV) light sensor <NUM>. Additionally, or alternatively, the sensor module <NUM> may include, for example, an olfactory sensor (e.g., an electronic nose (E-nose) sensor), an electromyography (EMG) sensor, an electroencephalogram (EEG) sensor, an electrocardiogram (ECG) sensor, an infrared (IR) sensor, an iris recognition sensor, and/or a fingerprint sensor. The sensor module <NUM> may further include a control circuit for controlling at least one sensor included therein. In an embodiment of the present disclosure, the electronic device <NUM> may further include a processor configured to control the sensor module <NUM> as a part of the processor <NUM> or separately, so that the sensor module <NUM> is controlled while the processor <NUM> is in a reduced power, or sleep, state.

The input device <NUM> may include, for example, a touch panel <NUM>, a (digital) pen sensor <NUM>, a key <NUM>, or an ultrasonic input device <NUM>. The touch panel <NUM> may employ at least one of a capacitive method, a resistive method, an infrared method, and an ultraviolet light sensing method. The touch panel <NUM> may further include a control circuit. The touch panel <NUM> may further include a tactile layer so as to provide a haptic feedback to a user.

The (digital) pen sensor <NUM> may include, for example, a sheet for recognition which is a part of a touch panel or is separate. The key <NUM> may include, for example, a physical button, an optical button, or a keypad. The ultrasonic input device <NUM> may sense ultrasonic waves generated by an input tool through a microphone <NUM> so as to identify data corresponding to the ultrasonic waves sensed.

The display module <NUM> (e.g., the display <NUM> of <FIG>) may include a panel <NUM>, a hologram device <NUM>, or a projector <NUM>. The panel <NUM> may be configured the same as or similar to that of the display <NUM> of <FIG>. The panel <NUM> may be, for example, flexible, transparent, or wearable. The panel <NUM> and the touch panel <NUM> may be integrated into a single module. The hologram device <NUM> may display a stereoscopic image in a space using a light interference phenomenon. The projector <NUM> may project light onto a screen so as to display an image. The screen may be disposed internally or externally to the electronic device <NUM>. According to an embodiment of the present disclosure, the display module <NUM> may further include a control circuit for controlling the panel <NUM>, the hologram device <NUM>, or the projector <NUM>.

The interface <NUM> may include, for example, an HDMI <NUM>, a USB <NUM>, an optical interface <NUM>, or a D-subminiature (D-sub) connector <NUM>. The interface <NUM>, for example, may be included in the communication circuit <NUM> of <FIG>. Additionally, or alternatively, the interface <NUM> may include, for example, a mobile high-definition link (MHL) interface, an SD/MMC interface, or an Infrared Data Association (IrDA) interface.

The audio module <NUM> may convert, for example, a sound into an electrical signal or vice versa. The audio module <NUM> may process sound information input or output through a speaker <NUM>, a receiver <NUM>, an earphone <NUM>, or the microphone <NUM>.

The camera module <NUM> is, for example, a device for taking a still image or a video. According to an embodiment of the present disclosure, the camera module <NUM> may include at least one image sensor (e.g., a front sensor or a rear sensor), a lens, an ISP, or a flash (e.g., a LED or a xenon lamp).

The power management module <NUM> may manage power of the electronic device <NUM>. According to an embodiment of the present disclosure, the power management module <NUM> may include a power management integrated circuit (PMIC), a charger IC, a battery, or a battery gauge. The PMIC may employ a wired and/or a wireless charging method. A wireless charging method may include, for example, a magnetic resonance method, a magnetic induction method, an electromagnetic method, or the like. An additional circuit for wireless charging, such as a coil loop, a resonant circuit, a rectifier, or the like, may be further included. The battery gauge may measure, for example, a remaining capacity of the battery <NUM> and a voltage, current, or temperature thereof while the battery is charged. The battery <NUM> may include, for example, a rechargeable battery and/or a solar battery.

The indicator <NUM> may display a certain state of the electronic device <NUM> or a part thereof (e.g., the processor <NUM>), such as a booting state, a message state, a charging state, or the like. The motor <NUM> may convert an electrical signal into a mechanical vibration, and may generate a vibration or a haptic effect. A processing device (e.g., a GPU) for supporting mobile television (TV) may be included in the electronic device <NUM>. The processing device for supporting mobile TV may process media data according to the standards of digital multimedia broadcasting (DMB), digital video broadcasting (DVB), MediaFLO™, or the like.

Each of the elements described herein may be configured with one or more components, and the names of the elements may be changed according to the type of an electronic device. In various embodiments of the present disclosure, an electronic device may include at least one of the elements described herein, and some elements may be omitted or other additional elements may be added. Furthermore, some of the elements of the electronic device may be combined with each other so as to form one entity, so that the functions of the elements may be performed in the same manner as before the combination.

<FIG> is a block diagram illustrating a program module according to an embodiment of the present disclosure.

Referring to <FIG>, a program module <NUM> (e.g., the program <NUM>) may include an OS for controlling a resource related to an electronic device (e.g., the electronic device <NUM>) and/or various applications (e.g., the application program <NUM>) running on the OS. The OS may be, for example, Android, iOS, Windows, Symbian, Tizen, or the like.

The program module <NUM> may include a kernel <NUM>, a middleware <NUM>, an API <NUM>, and/or an application <NUM>. At least a part of the program module <NUM> may be preloaded on an electronic device or may be downloaded from an external electronic device (e.g., the first electronic device <NUM>, the second external electronic device <NUM>, or the server <NUM>).

The kernel <NUM> (e.g., the kernel <NUM>) may include, for example, a system resource manager <NUM> or a device driver <NUM>. The system resource manager <NUM> may perform control, allocation, or retrieval of a system resource. According to an embodiment of the present disclosure, the system resource manager <NUM> may include a process management unit, a memory management unit, a file system management unit, or the like. The device driver <NUM> may include, for example, a display driver, a camera driver, a Bluetooth driver, a shared memory driver, a USB driver, a keypad driver, a Wi-Fi driver, an audio driver, or an inter-process communication (IPC) driver.

The middleware <NUM>, for example, may provide a function that the applications <NUM> require in common, or may provide various functions to the applications <NUM> through the API <NUM> so that the applications <NUM> may efficiently use limited system resources in the electronic device. According to an embodiment of the present disclosure, the middleware <NUM> (e.g., the middleware <NUM>) may include at least one of a runtime library <NUM>, an application manager <NUM>, a window manager <NUM>, a multimedia manager <NUM>, a resource manager <NUM>, a power manager <NUM>, a database manager <NUM>, a package manager <NUM>, a connectivity manager <NUM>, a notification manager <NUM>, a location manager <NUM>, a graphic manager <NUM>, a security manager <NUM>, and a payment manager <NUM>.

The runtime library <NUM> may include, for example, a library module that a complier uses to add a new function through a programming language while the application <NUM> is running. The runtime library <NUM> may perform a function for input/output management, memory management, or an arithmetic function.

The application manager <NUM> may mange, for example, a life cycle of at least one of the applications <NUM>. The window manager <NUM> may manage a GUI resource used in a screen. The multimedia manager <NUM> may recognize a format required for playing various media files and may encode or decode a media file using a codec matched to the format. The resource manager <NUM> may manage a resource such as a source code, a memory, or a storage space of at least one of the applications <NUM>.

The power manager <NUM>, for example, may operate together with a basic input/output system (BIOS) to manage a battery or power and may provide power information required for operating the electronic device. The database manager <NUM> may generate, search, or modify a database to be used in at least one of the applications <NUM>. The package manager <NUM> may manage installation or update of an application distributed in a package file format.

The connectivity manger <NUM> may manage wireless connection of Wi-Fi, Bluetooth, or the like. The notification manager <NUM> may display or notify an event such as message arrival, appointments, and proximity alerts in such a manner as not to disturb a user. The location manager <NUM> may manage location information of the electronic device. The graphic manager <NUM> may manage a graphic effect to be provided to a user or a user interface related thereto. The security manager <NUM> may provide various security functions required for system security or user authentication. According to an embodiment of the present disclosure, in the case in which an electronic device (e.g., the electronic device <NUM>) includes a phone function, the middleware <NUM> may further include a telephony manager for managing a voice or video call function of the electronic device.

The middleware <NUM> may include a middleware module for forming a combination of various functions of the above-mentioned elements. The middleware <NUM> may provide a module specialized for each type of an OS to provide differentiated functions. Furthermore, the middleware <NUM> may delete a part of existing elements or may add new elements dynamically.

The API <NUM> (e.g., the API <NUM>) which is, for example, a set of API programming functions may be provided in different configurations according to an OS. For example, in the case of Android or iOS, one API set may be provided for each platform, and, in the case of Tizen, at least two API sets may be provided for each platform.

The application <NUM> (e.g., the application program <NUM>), for example, may include at least one application capable of performing functions such as a home <NUM>, a dialer <NUM>, an short message service (SMS)/multimedia messaging service (MMS) <NUM>, an instant message (IM) <NUM>, a browser <NUM>, a camera <NUM>, an alarm <NUM>, a contact <NUM>, a voice dial <NUM>, an e-mail <NUM>, a calendar <NUM>, a media player <NUM>, an album <NUM>, a clock <NUM>, a payment application <NUM>, health care (e.g., measure an exercise amount or blood sugar), or environmental information provision (e.g., provide air pressure, humidity, or temperature information).

According to an embodiment of the present disclosure, the application <NUM> may include an information exchange application for supporting information exchange between the electronic device (e.g., the electronic device <NUM>) and an external electronic device (e.g., the first electronic device <NUM> or the second external electronic device <NUM>). The information exchange application may include, for example, a notification relay application for relaying specific information to the external electronic device or a device management application for managing the external electronic device.

For example, the notification relay application may have a function for relaying, to an external electronic device (e.g., the first electronic device <NUM> or the second external electronic device <NUM>), notification information generated in another application (e.g., an SMS/MMS application, an e-mail application, a health care application, an environmental information application, or the like) of the electronic device. Furthermore, the notification relay application may receive notification information from the external electronic device and may provide the received notification information to the user.

The device management application, for example, may manage (e.g., install, delete, or update) at least one function (e.g., turn-on/turn off of the external electronic device itself (or some elements) or the brightness (or resolution) adjustment of a display) of the external electronic device (e.g., the first electronic device <NUM> or the second external electronic device <NUM>) communicating with the electronic device, an application running in the external electronic device, or a service (e.g., a call service, a message service, or the like) provided from the external electronic device.

According to an embodiment of the present disclosure, the application <NUM> may include a specified application (e.g., a healthcare application of a mobile medical device) according to an attribute of the external electronic device (e.g., the first electronic device <NUM> or the second external electronic device <NUM>). The application <NUM> may include an application received from an external electronic device (e.g., the first electronic device <NUM> or the second external electronic device <NUM>). The application <NUM> may include a preloaded application or a third-party application downloadable from a server. The names of the elements of the program module <NUM> illustrated may vary with the type of an operating system.

According to various embodiments of the present disclosure, at least a part of the program module <NUM> may be implemented with software, firmware, hardware, or a combination thereof. At least a part of the program module <NUM>, for example, may be implemented (e.g., executed) by a processor (e.g., the processor <NUM>). At least a part of the program module <NUM> may include, for example, a module, a program, a routine, sets of instructions, or a process for performing at least one function.

The term "module" used herein may represent, for example, a unit including one of hardware, software and firmware or a combination thereof. The term "module" may be interchangeably used with the terms "unit", "logic", "logical block", "component" and "circuit". The "module" may be a minimum unit of an integrated component or may be a part thereof. The "module" may be a minimum unit for performing one or more functions or a part thereof. The "module" may be implemented mechanically or electronically. For example, the "module" may include at least one of an application-specific integrated circuit (ASIC) chip, a field-programmable gate array (FPGA), and a programmable-logic device for performing some operations, which are known or will be developed.

At least a part of devices (e.g., modules or functions thereof) or methods (e.g., operations) according to various embodiments of the present disclosure may be implemented as instructions stored in a non-transitory computer-readable storage medium in the form of a program module. In the case where the instructions are performed by a processor (e.g., the processor <NUM>), the processor may perform functions corresponding to the instructions. The non-transitory computer-readable storage medium may be, for example, the memory <NUM>.

The non-transitory computer-readable recording medium may include a hard disk, a floppy disk, a magnetic medium (e.g., a magnetic tape), an optical medium (e.g., compact disc-ROM (CD-ROM), digital versatile disc (DVD)), a magneto-optical medium (e.g., a floptical disk), or a hardware device (e.g., a ROM, a RAM, a flash memory, or the like). The program instructions may include machine language codes generated by compilers and high-level language codes that can be executed by computers using interpreters. The above-mentioned hardware device may be configured to be operated as one or more software modules for performing operations of various embodiments of the present disclosure and vice versa.

For example, an electronic device may include a processor and a memory for storing computer-readable instructions. The memory may include instructions for performing the above-mentioned various methods or functions when executed by the processor.

Claim 1:
An electronic device (<NUM>) comprising:
a camera (<NUM>) configured to sequentially obtain, when a panorama image capture starts, a first plurality of images; and
a motion sensor (<NUM>) configured to sense, during the panorama image capture, a motion of the electronic device (<NUM>), thereby obtaining motion information;
a panorama image generator (<NUM>) configured to:
generate, during the panorama image capture, image analysis information for each image in the first plurality of images,
select, during the panorama image capture, a second plurality of images (<NUM>, <NUM>) among the first plurality of images based on the image analysis information and on the motion information,
repeatedly perform image matching and stitching, during the panorama image capture based on the second plurality of images (<NUM>, <NUM>), on the image analysis information and on the motion information, thereby generating panorama image data which can be displayed as a panorama image;
a dynamic panorama image generator (<NUM>) configured to:
generate, during the panorama image capture, dynamic panorama data based on the second plurality of images (<NUM>, <NUM>), on the image analysis information and on the motion information,
wherein the dynamic panorama data can be displayed as a panorama video having a virtual field of view identical to the field of view of the panorama image, and
wherein the dynamic panorama data is temporally and spatially synchronized with the panorama image data; and
a last file generator (<NUM>) configured to:
generate, when the image capture stops, a panorama content file including at least the panorama image data and the dynamic panorama data, and
store, in a memory (<NUM>), the panorama content file comprising the panorama image data, the dynamic panorama data.