ELECTRONIC DEVICE, METHOD AND NON-TRANSITORY STORAGE MEDIUM FOR IMAGE PROCESSING

An electronic device includes a memory, a display and a processor configured to set at least one virtual plane in a three-dimensional (3D) virtual position based on at least a portion of at least one two-dimensional (2D) first image displayed on the display, set a 3D virtual space comprising the at least one virtual plane and the at least one 2D first image, insert a 3D object into the 3D virtual space, and generate at least one 2D second image by synthesizing the 3D object inserted into the 3D virtual space and the at least one 2D first image.

BACKGROUND

The disclosure relates to an electronic device, a method, and a non-transitory storage medium for image processing for providing a three-dimensional (3D) effect.

2. Description of Related Art

The technology for image processing may process an image by adding text, images, animations, or sounds to one or more images and provide content that meets the user's preference according to the processed image.

An electronic device for a general image processing technique may capture a three-dimensional (3D) image using a camera (e.g., depth camera) for 3D capturing based on augmented reality and apply a 3D object to the 3D space of the captured 3D image to create augmented reality-based content. The electronic device for image processing may change the 3D object into a two-dimensional (2D) image based on an image synthesis technique and synthesize the changed 2D image with the captured 2D image to thereby apply the 3D object to the 2D image

Since major general image processing techniques process images using the result of analysis of the 2D image, and actual synthesis solutions are based on synthesizing an image and another image, such methods are primarily used that convert 3D rendering images into 2D images and synthesize them.

However, the general image processing technique requires 3D-to-2D image conversion to process the 3D image and suffers from a complicated image processing process and need for additional components for image processing.

Further, a generally captured image is a 2D image and, to provide a 3D image, a 3D image may be processed by identifying 3D space information from the 3D image captured using a 3D capturing camera (e.g., depth camera).

The 3D capturing camera is expensive and is not widely used in reality. Further, with a general electronic device for image processing, it is difficult to identify information about the 3D space in a situation without a 3D capturing camera. Thus, only with general image processing technology, it is hard to provide a 3D effect to a 2D first image to be displayed on a display.

SUMMARY

Provided are an electronic device, method, and non-transient storage medium, for providing a three-dimensional (3D) effect to a two-dimensional (2D) image.

According to an aspect of the disclosure, an electronic device includes: a memory; a display; and a processor configured to: set at least one virtual plane in a three-dimensional (3D) virtual position based on at least a portion of at least one two-dimensional (2D) first image displayed on the display, set a 3D virtual space including the at least one virtual plane and the at least one 2D first image, insert a 3D object into the 3D virtual space, and generate at least one 2D second image by synthesizing the 3D object inserted into the 3D virtual space and the at least one 2D first image.

The processor may be further configured to: control the display to display the at least one 2D second image, map the at least one 2D second image with reproduction time information related to the at least one 2D first image, and store the mapped at least one 2D image in the memory.

The processor may be further configured to: identify a 2D position on the display to which the at least one virtual plane is projected, control the display to display a first control object for controlling the at least one virtual plane in the 2D position, and based on an edit request, execute an application for image processing and control the display to display an execution screen of the application including the at least one 2D first image and menus for processing the at least one 2D first image, and the first control object may be overlaid on a portion of the at least one 2D first image displayed in the 2D position and is configured to adjust at least one of a size, movement, rotation, or depth of the at least one virtual plane.

The processor may be further configured to: set a first virtual position for inserting the 3D object in the 3D virtual space based on a first type selected from among types for inserting the 3D object, the first virtual position being a 3D virtual position in which at least a portion of the 3D object is attached to the at least one virtual plane, add the 3D object in the first virtual position, control the display to display a second control object for controlling the 3D object, the second control object being displayed in a 2D position on an area of the display to which the 3D object is projected, and set a 3D visual effect by controlling the 3D object using the second control object, and the 3D object added in the first virtual position is configured to be capable of at least one of rotation about one axis, movement or resizing with respect to two axes.

The processor may be further configured to: set a second virtual position for inserting the 3D object in the 3D virtual space based on a second type selected from among types for inserting the 3D object, the second virtual position being a 3D virtual position in which the 3D object does not overlap the at least one virtual plane, add the 3D object in the second virtual position, control the display to display a third control object for controlling the 3D object, the third control object being displayed in a 2D position on an area of the display to which the 3D object is projected, and set a 3D visual effect by controlling the 3D object using the third control object, and the 3D object may be added in the second virtual position is configured to be capable of at least one of rotation about three axes, movement or resizing with respect to the three axes.

The processor may be further configured to: set an additional virtual plane based on a predetermined number of virtual planes or an additional selection, and based on the at least one virtual plane including a plurality of virtual planes, set a 3D visual effect to the 3D object such that the 3D object dynamically moves within the 3D virtual space without departing from the plurality of virtual planes in the 3D virtual space set by the plurality of virtual planes.

The processor may be further configured to: identify a moved position of the 3D object, based on a request for moving the 3D object from a first virtual plane among the at least one virtual plane to a second virtual plane, rotate the 3D object to allow the moved position of the 3D object to be parallel with the second virtual plane based on the moved position of the 3D object being a position in a boundary line of the second virtual plane, and insert the rotated 3D object into the second virtual plane.

The processor may be further configured to: obtain a plurality of first images included in a reproduction time period from a first time for reproducing image data including the at least one 2D first image to a second time, the plurality of first images including the at least one 2D first image, generate 2D second images respectively corresponding to the plurality of first images by applying the 3D object, inserted into the 3D virtual space, to each of the plurality of first images, and map the plurality of first images included in the reproduction time period to each of the 2D second images, or replace the plurality of first images with the 2D second images and store the 2D second images in the memory.

According to an aspect of the disclosure, a method for image processing in an electronic device, includes: setting at least one virtual plane in a three-dimensional (3D) virtual position related to at least a portion of at least one dimensional (2D) first image displayed on a display of the electronic device; setting a 3D virtual space including the at least one virtual plane and the at least one 2D first image; inserting a 3D object into the 3D virtual space; and generating at least one 2D second image by synthesizing the 3D object inserted into the 3D virtual space and the at least one 2D first image.

The method may further include: controlling the display to display the at least one 2D second image; mapping the at least one 2D second image with reproduction time information related to the at least one 2D first image; and storing the mapped at least one 2D second image in a memory.

The method may further include: identifying a 2D position on the display to which the at least one virtual plane is projected; and controlling the display to display a first control object for controlling the at least one virtual plane in the 2D position, the first control object may be overlaid on a portion of the at least one 2D first image displayed in the 2D position and adjusts at least one of a size, movement, rotation, or depth of the at least one virtual plane, and the method may further include, based on an edit request, executing an application for image processing and controlling the display to display an execution screen of the application including the at least one 2D first image and menus for processing the at least one 2D first image.

The inserting the 3D object into the 3D virtual space may include: setting a first virtual position for inserting the 3D object in the 3D virtual space based on a first type selected from among types for inserting the 3D object, the first virtual position being a 3D virtual position in which at least a portion of the 3D object is attached to the at least one virtual plane; adding the 3D object to the first virtual position; controlling the display to display a second control object for controlling the 3D object, the second control object being displayed in a 2D position on an area of the display to which the 3D object is projected; and setting a 3D visual effect by controlling the 3D object using the second control object, and the 3D object added in the first virtual position may be configured to be capable of at least one of rotation about one axis, movement or resizing with respect to two axes.

The inserting the 3D object into the 3D virtual space may include: setting a second virtual position for inserting the 3D object in the 3D virtual space based on a second type selected from among types for inserting the 3D object, the second virtual position being a 3D virtual position in which the 3D object does not overlap the at least one virtual plane; adding the 3D object to the second virtual position; controlling the display to display a third control object for controlling the 3D object, the third control object being displayed in a 2D position on an area of the display to which the 3D object is projected; and setting a 3D visual effect by controlling the 3D object using the third control object, and the 3D object added in the second virtual position may be configured to be capable of at least one of rotation about three axes, movement or resizing with respect to the three axes.

The inserting the 3D object into the 3D virtual space may include: identifying a moved position of the 3D object, based on a request for moving the 3D object from a first virtual plane among the at least one virtual plane to a second virtual plane; rotating the 3D object to allow the moved position of the 3D object to be parallel with the second virtual plane based on the moved position of the 3D object being a position in a boundary line of the second virtual plane; and inserting the rotated 3D object into the second virtual plane. According to an aspect of the disclosure, a non-transitory computer-readable storage medium stores instructions that, when executed by a processor, cause the processor to: set at least one virtual plane in a three-dimensional (3D) virtual position related to at least a portion of at least one two-dimensional (2D) first image displayed on a display of an electronic device; set a 3D virtual space including the at least one virtual plane and the at least one 2D first image; insert a 3D object into the 3D virtual space; and generate at least one 2D second image by synthesizing the 3D object inserted into the 3D virtual space and the at least one 2D first image.

By the electronic device and operation method in the electronic device according to an embodiment, it is possible to identify information about a 3D space related to a 2D image even in a situation free of a camera for 3D capturing (e.g., depth camera) and simply provide a 3D effect to a 2D image by generating a synthesized image of a 2D image and a 3D object using an image processing application.

By the electronic device and operation method in the electronic device according to an embodiment, it is also possible to simply perform image processing for providing a 3D effect using an image processing application without the need for performing a complicated process to process a 3D object into a 2D image.

DETAILED DESCRIPTION

Various embodiments of the present disclosure are now described with reference to the accompanying drawings.

FIG. 2is a diagram of an example configuration of an electronic device according to an embodiment.

Referring toFIGS. 1 and 2, according to an embodiment, an electronic device101(e.g., the first electronic device101ofFIG. 1) may include a processor120, a memory130, an input device150, and a display161(e.g., the display device160ofFIG. 1). Further, the electronic device101described with reference toFIG. 2may further include other components described with reference toFIG. 1.

According to an embodiment, the processor120of the electronic device101may be configured to operate while being electrically connected to the memory130, the input device150, and the display161.

According to an embodiment, the processor120of the electronic device101may be configured to obtain image data including two-dimensional (2D) images (hereinafter, referred to as a first image) for image processing and to perform an operation for processing at least one 2D first image included in the obtained image data. To process the at least one 2D first image, the processor120may execute an application (program or function) for image processing stored in the memory130. The image data may be at least one still image (image or photo) or a moving image (or video). The processor120may set at least one virtual plane in a three-dimensional (3D) virtual position related to at least a portion of the at least one 2D first image (still image or frame) displayed on the display161, using the executed application. The processor120may set a 3D virtual plane including the at least one set virtual plane and the at least one first image, insert a 3D object into the set 3D virtual plane, and synthesize the 3D object inserted into the set 3D virtual plane and the at least one first image displayed, generating at least one 2D synthesized image (hereinafter, referred to as a second image). The processor120may be configured to apply the at least one second image to the image data and play it, by mapping the at least one 2D first image to the at least one 3D object-inserted 2D second image or replacing with the at least one 3D object-inserted 2D second image. The at least one first image may be at least one 2D image of, for example, a main image, a thumbnail image, an intro image, or an outro image provided using the executed application for image processing.

According to an embodiment, the processor120may identify the position of the first image (e.g., the 2D position (x, y coordinates) in the display area displaying the first image) displayed on the display161to which the at least one set virtual plane is projected, using at least one sensor. The processor120may control the display161to display a first control object for controlling the at least one virtual plane in the 2D position. The first control object may be displayed overlaid on part of the at least one first image displayed in the 2D position and be configured to adjust at least one of the size, movement, rotation, or depth of the at least one virtual plane.

According to an embodiment, the processor120of the electronic device101may control to obtain image data of the at least one first image (image, still image, or picture) or moving image (video or dynamic image) and store the obtained image data. The processor120may be configured to process the operation of the image processing module210included in the memory130. The processor120may be configured to process operations related to image reproduction. The processor120may process (e.g., encode and decode) the image or image data using a predetermined image processing format. The processor120may perform image processing (e.g., encoding or decoding) on at least one image using a predetermined image processing format.

According to an embodiment, the memory130of the electronic device101may be configured to including the image processing module210corresponding to at least one application (e.g., the application146ofFIG. 1(e.g., an application for image processing)) processed and executed by the processor120. The memory may store multimedia content and store 3D objects for applying a 3D effect to part of a video or at least one first image for the multimedia content. The image processing module210may be configured software-wise, as a module corresponding to a program (e.g., the program140ofFIG. 1) and may correspond to an application for image processing (e.g., the application146ofFIG. 1).

According to an embodiment, the image processing module210may store instructions that, when executed by the processor120, enable the processor120to perform setting at least one virtual plane in a 3D virtual position related to at least a portion of at least one 2D first image displayed on the display161, setting a 3D virtual space including the at least one virtual plane and the at least one first image, inserting a 3D object into the 3D virtual space, and generating at least one 2D second image by synthesizing the 3D object inserted into the 3D virtual space and the at least one first image.

According to an embodiment, the image processing module210may be configured to include executable instructions that, when executed by the processor120, enable the processor120to perform operations related to image processing on at least part of a video or at least one first image.

According to an embodiment, the image processing module210may be configured to include executable instructions that, when executed by the processor120, enable the processor120to perform an operation for applying a 3D effect to at least one first image which is at least part of a video frame in response to a request for image processing. The image processing module210may include an image processing execution module211, an image processing setting module213, and a 3D object management module215to apply a 3D effect. However, the components included in the image processing module210are not limited thereto, and the image processing module210may further include other modules. The image processing execution module211, the image processing setting module213, and the 3D object management module215may be software modules and correspond to at least part of the image processing application. According to an embodiment, although the image processing module210is described as being a software module, embodiments are not limited thereto, and the whole or at least part of the image processing module210may be configured as a hardware module.

According to an embodiment, the processor120may execute the application for image processing by execution of the image processing execution module211, set execution screens for image-processing at least one first image, and control the display161to display the set execution screens. The processor120may perform operations related to reproduction of image data by execution of the image processing execution module211and control the display161to select and display at least one 2D first image for image processing on the image data. The processor120may control to display and store at least one second image generated by the image processing setting module213by execution of the image processing execution module211.

According to an embodiment, by execution of the image processing setting module213, the processor120may set at least one virtual plane related to at least one 2D first image, set a 3D virtual space based on the at least one set virtual plane, and set to insert a 3D object into the set 3D virtual space. By execution of the image processing setting module213, the processor120may set a first control object for adjusting the size, movement, rotation, or depth of the at least one set virtual plane and set at least one of a second control object or third control object for adjusting the size, movement, rotation, or depth of the 3D virtual object inserted into the 3D virtual space. By execution of the image processing setting module213, the processor120may generate at least one 2D second image by synthesizing the 3D object inserted into the set 3D virtual space and at least one 2D first image. By execution of the image processing setting module213, the processor120may display the at least one generated 2D second image on the execution screen, map the at least one generated second image with information related to the reproduction time of the at least one first image (e.g., a reference point for image processing of a timeline) and store it in the memory130. By execution of the image processing setting module213, the processor120may map the at least one generated second image with the at least one first image and store it in the memory130or may replace the at least one first image with the at least one second image and store it in the memory130. According to an embodiment, by execution of the image processing setting module213, if a predetermined reproduction time (cut) is selected on a time ruler of the timeline, the processor120may insert the 3D object into the first image corresponding to the selected reproduction time (cut), generating the second image. According to an embodiment, by execution of the image processing setting module213, upon selecting to process a first image to be reproduced during a predetermined reproduction time period on the time ruler of the timeline, the processor120may insert the same 3D object into the first images (frames) corresponding to the reproduction time period from the selected start point (Tcut0) to the end point (Tcut1), generating each second image. The 3D object may be inserted into each of a plurality of first images of the reproduction time period selected in the same 3D virtual position and under the same insertion condition (e.g., a condition related to at least one of the type of the 3D object, movement, lighting, material, shape, size, color, or brightness). By execution of the image processing setting module213, the processor120may set the at least one set virtual plane and set 3D virtual space to be identical for the plurality of first images of the selected reproduction time period and may insert the 3D object, which is dynamically moved over time, onto the 3D virtual space set as identical.

According to an embodiment, by execution of the image processing setting module213, the processor120may set at least one virtual plane in the virtual 3D position corresponding to the at least one 2D first image. By execution of the image processing setting module213, the processor120may set at least one of the set type of the virtual plane, the set position of the virtual plane, the number of virtual planes, or the shape of the virtual plane. By execution of the image processing setting module213, the processor120may control the display161to display at least one menu (e.g., a menu for indicating at least one of the set type of the virtual plane, the set position of the virtual plane, the number of the virtual planes, or the shape of the vehicle) for setting the virtual plane on the execution screen of image processing. By execution of the image processing setting module213, the processor120may set a 3D virtual space including the set virtual plane and the at least one 2D first image. By execution of the image processing setting module213, the processor120may set a control object for controlling the set 3D virtual space and control the display161to display the control object overlaid on the at least one 2D first image.

According to an embodiment, by execution of the 3D object management module215, the processor120may configure, or obtain from an external device, at least one 3D object to be inserted into the 3D virtual space to apply a 3D effect to the at least one first image, and control to store the at least one configured or obtained 3D object in the memory130. By execution of the 3D object management module215, the processor120may transfer at least one selected from among the 3D objects stored in the memory130to the image processing execution module211or the image processing setting module213to synthesize with the at least one 2D first image. By execution of the 3D object management module215, the processor120may generate or edit a 3D object using the text or image input from the user through the input device150.

According to an embodiment, the image processing module210may include at least one of a 3D deco module, a thumbnail module, an intro module, an outro module, a subtitle module, an edit module, an export module, or a setup module. The image processing execution module211, the image processing setting module213, and the 3D object management module215may be included in, e.g., the 3D deco module.

According to an embodiment, the display161of the display device160in the electronic device may display screens related to the application for image processing corresponding to the image processing module210executed by the processor120. The display161may display at least one first image or a video. The display161may display at least part of the video or at least one first image to be processed through the application for image processing and display the second image generated by processing the at least part of the video or at least one first image.

Major components of the electronic device101have been described above in connection withFIG. 2. According to an embodiment, however, all of the components ofFIG. 2are not essential components, and each electronic device101may be implemented with more or less components than those shown. The positions of the major components of each electronic device101described above in connection withFIG. 2may be varied according to various embodiments.

FIGS. 3A, 3B, 3C and 3Dare diagrams of an example of an edit screen of an electronic device according to an embodiment. Described in connection withFIGS. 3A to 3Dis an example of processing at least one first image corresponding to at least one frame of a video executed on an application for convenience of description, and a plurality of still images, such as photos, may be edited in the same manner.

Referring toFIGS. 3A and 3B, according to an embodiment, a processor (e.g., the processor (e.g., the processor120ofFIGS. 1 and 2) of an electronic device (e.g., the electronic device101ofFIGS. 1 and 2) may control the image processing module210stored in the memory130to execute an application (program or function)310for image processing. If the application for image processing is executed, the processor120may control the display161to display an execution screen311for image processing. The execution screen311may be an execution screen displayed by execution of a 3D deco module, e.g., according to selection of a menu (image, button, or object)341indicating the 3D deco module. The execution screen311may be set as a first area311afor displaying at least one first image301, a second area311bfor displaying information indicating the type of the virtual plane or information related to selection or input of 3D objects to be applied onto the virtual plane, a third area311cfor displaying an image (e.g., timeline or time ruler)330related to the reproduction time of the video, and a fourth area311dfor displaying a menu (image, button, or object) (e.g., at least one of a home, thumbnail, intro, edit, 3D deco subtitle, or outro) indicating at least one operation (e.g., modules included in the image processing module210) of the application (program or function)310for image processing. The fourth area311dof the execution screen311may include a menu (image, button, or object)343for indicating the operation for setting the virtual plane and a menu (image, button, or object)345for indicating the operation for inputting or selecting the 3D object. The second area311bof the execution screen311may include at least one of a menu (image, button, or object) for indicating virtual plane setting types321and323, a menu (image, button, or object) (add) selectable to add a new virtual plane, a menu (image, button, or object) (list) for displaying the virtual plane set thus far, or a menu (help) for guiding a method for inserting a virtual plane. The second area311bmay further include various menus (image, button, or object) for processing. The processor120may control the display161to display, in the first area311a, a 2D (x, y) first image (still image or frame) corresponding to the reproduction time (Tcut0)331selected from the obtained video data in response to an image processing request. As shown inFIG. 3B, in response to selection of a first type (e.g., one plane)321from among the virtual plane setting types on the execution screen311, the processor120may set at least one virtual plane303ain a virtual 3D (x, y, z) position so that at least part of the 2D first image301displayed on the first area311ais positioned apart in the vertical direction and in parallel therewith. The processor120may display a first control object305on the first area311ato overlap at least part of the 2D first image. The first control object305may include at least one of a first element305a(e.g., vertexes or guide line of the rectangular box) of the x- and y-axis movement for resizing the virtual plane303a, a second element305bfor rotation on the z axis for rotation, a third element305cof left/right rotation on the y axis or upward/downward rotation on the x axis for rotation, or a fourth element305dof movement on the z axis for depth adjustment. The first control object305may be automatically set and displayed as the menu displayed on the execution screen311is selected or at least one virtual plane is set. The first control object305may perform one-axis rotation or two-axis movement.

Referring toFIGS. 3C and 3D, the processor120may change the virtual plane303aby adjusting the size, movement, or rotation of the control object305to allow the virtual plane303ato correspond to another area of the 2D first image301. As the control object305is manipulated, the virtual plane303amay be rotated by 90 degrees, with its x axis fixed, so that the processor120may set a changed virtual plane303bas shown inFIG. 3D. As shown inFIG. 3A or 3C, as a predetermined number of virtual planes or a first type (e.g., one plane)321for setting a virtual plane is additionally selected, the processor120may additionally set another virtual plane in another 3D virtual position (x, y, and z values). For example, the processor120may adjust the control object305to allow the virtual plane303ato be rotated by 90 degrees, with the y axis of the added virtual plane fixed, to thereby change the 3D virtual position of the other virtual plane so that a virtual wall surface may be formed on the y-axis side surface perpendicular to the virtual plane303b.

FIGS. 4A, 4B, 4C and 4Dare diagrams of an example for image processing in an electronic device according to an embodiment. Described in connection withFIGS. 4A to 4Dis an example of processing at least one first image corresponding to at least one frame of a video executed on an application for convenience of description, and a plurality of still images, such as photos, may be processed in the same manner.

Referring toFIGS. 4A and 4B, according to an embodiment, the processor (e.g., the processor120ofFIGS. 1 and 2) of the electronic device (e.g., the electronic device101ofFIGS. 1 and 2) may control the display161to display an execution screen311for image processing and, in response to an image edit request, display a 2D (x, y) image (still image or frame)301corresponding to a reproduction time (Tcut0)331selected from obtained video data in a first area311aof the execution screen311. In response to selection of a second type (e.g., cube plane)323from among the virtual plane setting types on the execution screen311, the processor120may set virtual planes, shaped as a cube401a, in a virtual 3D (x, y, z) position, in a vertical direction, corresponding to the 2D first image301displayed on the first area311a. The processor120may display a first control object305on the first area311ato overlap at least part of the 2D first image301. The first control object305may be set to have the same shape and size, corresponding to the center plane411among the virtual planes of the cube shape401a. The first control object305may include at least one of a first element305a(e.g., vertexes or guide line of the rectangular box) of the x- and y-axis movement for resizing the center plane411among the virtual planes, a second element305bfor rotation on the z axis for rotation, a third element305cof left/right rotation on the y axis or upward/downward rotation on the x axis for rotation, or a fourth element305dof movement on the z axis for depth adjustment. The first control object305may be automatically set and displayed as the menu displayed on the execution screen311is selected or at least one virtual plane is set. The first control object305may perform three-axis rotation or three-axis movement.

Referring toFIGS. 4C and 4D, the processor (e.g., the processor120ofFIGS. 1 and 2) may change the virtual planes of the cube shape401a, as shown inFIGS. 4A and 4B, by resizing the first control object305. The processor120may resize the center plane411among the virtual planes by adjusting at least one of four vertexes of the first control object305. As the center plane411is enlarged in the x-axis and y-axis directions with respect to the four vertexes of the first control object305, the remaining virtual planes all are also resized as shown inFIG. 4C, so that the processor120may set virtual planes shaped as the changed cube401bas shown inFIG. 4D.

FIGS. 5A and 5Bare diagrams of an example for image processing in an electronic device according to an embodiment.

Referring toFIG. 5A, according to an embodiment, the processor (e.g., the processor120ofFIGS. 1 and 2) of the electronic device (e.g., the electronic device101ofFIGS. 1 and 2) may identify that, among the functions of the application for image processing corresponding to the image processing module210, e.g., the function corresponding to the 3D deco module is selected, and a menu (image, button, or object)345for indicating the operation for inputting or selecting a 3D object is selected. In this case, the processor120may control the display161to display information for selecting or inputting the 3D object in the second area311bof the execution screen311. The information for selection or input may include a menu for selecting the type of the 3D object, e.g., at least one of media, text, sound, effect, or sticker and may include a menu (image or object)503for inputting or searching for the 3D object or a menu (image, button, or object)505indicating the functions for changing the shape of the object, e.g., font. The second area311bmay be further set with various menus (images, buttons, or objects). If, e.g., a text-type 3D object (e.g., blockbuster)501is selected or input, the processor120may apply the 3D object501to the first image301displayed in the first area311by inserting the 3D object onto the set virtual plane to present a 3D effect as described in connection withFIGS. 3A to 3D.

Referring toFIG. 5B, according to an embodiment, the processor (e.g., the processor120ofFIGS. 1 and 2) of the electronic device (e.g., the electronic device101ofFIGS. 1 and 2) may set a 3D virtual space including the set virtual plane303band the 2D first image301as described in connection withFIG. 3C. The processor120may insert the selected or predetermined 3D object511or521onto the 3D virtual space. When inserting the 3D object511or521into the set virtual space, the processor120may insert the 3D object511or521based on one type selected from among set types of the 3D object. The types of the 3D object may include a first type (plane attach type) of attaching and inserting the 3D object511to the set plane to present a 3D visual effect and a second type (plane interaction type)520of inserting the 3D object521to present such a 3D visual effect as if the 3D object floats over the set plane in a direction perpendicular to the set plane. The first type510may be divided into a static type capable of setting a media (e.g., image or video) and 2D sticker (e.g., PNG image) object and a dynamic type (animated type) capable of setting a text and visual effect (VFX) (e.g., magic circle) object. The second type520may be divided into a static type capable of setting a text (e.g., shadowed text) object and a dynamic type (animated type) capable of setting a text (e.g., standing text), visual effect (VFX) (e.g., soap bubbles, fireworks, bouquet, or smoke), emotion sticker (emoticon) (e.g., joy, anger, annoyance, and cry), or 3D sticker (e.g., like or subscribe) object.

FIGS. 6A and 6Bare diagrams of an example for image processing in an electronic device according to an embodiment.

Referring toFIGS. 6A and 6B, according to an embodiment, the processor (e.g., the processor120ofFIGS. 1 and 2) of the electronic device (e.g., the electronic device101ofFIGS. 1 and 2) may set a 3D virtual space including the set virtual plane303band the 2D first image301as described in connection withFIG. 3C. If the selected type of the 3D object is the first type, the processor120may insert the 3D object601onto the set virtual space by attaching the 3D object601to the set plane303bto present a visual effect. The processor120may generate at least one 2D second image by synthesizing the 3D object601, inserted based on the first type, and the 2D first image301.

Referring back toFIGS. 6A and 6B, when inserting the 3D object601, the processor120may control the display161to display a second control object603to adjust the size and direction of the 3D object601. The second control object603may be displayed in response to a selection input of the 3D object601and be configured to be capable of one-axis rotation, two-axis movement, and scaling. The second control object603may include an element603a(vertexes of the rectangular box or guide lines) for dependently moving on the virtual plane303band an element603bfor dependently rotating on the virtual plane303b.

FIGS. 7A and 7Bare diagrams of an example for image processing in an electronic device according to an embodiment.

Referring toFIGS. 7A and 7B, according to an embodiment, the processor (e.g., the processor120ofFIGS. 1 and 2) of the electronic device (e.g., the electronic device101ofFIGS. 1 and 2) may set a 3D virtual space including the set virtual plane303band the 2D first image301as described in connection withFIG. 3C. If the selected type of the 3D object701is the second type (plane interaction type), the processor120may insert the 3D object701to present such a 3D visual effect as if the 3D object701floats over the set plane303bin the direction perpendicular to the set plane303b. The processor120may generate a 2D second image by synthesizing the 3D object701, inserted based on the second type, and the 2D first image301.

Referring back toFIGS. 7A and 7B, the processor120may control the display161to display a third control object703to adjust the size, direction, and depth of the 3D object701. The third control object703may be displayed in response to a selection input of the 3D object701and be configured to be capable of three-axis rotation, tree-axis movement, and scaling. The third control object703may include at least one of an element703a(e.g., vertexes of the rectangular box or guide lines) for resizing (x-axis or y-axis movement) of the 3D object701, an element703bfor rotation on the z axis of the 3D object701, an element703cfor left/right rotation on the y axis or upward/downward rotation on the x axis of the 3D object701, or an element703dfor movement on the z axis for depth adjustment.

FIGS. 8A, 8B, and 8Care diagrams of an example for image processing in an electronic device according to an embodiment.

Referring toFIG. 8A, according to an embodiment, the processor (e.g., the processor120ofFIGS. 1 and 2) of the electronic device (e.g., the electronic device101ofFIGS. 1 and 2) may set a 3D virtual space803including virtual planes having a cube shape401band the 2D first image301as described in connection withFIG. 4C. The set 3D virtual space803may correspond to an inner space formed by the virtual planes of the cube shape401b. The processor120may insert the selected or predetermined 3D object805aonto the 3D virtual space. If the selected type of the 3D object805ais the first type, the processor120may insert the 3D object805aonto the 3D virtual space803by attaching the 3D object805ato at least one of the virtual planes of the cube shape401bto present a 3D visual effect. The processor120may generate a 2D second image by synthesizing the 3D object805a, inserted based on the first type, and the 2D first image801.

Referring toFIG. 8B, according to an embodiment, the processor (e.g., the processor120ofFIGS. 1 and 2) of the electronic device (e.g., the electronic device101ofFIGS. 1 and 2) may set a 3D virtual space803including virtual planes having a cube shape401band the 2D first image801as described in connection withFIG. 4C. The set 3D virtual space803may correspond to an inner space formed by the virtual planes of the cube shape 40 lb. The processor120may insert the selected or predetermined 3D object805bonto the 3D virtual space803. If the selected type of the 3D object is the second type, the processor120may insert the 3D object805bto present such a 3D effect as if the 3D object805bfloats over the center virtual plane811among the virtual planes of the cube shape 40 lb. The processor120may set a 3D visual effect (e.g., shadow)807to the inserted 3D object805b. The processor120may generate a 2D second image by synthesizing the 3D object805b, inserted based on the second type, and the 2D first image801.

Referring toFIG. 8C, according to an embodiment, the processor120of the electronic device101may set a 3D virtual space803aincluding the set cube shape (e.g., the virtual planes of401bofFIG. 4C) and the 2D first image801. The set 3D virtual space803may correspond to an inner space formed by the virtual planes of the cube shape (e.g.,401bofFIG. 4C). The processor120may insert the selected or predetermined 3D object809aonto the 3D virtual space803. For example, the processor120may insert the 3D object809ato present such a 3D visual effect as if the 3D object801is attached to one plane813among the virtual planes of the 3D object801in the 3D virtual space803. The processor120may set a 3D object insertion condition to present such a visual effect as if the 3D object809ais moved only inside the 3D virtual space803and, if the 3D object809amoves and reaches the boundary between the virtual planes, insert a 3D object809bset by rotating the 3D object809awithout continuously moving the 3D object809ain the extending direction on the displayed virtual plane813, onto the virtual plane set toward the moving direction, e.g., the virtual plane815on the left side of the virtual plane803.

The operation procedure in the above-described electronic device (e.g., the first electronic device101ofFIGS. 1 and 2) is described below in detail with reference to the accompanying drawings.

FIG. 9is a diagram of an example of an operation procedure of an electronic device according to an embodiment.

Referring toFIG. 9, in operation901, according to an embodiment, the electronic device (e.g., the first electronic device101ofFIGS. 1 and 2) may set at least one virtual plane in a virtual 3D virtual position related to at least a portion of at least one 2D first image displayed on the display (e.g., the display device ofFIG. 1and the display161ofFIG. 2). The electronic device may display the at least one 2D first image along with menus for image processing, on an edit execution screen of an image edit application. The at least one 2D first image may be at least some frames included in a video or at least one still image. The electronic device may set a virtual plane in a 3D virtual position parallel with the image displayed on the edit execution screen, identify the position of the first image displayed on the display, projected to the set virtual plane, and display, on the display, a first control object for controlling the virtual plane by interacting with the virtual plane set in the identified position of the first image. The identified position of the first image may be a position corresponding to the coordinate value (e.g., the 2D coordinate value (x, y)) of the first image displayed on the display. The electronic device may adjust at least one of the size, movement, or rotation of the set virtual plane using the first control object. The electronic device may display the first control object to correspond to the display area where the 3D object is to be inserted, by adjusting the first control object. The electronic device may identify the 3D virtual position back-projected to the virtual plane interacting according to the adjustment of the first control object and set the virtual plane in the identified 3D virtual position.

In operation903, the electronic device may set a virtual space including the at least one set virtual plane and the at least one first image.

In operation905, the electronic device may insert the 3D object to the set 3D virtual space. The electronic device may select a type for adding a 3D object and insert a predetermined or selected 3D object to the virtual space based on the selected type.

If the first type (plane attached type) is selected from among the 3D object types, the electronic device may set a 3D virtual position (x, y, z) where at least a portion of the 3D object is attached to the at least one virtual plane in the virtual space and add the 3D object in the set virtual position. The electronic device may display, on the edit execution screen, a second control object for adjusting the size or direction of the inserted 3D object. The second control object may be displayed overlaid on at least one first image displayed in the 2D position (x, y) of the display corresponding to the 3D virtual position where the 3D object is inserted. The electronic device may change at least one of the size, shape, or position of the inserted 3D object by moving or rotating the 3D object or resizing the 3D object by adjusting the second control object. The second control object may be rotated or moved on the virtual plane dependently on the virtual plane.

In operation907, the electronic device may generate at least one 2D second image by synthesizing the 3D object inserted into the 3D virtual space and the at least one first image. The electronic device may generate at least one 2D second image by rendering a 3D visual effect (e.g., at least one of movement, light, shadow, or animation) for the 3D object to interact with the image and the virtual plane on the set virtual space. The electronic device may map the at least one first image with the at least one second image or replace the at least one first image with the at least one second image and store in the memory.

FIG. 10is a diagram of an example of an operation procedure of an electronic device according to an embodiment.FIGS. 11A, 11B, 11C, 11D and 11Eare diagrams of an example of a screen for image processing in an electronic device according to an embodiment.

Referring toFIGS. 10 and 11A, according to an embodiment, in operation1001, the electronic device (e.g., the first electronic device101ofFIGS. 1 and 2) may execute an application (program or function)310for image processing and display execution screens of the application310for image processing on the display161(e.g., the display device ofFIG. 1and the display161ofFIG. 2). The application310for image processing may be an all-in-one application capable of performing all of, e.g., video reproduction, image (or video) creation, and image processing or an individual application for image (or video) creation or image processing. As shown inFIG. 11A, the electronic device may display a first execution screen1110(e.g., a home screen) of the executed application and display menus1111for selecting at least one multimedia content on the first execution screen1110. The menus1111are menus in which multimedia content is divided per theme (or genre). Some images1113of at least one multimedia content corresponding to the menu selected from among the menus1111may be displayed in a partial area of the first execution screen.

In operation1003, the electronic device may obtain image data for processing. According to an embodiment, as shown inFIG. 11B, in response to selection of one from among the menus1011displayed on the first execution screen, the electronic device may obtain image data of at least one multimedia content corresponding to the selected menu. As shown inFIG. 11B, the electronic device may select specific images (e.g., a main image1121of the multimedia content, a thumbnail image1123, an intro image1125, and an outro image1127) from the obtained image data and display the selected specific images on a second execution screen (e.g., home)1120of the application310. The electronic device may display, on the first execution screen1110, menus (or object) for checking whether each of the selected specific images1121,1123,1125, and1127is edited. According to an embodiment, the electronic device may perform an image edit operation on each of the main image1121, the thumbnail image1123, the intro image1125, and the outro image1127. According to another embodiment, the electronic device may apply the second images generated according to image processing on the main image1121to at least one of the thumbnail image1123, the intro image1125, or the outro image1127.

In operation1005, in response to an edit request, the electronic device may select at least one first image for processing from the obtained image data and display the at least one selected first image on a third execution screen (e.g., 3D deco)1030which is an edit execution screen. As shown inFIG. 10C, the electronic device may display, on the third execution screen (e.g., 3D deco)1030of the application310, a first image1121for image processing (e.g., main image), the virtual plane types1131and1133, and an object (e.g., a timeline or time ruler)1150related to the reproduction time of the video including the first image1121. The first image1121may be an image corresponding to a first time point1151selected in the reproduction time.

In operation1007, the electronic device may set at least one virtual plane in a virtual 3D position, corresponding to at least a portion of at least one selected first image. If one of the virtual plane types1131and1133is selected on the third execution screen1130as shown inFIG. 11C, the electronic device may set at least one virtual plane in the virtual 3D position corresponding to at least part of the first image1121.

Referring toFIG. 11C, according to an embodiment, if the first type1131is selected from among the virtual plane types1131and1133, the electronic device may set one virtual plane on a 3D virtual position parallel with the first image1121based on the virtual plane type (e.g., one plane) corresponding to the first type1131. In response to selection of the first type1131, the electronic device may identify the display position of the first image1121projected to the set virtual plane on a partial display area of the first image1121and display a rectangularFIG. 1101corresponding to the virtual plane to facilitate the user's identification in the identified display position of the first image1121. For example, the rectangularFIG. 1101may be displayed to be distinguished from the first image1121using a difference in color or brightness. The identified display position of the first image1121may be a position corresponding to the coordinate value (e.g., the 2D coordinate value (x, y)) of the first image1121displayed on the display161.

Referring back toFIG. 11C, according to an embodiment, if the second type1133is selected from among the virtual plane types1131and1133, the electronic device may set a plurality of virtual planes having a designated cube shape on a 3D virtual position parallel corresponding to the first image1121based on the virtual plane type (e.g., cube plane) corresponding to the second type1133. In response to selection of the second type1133, the electronic device may identify the display position of the first image1121projected to the set virtual planes on the display area of the first image1121and display a cube figure corresponding to the virtual planes to facilitate the user's identification in the identified display position of the first image1121. For example, the cube figure may be displayed to be distinguished from the first image1121using a difference in color or brightness. The identified display position of the first image1121may be a position corresponding to the coordinate value (e.g., the 2D coordinate value (x, y)) of the first image1121displayed on the display161.

In operation1009, the electronic device may set a 3D virtual space including the at least one set virtual plane (e.g., one plane or a plurality of virtual planes having a cube shape) and the first image1121. For example, the 3D virtual space may be a space set by vertically extending from the 2D position (x, y) where the set virtual plane, i.e., the rectangularFIG. 1101, is displayed to the 3D virtual position. As another example, the 3D virtual space may be a space in a cube formed by a plurality of set virtual planes.

In operation1011, the electronic device may display a first control object1103for controlling the virtual plane in the identified 2D position based on the virtual plane type1131or1133selected from the third execution screen1130as shown inFIGS. 11D and 11E. The first control object1103may be displayed to correspond to or match the rectangularFIG. 1101and be displayed overlaid on a partial area of the first image1121. The electronic device may display, on the third execution screen1130, a user interface (screen, popup window, or layer)1140including menus for setting a 3D visual effect1107of the 3D object1105or configuring the 3D object1105to be inserted to the set virtual space.

In operation1013, as shown inFIGS. 11D and 11E, the electronic device may insert the 3D object1105to the set virtual space based on the virtual plane type1131or1133selected from the third execution screen1130and set a 3D visual effect1107to the 3D object1105. Referring toFIGS. 11D and 11E, the electronic device may insert the configured or input 3D object1105into the set virtual space using the menus included in the user interface1140and set a 3D visual effect (e.g., a shadow corresponding to the 3D object)1107.

In operation1015, the electronic device may generate at least one 2D second image by synthesizing the 3D object1105, set with the 3D visual effect1107, to the first image1121. The electronic device may apply the generated 2D second image to the image data, map it with time information about the reproduction time (Tcut0)1151of the first image1121, and store the generated second image. The electronic device may map or replace the first image1121selected from the image data, with the at least one second image and store it. As shown inFIGS. 11D and 11E, the electronic device may display the image (e.g., a small image, an icon, an emoticon, a button, or a graphic element) corresponding to the second image together, in the area displaying the reproduction time (Tcut0)1151.

FIGS. 12A, 12B, 12C, 12D and 12Eare diagrams of an example of a screen for image processing in an electronic device according to an embodiment.

Referring toFIGS. 12A to 12C, according to an embodiment, the electronic device (e.g., the first electronic device101ofFIGS. 1 and 2) may further selected an additional image (e.g., at least one of the thumbnail image1123, the intro image1125, or the outro image1127) from among the specific images (e.g., the main image1121of the multimedia content, thumbnail image1123, intro image1125, and outro image1127) displayed on the second execution screen1120of the application310as shown inFIG. 11B. As described in connection withFIGS. 11C to 11E, the electronic device may generate a second image corresponding to the additionally selected image (e.g., at least one of the thumbnail image1123, intro image1125, or outro image1127) by applying the same edit procedure as the first image1121(e.g., the main image).

Referring toFIG. 12A, the electronic device may display the thumbnail image1123(e.g., the thumbnail image1123ofFIG. 11B) on a fourth execution screen (e.g., a thumbnail execution screen)1210of the application310by execution of the thumbnail module and display some images of the image data, as thumbnail candidate images1211to be applied to the thumbnail image1123, on the fourth execution screen1210. The electronic device may apply an image selected from among the thumbnail candidate images1211to the thumbnail image1123and, in response to an edit request for the thumbnail image1123, display the third execution screen (e.g., the third execution screen1130ofFIGS. 11C to 11E), which is the edit execution screen, and display the thumbnail image1123on the third execution screen. The electronic device may perform processing on the thumbnail image1123in the same method as the edit operations described in connection withFIGS. 10 and 11C to 11E, generating a second image corresponding to the thumbnail image1123.

Referring toFIG. 12B, the electronic device may display the intro image1125(e.g., the intro image1125ofFIG. 11B) on a fifth execution screen (e.g., the intro execution screen)1220of the application310by execution of the intro module and display, on the fifth execution screen1220, at least one intro candidate image1221(e.g., at least one of part of the image data, downloaded images, captured images, or created images) to be applied to the intro image1125. The electronic device may apply an image selected from among the at least one intro candidate image1221to the intro image1125and, in response to an edit request for the intro image1125, display the third execution screen (e.g., the third execution screen1130ofFIGS. 11C to 11E), which is the edit execution screen, and display the intro image1125on the third execution screen. The electronic device may perform processing on the intro image1125in the same method as the edit operations described in connection withFIGS. 10 and 11C to 11E, generating a second image corresponding to the intro image1125.

Referring toFIG. 12C, the electronic device may display the outro image1127(e.g., the outro image1127ofFIG. 11B) on a sixth execution screen (e.g., the outro execution screen)1230of the application310by execution of the outro module and display, on the sixth execution screen1230, at least one outro candidate image1231(e.g., part of the image data) to be applied to the outro image1127. The electronic device may apply an image selected from among the at least one outro candidate image1231to the outro image1127and, in response to an edit request for the outro image1127, display the third execution screen (e.g., the third execution screen1130ofFIGS. 11C to 11E), which is the edit execution screen, and display the outro image1127on the third execution screen. The electronic device may perform processing on the outro image1127in the same method as the edit operations described in connection withFIGS. 10 and 11C to 11E, generating a second image corresponding to the outro image1127.

Referring toFIG. 12D, according to an embodiment, the electronic device (e.g., the first electronic device101ofFIGS. 1 and 2) may select at least one first image1241for processing from the image data and display a seventh execution screen1240of the application310which is an execution screen for applying music or input voice (or sound)1243to the image. The electronic device may display the at least one selected first image1241on the third execution screen (e.g., the third execution screen1130ofFIGS. 11C to 11E) which is the edit execution screen of the application310. The at least one selected first image1241may be an image corresponding to the image1121displayed on the third execution screen described in connection withFIG. 11C. According to an embodiment, the electronic device may load at least one generated second image on the third execution screen corresponding to the at least one selected first image1241and apply the music or input voice (or sound)1243to the at least one second image.

Referring toFIG. 12E, according to an embodiment, the electronic device (e.g., the first electronic device101ofFIGS. 1 and 2) may display an eighth execution screen1250for generating a caption in the image data. The electronic device may display, on the eighth execution screen1250, the selected image1251, menus1253for inputting text, and a menu for editing text (at least one of font, size, thickness, letter color, shading, superscript, subscript, tilt, rotate, style, or other edit functions) for generating a caption in the image data.

FIGS. 13A and 13Bare diagrams of an example of a screen for image processing in an electronic device according to an embodiment.

Referring toFIGS. 13A and 13B, according to an embodiment, the electronic device (e.g., the first electronic device101ofFIGS. 1 and 2) may display a first image1301for processing and set an additional virtual plane1303(hereinafter, referred to as a second virtual plane) in the same operations as the operations for setting a virtual plane (hereinafter, referred to as a first virtual plane) described in connection withFIG. 11C. The electronic device may further set an additional virtual plane according to, e.g., a predetermined number of virtual planes or additional selection.

Referring back toFIG. 13A, the electronic device may set a 3D virtual space by back-projecting the second virtual plane1303, e.g., in a vertical direction. According to an embodiment, the electronic device may be configured to interact with a virtual space set based on the first virtual plane (hereinafter, referred to as a first virtual space) and a virtual space set based on the second virtual plane1303(hereinafter, referred to as a second virtual space). According to another embodiment, the electronic device may form a new virtual space by combining the first virtual space and the second virtual space. If a 3D object (e.g., text or image)1305ais input after the second virtual plane1303is set, the electronic device may insert the input 3D object1305aonto the second virtual space dependently on the second virtual plane1303based on the first type selected from among the types of the 3D object, for example. If the 3D object1305apartially departs from the 2D display area displaying the first image1301as shown in (c) ofFIG. 13A, the electronic device may adjust the 3D virtual position where the 3D object1305ais to be inserted to be positioned in the virtual space where the first virtual space and the second virtual space overlap each other. The 3D object1305amay be moved dependently on the first virtual plane or the second virtual plane inside the virtual space where the first virtual space and the second virtual space overlap each other.

According to an embodiment, if the second virtual plane1303is additionally set, and the 3D object1305ais in a state of having been dependently inserted into the prior-set first virtual plane, the electronic device may move the 3D object1305ato the second virtual plane1303set so that the 3D object1305ais dependently inserted to the second virtual plane1303in response to a move request. In this case, when the moved position of the 3D object1305ais a position in the boundary line between the first virtual plane and the second virtual plane1303, the electronic device may rotate the 3D object1305ato be parallel to the second virtual plane1303and dependently insert the rotated 3D object1305aonto the second virtual plane.

According to another embodiment, the electronic device may identify the moved position of the 3D object1305aas the 3D object1303ainserted onto the second virtual plane1303is moved to another virtual plane (e.g., the first virtual plane) as shown inFIG. 13A. When the moved position of the 3D object1305ais a position in the boundary line between the second virtual plane1303and the other virtual plane (e.g., the first virtual plane), the electronic device may rotate the 3D object1305ato be parallel to the other virtual plane (e.g., the first virtual plane) and dependently insert the rotated 3D object1305ato the other virtual plane (e.g., the first virtual plane).

Referring back toFIG. 13B, according to an embodiment, if the second virtual plane1303is set and then a 3D object (e.g., text or image)1305bis input, the electronic device may insert the input 3D object1305bin the 3D virtual position on the second virtual space so that the 3D object1305binteracts with the second virtual plane1303, based on the second type selected from among the types of the 3D object1305b. The electronic device may adjust the 3D virtual position so that the 3D object1305bis inserted into the inside of the virtual space where the second virtual space and the first virtual space corresponding to the prior-set first virtual plane overlap each other. The electronic device may set a 3D visual effect1307to the 3D object1305b. The 3D object1305band the 3D visual effect may be moved to interact with the first virtual plane and the second virtual plane, inside the virtual space where the first virtual space and the second virtual space overlap each other.

FIG. 14is a diagram of an example of a screen for image processing in an electronic device according to an embodiment.

Referring toFIG. 14, according to an embodiment, the electronic device (e.g., the first electronic device101ofFIGS. 1 and 2) may select a reproduction time period1411for processing in the reproduction time for image data and display a first image1401to be reproduced at a first time point Tcut0of the reproduction time period1411on the edit execution screen (e.g., the third execution screen1130) of the application310for processing. The electronic device may insert the 3D object1407onto the virtual space corresponding to the virtual plane1403described in connection withFIGS. 3A to 3D and 11Eand set a 3D visual effect1409to the inserted 3D object1407. The electronic device may generate a 2D second image to be mapped to or replace the first image1401for reproduction at the first time point Tcut0by synthesizing the at least one first image1401and the 3D object inserted into the 3D virtual space as described in connection withFIG. 11E. The electronic device may generate edit images respectively corresponding to the plurality of first images by equally inserting the same 3D object1407, inserted at the first time point Tcut0, to the plurality of first images corresponding to the reproduction time period1411.

FIGS. 15A and 15Bare diagrams of an example of a screen for image processing in an electronic device according to an embodiment.

Referring toFIGS. 15A and 15B, according to an embodiment, the electronic device (e.g., the first electronic device101ofFIGS. 1 and 2) may display a first image1501selected from image data, on an edit execution screen (e.g., the third execution screen1130ofFIG. 11C) of an application310for processing and insert at least one 3D object (e.g., a dynamic image or animation image), which is moved according to the reproduction time, on the virtual space set based on cube-shaped virtual planes (hereinafter, referred to as a cube)1503as described in connection withFIGS. 4A to 4D. The electronic device may insert at least one same 3D object, inserted at the first time point Tcut0, to the plurality of first images of the image data corresponding to the selected reproduction time period (e.g., the first time point Tcut0to the second time point Tcut1). The at least one 3D object may present an effect of dynamically moving inside the cube1503during the reproduction time period (e.g., the first time point Tcut0to the second time point Tcut1) upon reproducing the image data.

Referring toFIGS. 15A and 15B, the electronic device may set the 3D object1505ato move from the 3D first virtual position1511inserted at the first time point Tcut0to the virtual plane1503bset on the left side of the virtual plane1503aover time. At this time, the electronic device may set the 3D object1505ato present an effect of being blocked by, and bouncing off, the virtual plane1503brather than further moving in the direction. For example, if the 3D object1505ais positioned on the boundary line of the left virtual plane1503b, the electronic device may set the 3D object1505ato move from the direction to the virtual plane1503bto the direction to the inside of the cube1503. As shown inFIG. 15B, the electronic device may generate a second image by synthesizing the 3D object1505a, which is changed in position over time, to each of the plurality of images corresponding to the reproduction time period (e.g., the first time point Tcut0to the second time point Tcut1) so that the 3D object1505ais moved from the first virtual position1511toward the inside the cube1503to be located in the 3D second virtual position1513at the second time point Tcut1.

According to another embodiment, when setting a 3D visual effect by light, e.g., on the cube-shaped virtual planes1503as shown inFIGS. 15A and 15B, the electronic device (e.g., the first electronic device101ofFIGS. 1 and 2) may set the inside of the set virtual space to be bright while setting the outside of the virtual space to be dark by light, based on the virtual planes1503. When inserting the 3D object to each of the plurality of images corresponding to the selected reproduction time period, the electronic device may set a 3D visual effect to each inserted 3D object so that the light inside the virtual space gradually brightens or gradually darkens over time when the plurality of images are reproduced.

According to various embodiments, the electronic device (e.g., the first electronic device101ofFIGS. 1 and 2) may set attributes (e.g., the attribute of a virtual camera) for image processing, set software-wise by the processor (e.g., the processor120ofFIGS. 1 and 2), to correspond to the physical attributes of a capturing device (e.g., the camera module180ofFIG. 1). The electronic device may set virtual planes, similar to surfaces or wall surfaces of objects, on an actual space.

According to an embodiment, a method for image processing in an electronic device may include setting at least one virtual plane in a 3D virtual position related to at least a portion of at least one 2D first image displayed on a display of the electronic device, setting a 3D virtual space including the at least one virtual plane and the at least one first image, inserting a 3D object into the 3D virtual space, and generating at least one 2D second image by synthesizing the 3D object inserted into the 3D virtual space and the at least one first image.

According to an embodiment, the method may further include controlling the display to display the at least one second image and mapping the at least one second image with reproduction time information related to the at least one first image and storing in a memory.

According to an embodiment, the method may further include identifying a 2D position on the display to which the at least one virtual plane is projected and controlling the display to display a first control object for controlling the at least one virtual plane in the 2D position. The first control object may be displayed overlaid on some of the at least one first image displayed in the 2D position and may adjust at least one of a size, movement, rotation, or depth of the at least one virtual plane.

According to an embodiment, the method may further include, in response to an edit request, executing an application for image processing and controlling the display to display an execution screen of the application including the at least one first image and menus for processing the at least one first image.

According to an embodiment, inserting the 3D object into the 3D virtual space may include setting a first virtual position for inserting the 3D object in the virtual space based on a first type selected from among types for inserting the 3D object, the first virtual position being a 3D virtual position in which at least a portion of the 3D object is attached to the at least one virtual plane, inserting the 3D object in the first virtual position, controlling the display to display a second control object for controlling the 3D object, the second control object being displayed in a 2D position on an area of the display to which the 3D object is projected, and setting a 3D visual effect by controlling the 3D object using the second control object. The 3D object added in the first virtual position may be configured to be capable of at least one of rotation about one axis, movement or resizing with respect to two axes.

According to an embodiment, inserting the 3D object into the 3D virtual space may include setting a second virtual position for inserting the 3D object in the virtual space based on a second type selected from among types for inserting the 3D object, the second virtual position being a 3D virtual position in which the 3D object does not overlap the at least one virtual plane, inserting the 3D object in the second virtual position, controlling the display to display a third control object for controlling the 3D object, the third control object being displayed in a 2D position on an area of the display to which the 3D object is projected, and setting a 3D visual effect by controlling the 3D object using the third control object.

According to an embodiment, the 3D object added in the second virtual position may be configured to be capable of at least one of rotation about three axes, movement or resizing with respect to the three axes.

According to an embodiment, inserting the 3D object into the 3D virtual space may include, when the at least one virtual plane includes a plurality of virtual planes, setting a 3D visual effect to the 3D object so that the 3D object dynamically moves within the virtual space without departing from the plurality of virtual planes in the virtual space set by the plurality of virtual planes.

According to an embodiment, inserting the 3D object into the 3D virtual space may include identifying a moved position of the 3D object, in response to a request for moving the 3D object from a first virtual plane among the at least one virtual plane to a second virtual plane, rotating the 3D object to allow the moved position of the 3D object to be parallel with the second virtual plane when the moved position of the 3D object is a position in a boundary line of the second virtual plane, and inserting the rotated 3D object onto the second virtual plane.

According to an embodiment, the method may further include obtaining a plurality of images included in a reproduction time period from a first time to a second time for reproducing image data including the at least one first image, generating 2D second images respectively corresponding to the plurality of images by equally applying the 3D object, inserted to the virtual space, to each of the plurality of images, and mapping or replacing a plurality of first images included in the reproduction time period with the 2D second images, respectively, and storing them in the memory.

The storage medium (e.g., the internal memory136or external memory138) readable by a machine (e.g., the electronic device101) may include hard disks, floppy disks, magnetic media (e.g., magnetic tapes), optical media (e.g., compact disc read only memories (CD-ROMs), digital versatile discs (DVDs), or magneto-optical media (e.g., floptical disks)), or hardware devices (e.g., read only memories (ROMs), random access memories (RAMs), or flash memories). The above-described hardware devices may be configured to operate as one or more software modules to carry out various embodiments, and vice versa.

According to an embodiment, a non-transitory storage medium may store instructions that, when executed by a processor of an electronic device, enable the electronic device to perform setting at least one virtual plane in a 3D virtual position related to at least a portion of at least one 2D first image displayed on a display of the electronic device, setting a 3D virtual space including the at least one virtual plane and the at least one first image, inserting a 3D object into the 3D virtual space, and generating at least one 2D second image by synthesizing the 3D object inserted into the 3D virtual space and the at least one first image.

The embodiments disclosed herein are proposed for description and understanding of the disclosed technology and does not limit the scope of the disclosure. Accordingly, the scope of the disclosure should be interpreted as including all changes or various embodiments based on the technical spirit of the disclosure.