Patent Description:
In recent years, the role of a display has become important in portable electronic devices. In particular, portable electronic devices having a touch-sensitive display such as a touch screen are capable of receiving various kinds of user inputs (e.g., a touch input, a stylus pen input, a hovering input, and the like) through the display and performing particular functions corresponding to the received user inputs.

A portable electronic device may display a user interface (e.g., an augmented reality (AR) interface) through a display and, based on the user interface, generate or store an object (e.g., content). That is, the portable electronic device may identify input data values in response to a user input and then generate or store a certain object based on the identified data values.

Patent document <CIT>, discloses a intelligent mobile device grafting the augmented reality on the mesh up technology.

Patent document <CIT>, discloses a method for providing an augmented reality are capable of providing environment information data in a direction viewed by a user from a current position.

Patent document <CIT>, discloses a recognition method of the special symbol sign in an individual augmented reality applications environment. Patent document <CIT>, discloses a "Text Rectifier" providing various techniques for processing selected regions of an image containing text or characters by treating those images as matrices of low-rank textures and using a rank minimization technique that recovers and removes image deformations (e.g., affine and projective transforms as well as general classes of nonlinear transforms) while rectifying the text or characters in the image region.

In order to generate such an object based on the AR interface, the electronic device may capture an image through a camera thereof and receive the captured image and position and orientation data associated with the captured image. Then, the electronic device may implement the AR interface based on the captured image, the position data, and the orientation data.

When generating a certain object (e.g., content) based on the AR interface, the electronic device needs information about the object to be generated (e.g., size, length, slope, height, area, and/or coordinate data such as coordinate values with respect to the X, Y and Z axes). In general, such object information may be data values entered by the user. For example, the electronic device may identify numerical values entered by the user and generate a certain object corresponding to the identified numerical values. In particular, for the generation of a three-dimensional object, the user further inputs numerical values for rotation information and depth information of the object. Unfortunately, this may cause operations of generating the object to be complicated.

In accordance with another aspect of the present disclosure, an electronic device is provided as defined by the appended claims.

An aspect of the present disclosure provides a touch-sensitive environment that allows a user to easily input information about a three-dimensional (3D) object into an electronic device when the 3D object is generated in a user interface based on an AR interface and displaying a two-dimensional plane. This environment not only allows the user to easily input 3D related information (e.g., rotation information and depth information) into the electronic device through a touch input, but also allows the electronic device to generate the 3D object based on the 3D related information entered by the user.

Various embodiments of the present disclosure are described below in detail with reference to the accompanying drawings.

<FIG> is a block diagram of an electronic device <NUM> in a network environment <NUM> according to an embodiment.

The electronic device <NUM> may include a processor <NUM>, memory <NUM>, an input device <NUM>, a sound output device <NUM>, a display device <NUM>, an audio module <NUM>, a sensor module <NUM>, an interface <NUM>, a connection terminal <NUM>, a haptic module <NUM>, a camera module <NUM>, a power management module <NUM>, a battery <NUM>, a communication module <NUM>, a subscriber identification module (SIM) <NUM>, or an antenna module <NUM>. At least one of the components may be omitted from the electronic device <NUM>, or one or more other components may be added in the electronic device <NUM>. Some of the components may be implemented as single integrated circuitry.

The processor <NUM> may include a main processor <NUM> (e.g., a central processing unit (CPU) or an application processor (AP)), and a coprocessor <NUM> (e.g., a graphics processing unit (GPU), an image signal processor (ISP), a sensor hub processor, or a communication processor (CP)) that is operable independently from, or in conjunction with, the main processor <NUM>. Additionally or alternatively, the coprocessor <NUM> may be adapted to consume less power than the main processor <NUM>, or to be specific to a specified function. The coprocessor <NUM> may be implemented as separate from, or as part of the main processor <NUM>.

The coprocessor <NUM> may control at least some of functions or states related to at least one component (e.g., the display device <NUM>, the sensor module <NUM>, or the communication module <NUM>) among the components of the electronic device <NUM>, instead of the main processor <NUM> while the main processor <NUM> is in an inactive (e.g., sleep) state, or together with the main processor <NUM> while the main processor <NUM> is in an active state (e.g., executing an application). According to an embodiment, the coprocessor <NUM> (e.g., an ISP or a CP) may be implemented as part of another component (e.g., the camera module <NUM> or the communication module <NUM>) functionally related to the auxiliary processor <NUM>.

The memory <NUM> (e.g., dynamic random access memory (DRAM), static RAM (SRAM) or synchronous DRAM (SDRAM)) may include the volatile memory <NUM> or the non-volatile memory <NUM>.

The program <NUM> may be stored in the memory <NUM> as software, and may include, for example, an operating system (OS) <NUM>, middleware <NUM>, or an application <NUM> (e.g., an application program).

According to an embodiment, the processor <NUM> may sense an input of a stylus pen on the display device <NUM> (e.g., a touch screen). The stylus pen having a separate resonance circuit therein may interact with an electromagnetic induction panel (e.g., a digitizer) contained in the display device <NUM>. The stylus pen may be implemented using a technique of electro-magnetic resonance (EMR), active electrical stylus (AES), or electric coupled resonance (ECR).

The speaker may be used for general purposes, such as playing multimedia or playing a record, and the receiver may be used for incoming calls. According to an embodiment, the receiver may be implemented as separate from, or as part of, the speaker.

According to an embodiment, the audio module <NUM> may obtain the sound via the input device <NUM>, or output the sound via the sound output device <NUM> or a headphone of an external electronic device (e.g., an electronic device <NUM>) directly (e.g., wired) or wirelessly coupled with the electronic device <NUM>.

The interface <NUM> may support one or more specified protocols to be used for the electronic device <NUM> to be coupled with the external electronic device (e.g., the electronic device <NUM>) directly (e.g., wired) or wirelessly.

The connection terminal <NUM> may include a connector via which the electronic device <NUM> may be physically connected with the external electronic device (e.g., the electronic device <NUM>). According to an embodiment, the connection terminal <NUM> may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

The haptic module <NUM> may convert an electrical signal into a mechanical stimulus (e.g., a vibration or a movement) or an electrical stimulus which may be recognized by a user via the user's tactile sensation or kinesthetic sensation.

According to an embodiment, the processor <NUM> may capture an image by using the camera module <NUM> and implement an AR interface based on the captured image. In addition, the processor <NUM> may display the implemented AR interface through the display device <NUM>.

The communication module <NUM> may support establishing a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device <NUM> and the electronic device <NUM>, the electronic device <NUM>, or the server <NUM> and performing communication via the established communication channel. The communication module <NUM> may include one or more communication processors that are operable independently from the processor <NUM> (e.g., the AP) and supports a direct (e.g., wired) communication or a wireless communication. According to an embodiment, the communication module <NUM> may include a radio communication module <NUM> (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module <NUM> (e.g., a local area network (LAN) communication module or a power line communication (PLC) module). A corresponding one of these communication modules may communicate with the external electronic device via the first network <NUM> (e.g., a short-range communication network, such as BluetoothTM, wireless-fidelity (Wi-Fi) direct, or a standard of the Infrared Data Association (IrDA)) or the second network <NUM> (e.g., a long-range communication network, such as a cellular network, the Internet, or a computer network (e.g., LAN or wide area network (WAN)). These various types of communication modules may be implemented as a single component (e.g., a single integrated circuit or chip), or may be implemented as multi components (e.g., multi chips) separate from each other. The radio communication module <NUM> may identify and authenticate the electronic device <NUM> in a communication network, such as the first network <NUM> or the second network <NUM>, using subscriber information (e.g., international mobile subscriber identity (IMSI)) stored in the SIM <NUM>.

According to an embodiment, the antenna module <NUM> may include one or more antennas, and, therefrom, at least one antenna appropriate for a communication scheme used in the communication network, such as the first network <NUM> or the second network <NUM>, may be selected, for example, by the communication module <NUM> (e.g., the radio communication module <NUM>).

At least some of the above-described components may be coupled mutually and communicate signals (e.g., commands or data) therebetween via an inter-peripheral communication scheme (e.g., a bus, a general purpose input and output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI)).

All or some operations to be executed at the electronic device <NUM> may be executed at one or more of the external electronic devices <NUM>, <NUM>, or <NUM>.

<FIG> is a flowchart of a method for generating an object based on an augmented reality interface according to an example not explicitly disclosing all the features of the claims.

At step <NUM>, a processor <NUM> of an electronic device <NUM> captures an image by using a camera module <NUM>. For example, the processor <NUM> may activate the camera <NUM> and capture a still image through the activated camera <NUM>.

At step <NUM>, the processor <NUM> receives the captured image and position and orientation data associated with the captured image. The processor <NUM> receives the position data associated with the captured image by using a position sensor (e.g., the sensor module <NUM> of <FIG>) and also receives the orientation data associated with the captured image by using a motion sensor (e.g., the sensor module <NUM> of <FIG>). The position data includes data on coordinates where the electronic device <NUM> is located, and the orientation data includes data on a direction that the camera <NUM> of the electronic device <NUM> is facing. According to an embodiment, the electronic device <NUM> may implement an AR interface, based on the captured image and the position and orientation data associated with the captured image.

At step <NUM>, the processor <NUM> receives a user input (e.g., a stylus pen input) for adding an object to the captured image. For example, the processor <NUM> may receive a user input through an input device <NUM>, based on the implemented AR interface. This user input includes a touch input for generating an object (e.g., content). According to an embodiment, this object includes text, and may further include a symbol, a geometric form, or an image.

At step <NUM>, the processor <NUM> transmits the object generated in response to the user input, the captured image, the position data, and the orientation data to a server <NUM>. In addition, the processor <NUM> may store the generated object in a memory <NUM>. In an embodiment, the processor <NUM> may transmit the generated object, the image captured at step <NUM>, and the position/orientation data received at step <NUM> to the server <NUM> at least in part.

As described hereinbefore, the electronic device <NUM> captures a certain image through the camera <NUM> and receives position and orientation data associated with the captured image. Then, the electronic device <NUM> may implement an AR interface, based on the captured image, the received position data, and the received orientation data. In addition, the electronic device <NUM> may receive a user input through the AR interface and then generate an object in response to the received user input. In addition, the electronic device <NUM> may store the generated object in the memory <NUM> and transmit the generated object, the captured image, the position data, and the orientation data to the server.

<FIG> and <FIG> are flowcharts <NUM> and <NUM> of methods for generating a 3D object according to examples not explicitly disclosing all the features of the claims.

Referring to <FIG>, flowchart <NUM> regards a method for generating a 3D object. Flowchart <NUM> corresponds to the above-described steps <NUM> and <NUM> in <FIG>.

At step <NUM>, a processor <NUM> of an electronic device <NUM> receives a user input associated with an object (or content). The processor <NUM> displays a user interface to be used for generating the object through a touch screen display (e.g., the display device <NUM> of <FIG>) and receives a user input through the user interface. The user input includes a touch input by a finger or a touch input by a pen (e.g., a stylus pen). According to an embodiment, the user interface may be an AR interface. For example, the processor <NUM> may activate the camera module <NUM>, capture a certain image through the activated camera, display the captured image through the display, and implement the AR interface based on the captured image.

At step <NUM>, the processor <NUM> acquires 3D-related information for generating the object from the received user input. The processor <NUM> acquires the 3D-related information (e.g., data on coordinates, angle, slope, width, length, height, and/or area) about the object (e.g., text, a geometric form, etc.) to be generated in accordance with the received user input.

At step <NUM>, the processor <NUM> transmits the object, generated based on the acquired 3D-related information, to the server <NUM>. For example, the processor <NUM> generates the object based on the acquired 3D-related information, displays the generated object on the display <NUM>, and stores the generated object in the memory <NUM>. In addition, the processor <NUM> transmits the generated object to an external electronic device (e.g., the server <NUM>).

As described above, the electronic device <NUM> receives a user input through the display <NUM> and then acquires 3D-related information for generating an object from the received user input. In addition, the electronic device <NUM> generates the object based on the acquired 3D-related information, displays the generated object on the display <NUM>, stores the generated object in the memory <NUM>, and transmits the generated object to the server <NUM>.

Referring to <FIG>, flowchart <NUM> regards a method for generating a 3D object according to an example not explicitly disclosing all the features of the claims. Flowchart <NUM> corresponds to the above-described steps <NUM> and <NUM> in <FIG>.

At step <NUM>, a processor <NUM> of an electronic device <NUM> may receive a first user input associated with an object. For example, the processor <NUM> may receive a user input of writing a text object "HELLO" through the touch screen display.

At step <NUM>, the processor <NUM> may identify a geometric form having 3D-related information. For example, the processor <NUM> may receive an input of constructing a certain geometric form on the display and identify the geometric form from the received input. The geometric form may have 3D-related information such as length, height, and slope, so that the processor <NUM> may also identify the 3D-related information from the identified geometric form.

Although it is described that step <NUM> is performed after step <NUM>, step <NUM> may be alternatively performed before step <NUM>. The geometric form having the 3D-related information may be in connection with the first user input received at step <NUM>. In one example described above, the input of constructing the geometric form may be received after the first user input associated with the object is received. However, in another example, the processor <NUM> may identify the geometric form having the 3D-related information and then receive the first user input associated with the object. For example, the processor <NUM> may recognize the first user input in a certain region defined by the geometric form.

At step <NUM>, the processor <NUM> may generate the object in response to the first user input and in consideration of the identified geometric form. For example, considering the 3D-related information of the identified geometric form, the processor <NUM> may generate a certain object corresponding to the first user input. That is, the generated object may be a 3D object to which the 3D-related information is applied.

At step <NUM>, the processor <NUM> may receive a second user input for the generated object. For example, the second user input may be at least one input that directly touches the generated object. For example, the second user input may include a drag input.

At step <NUM>, the processor <NUM> may transmit the object to the server <NUM> in response to the received second user input. For example, when the second user input is a drag input for the object, the processor <NUM> may move the object in response to the drag input. At this time, the processor <NUM> may display real-time dragged states of the object through the display <NUM>. In addition, the processor <NUM> may store information (e.g., a position, coordinates, a slope, a width, a length, etc.) related to the object in the memory <NUM> and transmit the object to an external electronic device (e.g., the server <NUM>).

<FIG>, <FIG> are illustrations of generating a 3D object according to an example not explicitly disclosing all the features of the claims.

Referring to <FIG>, a processor <NUM> of an electronic device <NUM> may display a 3D-based user interface (e.g., the AR interface) through a display <NUM> such as a touch screen display. For example, the 3D-based user interface may include a 3D image, a photograph, a picture, or a real-time preview image being obtained through the camera module <NUM>. The processor <NUM> may receive a user input for generating a 3D object on the 3D-based user interface. For example, the processor <NUM> may receive a first user input of writing a text object <NUM> (e.g., "HELLO") and display the text object <NUM> on the display <NUM>. This user input may be a user's touch input via a finger or a stylus pen. According to an embodiment, the object generated in response to this user input is not limited to the text object <NUM>.

Referring to <FIG>, the processor <NUM> may receive a second user input of constructing a certain geometric form <NUM> (e.g., defining a 3D-based plane in the 3D-based user interface) which surrounds, at least in part, the text object <NUM>. Then, the processor <NUM> may identify the geometric form <NUM> from the received second user input, and display the identified geometric form <NUM> on the display <NUM>. This geometric form <NUM> may have 3D-related information such as data on coordinates, angle, slope, width, length, height, and/or area. Thus, the processor <NUM> may acquire the 3D-related information from the identified geometric form <NUM>.

Referring to <FIG>, the processor <NUM> may receive a third user input of constructing a complete geometric form <NUM> which completely surrounds the text object <NUM>. Then, the processor <NUM> may identify the complete geometric form <NUM> from the received third user input, display the identified geometric form <NUM> on the display <NUM>, and acquire the 3D-related information from the identified geometric form <NUM>. That is, based on the second and third user inputs, the complete geometric form <NUM> desired by the user is constructed to surround the text object <NUM>. As a result, the text object <NUM> such as "HELLO" is contained within the complete geometric form <NUM>.

Referring to <FIG>, the processor <NUM> may modify the shape of the text object <NUM>, based on the complete geometric form <NUM>. For example, in accordance with the acquired 3D-related information of the complete geometric form <NUM>, the processor <NUM> may change corresponding 3D-related information of the text object <NUM>.

Referring to <FIG>, a processor <NUM> of an electronic device <NUM> may display a 3D-based user interface (e.g., the AR interface) through a display <NUM> such as a touch screen display. In addition, the processor <NUM> may receive a first user input of constructing a certain geometric form <NUM> that defines, for example, a 3D-based plane in the 3D-based user interface. Then, the processor <NUM> may identify the geometric form <NUM> from the received first user input, and display the identified geometric form <NUM> on the display <NUM>. The first user input may be a user's touch input via a finger or a stylus pen. The geometric form <NUM> may have 3D-related information such as data on coordinates, angle, slope, width, length, height, and/or area. Thus, the processor <NUM> may acquire the 3D-related information from the identified geometric form <NUM>.

Referring to <FIG>, the processor <NUM> may receive a second user input within the geometric form <NUM> displayed in the 3D-based user interface. The second user input is for generating a 3D object. For example, the processor <NUM> may receive the second user input of writing a text object <NUM> "HELLO" and display the text object <NUM> within the geometric form <NUM> on the display <NUM>.

Referring to <FIG>, the processor <NUM> may modify the shape of the text object <NUM>, based on the geometric form <NUM>. For example, in accordance with the acquired 3D-related information of the geometric form <NUM>, the processor <NUM> may change corresponding 3D-related information of the text object <NUM>.

Referring to <FIG>, the processor <NUM> may receive a third user input <NUM> associated with the text object <NUM>. For example, the third user input may be a drag input for moving the text object <NUM> in the 3D-based user interface. Thus, the processor <NUM> may move the text object <NUM> in response to the third user input <NUM>. At this time, the geometric form <NUM> may disappear from the display <NUM>.

Referring to <FIG>, the processor <NUM> may display the moved text object <NUM> on the display <NUM> after the third user input <NUM> is released. As a result, the moved text object <NUM> may act as one of things contained in the 3D-based user interface.

<FIG> is a flowchart <NUM> of a method for generating a 3D object by using pressure information according to an example not explicitly disclosing all the features of the claims.

At step <NUM>, a processor <NUM> of an electronic device <NUM> receives a first user input associated with an object. The first user input is a touch input and includes pressure information associated with the touch input. The processor <NUM> displays a user interface (e.g., an AR interface) to be used for generating the object through a touch screen display and receives the first user input through the user interface.

At step <NUM>, the processor <NUM> acquires 3D-related information from the received first user input including the pressure information. The 3D-related information includes data on coordinates and may further include data on angle, slope, width, length, height, and/or area for generating the object such as text or a geometric form.

At step <NUM>, the processor <NUM> generates the object, based on the acquired 3D-related information. In addition, the processor <NUM> displays the generated object on the display <NUM> and may also store the generated object in a memory <NUM>.

At step <NUM>, the processor <NUM> may receive a second user input associated with the object. The second user input may be a touch input and include pressure information associated with the touch input. For example, the second user input may be an input for moving or rotating the generated object in the user interface or for modifying the shape of the generated object.

At step <NUM>, the processor <NUM> may process the object in response to the received second user input including the pressure information. For example, the processor <NUM> may rotate the object displayed on the display <NUM> in accordance with the second user input including the pressure information. In an embodiment, a touch position of the second user input may determine a rotation axis of the object, and the pressure information may determine a rotation angle of the object. That is, as a pressure input is increased, the rotation angle may be increased.

At step <NUM>, the processor <NUM> transmits a processing result for the object to the server <NUM>. For example, the processor <NUM> may store information about the rotated object in the memory <NUM> and transmit such information to an external electronic device (e.g., the server <NUM>).

<FIG>, <FIG>, and <FIG> are illustrations of generating a 3D object by using pressure information according to an embodiment.

Referring to <FIG>, a processor <NUM> of an electronic device <NUM> receives a user input associated with an object. This user input is a touch input and includes pressure information associated with the touch input. Therefore, upon receiving the user input, the processor <NUM> extracts the pressure information from the received user input.

For example, when it is assumed that the user input has the pressure information within <NUM> pressure units, the processor <NUM> may define in advance a pressure level according to the range of pressure units as shown in Table <NUM> below. Table <NUM> below shows an example where four pressure levels are defined. In this example, the processor <NUM> may detect a certain pressure unit of the received user input by extracting the pressure information and find a certain pressure level corresponding to the detected pressure unit by referring to Table <NUM> below.

After finding the pressure level associated with the user input, the processor <NUM> determines, based on the pressure level, Z-axis coordinate information (i.e., depth information) of an object to be generated. For example, the processor <NUM> may establish a virtual coordinate system composed of X, Y and Z-axes in a user interface (e.g., an AR interface) displayed on a display <NUM>. When the received user input for generating an object (e.g., a text object "HELLO") is a touch input having a certain pressure level, the processor <NUM> may determine an X-axis coordinate and a Y-axis coordinate from the position of the touch input and also determine a Z-axis coordinate from the pressure level. For example, in case of a pressure level of "<NUM>", the processor <NUM> may generate and display the object to have a Z-axis coordinate "a" as shown in <FIG>.

Referring to <FIG>, the user input for generating the text object "HELLO" may have different pressure levels for respective letters of "HELLO". For example, when a letter "H" is inputted with a pressure level of "<NUM>" and then a letter "E" is inputted with a pressure level of "<NUM>", the processor <NUM> may generate and display the letter "H" to have a Z-axis coordinate "a" and also generate and display the letter "E" to have a Z-axis coordinate "<NUM>".

Similarly, referring to <FIG>, when letters "H", "L" and "O" are inputted with a pressure level of "<NUM>" and also letters "E" and "L" are inputted with a pressure level of "<NUM>", the processor <NUM> may generate and display the letters "H", "L" and "O" to have a Z-axis coordinate "b" and also generate and display the letters "E" and "L" to have a Z-axis coordinate "<NUM>".

As described herein, the object generated in response to a user input having a certain pressure level has a depth determined according to the pressure level when displayed in the user interface.

<FIG> is an illustration of generating a 3D object by using pressure information according to an example not explicitly disclosing all the features of the claims.

Referring to <FIG>, a processor <NUM> of an electronic device <NUM> may receive a user input associated with an object. This user input may be a touch input and include pressure information associated with the touch input. Therefore, upon receiving the user input, the processor <NUM> may extract the pressure information from the received user input.

For example, when it is assumed that the received user input is an input for constructing a heart form, an input of drawing a left portion <NUM> of the heart form and an input of drawing a right portion <NUM> of the heart form may have different pressure levels. In this case, the processor <NUM> may calculate a difference in pressure level between both inputs and, based on the calculated difference, determine a rotation angle of the heart form. A related example is shown in Table <NUM> below.

For example, when the input for the left portion <NUM> may have a pressure level of "<NUM>", and when the input for the right portion <NUM> may have a pressure level of "<NUM>", a difference in pressure level becomes "<NUM>". Therefore, the processor <NUM> may determine the rotation angle as <NUM>° by referring to Table <NUM> above. Then, the processor <NUM> may rotate one portion of the heart form having the greater pressure level by the determined rotation angle. That is, the right portion <NUM> of the heart form having the greater pressure level is rotated by <NUM>°.

As described herein, the object generated in response to user inputs having different pressure levels may be partially rotated according to a difference in the pressure level when displayed in the user interface.

Referring to <FIG>, a processor <NUM> of an electronic device <NUM> may receive a user input for generating a text object "HELLO", generate the text object "HELLO" in response to the received user input, and display the generated text object on a display <NUM>. In this case, the depth of the text object may vary according to pressure information of the received user input.

In three examples of <FIG>, a first exemplary text object <NUM> may have a lower depth according to a pressure level of "<NUM>". On the other hand, a second exemplary text object <NUM> may have a middle depth according to a pressure level of "<NUM>", and a third exemplary text object <NUM> may have a higher depth according to a pressure level of "<NUM>".

<FIG>, <FIG>, <FIG>, and <FIG> are illustrations of rotating a 3D object in response to an additional user input according to an example not explicitly disclosing all the features of the claims.

Referring to <FIG>, a processor <NUM> of an electronic device <NUM> may establish a virtual coordinate system composed of X, Y and Z-axes in a user interface (e.g., an AR interface) displayed on a display <NUM>. For example, the X-axis may be set to correspond to the length A of the electronic device <NUM>, and the Y-axis may be set to correspond to the width B of the electronic device <NUM>. In addition, the Z-axis may be set to correspond to a vertical direction from the display <NUM>. According to an embodiment, the processor <NUM> displays an object on the display <NUM>. Then, the processor <NUM> may receive a user input for the displayed object and rotate the object at least in part in response to the received user input.

The processor <NUM> displays a text object <NUM> (e.g., "HELLO") on the display <NUM>. Then, the processor <NUM> may receive a multi-touch input including a first user input <NUM> and a second user input <NUM> with respect to the text object <NUM>. For example, the first user input <NUM> may be a finger touch input, and the second user input <NUM> may be a pen touch input.

In an embodiment, the processor <NUM> may determine a rotation axis of the object, based on the first user input <NUM>, and also determine a rotation direction of the object, based on the second user input <NUM>. For example, when the first user input <NUM> is detected from the left end (i.e., a letter "H") of the object "HELLO", and when the second user input <NUM> is detected from the right end (i.e., a letter "O") of the object, the processor <NUM> may determine the rotation axis of the object as the Y-axis <NUM> passing the left end of the object, based on the first user input <NUM>, and also determine the rotation direction of the object as the rightward direction on the rotation axis, based on the second user input <NUM>. Here, a rotation angle may be determined in proportion to time duration of the second user input <NUM> or a pressure level of the second user input <NUM>.

In an embodiment, when a first user input <NUM> is detected from the lower end of the text object <NUM> "HELLO", and when a second user input <NUM> is detected from the upper end of the object, the processor <NUM> may determine the rotation axis of the object as the X-axis <NUM> passing the lower end of the object, based on the first user input <NUM>, and also determine the rotation direction of the object as the downward direction on the rotation axis, based on the second user input <NUM>. Here, a rotation angle may be determined in proportion to time duration of the second user input <NUM> or a pressure level of the second user input <NUM>.

Referring to <FIG>, the processor <NUM> may receive a multi-touch input including a first user input <NUM> such as a finger touch input and a second user input <NUM> such as a pen drag input. Then, the processor <NUM> may determine a rotation axis of the object, based on the first user input <NUM>, and also determine a rotation direction of the object, based on the second user input <NUM>.

For example, when the first user input <NUM> is detected from the left end (i.e., a letter "H") of the object "HELLO", and when the second user input <NUM> is detected from the right end (i.e., a letter "O") of the object, the processor <NUM> may determine the rotation axis of the object as the Z-axis <NUM> near the left end of the object, based on the first user input <NUM>, and also determine the rotation direction <NUM> of the object according to the second user input <NUM> (i.e., a dragging direction).

Referring to <FIG>, the processor <NUM> may receive a multi-touch input including a first user input <NUM> such as a finger touch input and a second user input <NUM> such as a pen touch input with respect to a predetermined single region <NUM> in the object <NUM>. In this case, the processor <NUM> may linearly move the displayed object <NUM> in response to the received multi-touch input.

For example, when the first and second user inputs <NUM> and <NUM> are detected from the same region <NUM> (i.e., a middle letter "L") of the object <NUM> "HELLO", the processor <NUM> may linearly move the object <NUM> along the Z-axis, that is, adjust the depth of the object. Here, a moving distance or adjusted depth of the object may be determined in proportion to time duration of both the first and second user inputs <NUM> and <NUM> or a pressure level of at least one of the first and second user inputs <NUM> and <NUM>.

Referring to <FIG>, the processor <NUM> may receive a multi-touch input including a first user input <NUM> such as a finger touch input and a second user input <NUM> such as a pen touch input. Then, the processor <NUM> may determine a rotation axis of the object <NUM>, based on the first user input <NUM>, and also determine a rotation direction of the object, based on the second user input <NUM>.

For example, when the first user input <NUM> is detected from a certain inside point (e.g., a space between letters "E" and "L") of the object <NUM> "HELLO", and when the second user input <NUM> is detected from the right end (i.e., a letter "O") of the object, the processor <NUM> may determine the rotation axis of the object as the Y-axis <NUM> passing the inside point of the object, based on the first user input <NUM>, and also determine the rotation direction of the object as the rightward direction on the rotation axis, based on the second user input <NUM>. Here, a rotation angle may be determined in proportion to time duration of the second user input <NUM> or a pressure level of the second user input <NUM>.

<FIG> is a flowchart <NUM> of a method for generating a 3D object by using pen tilt information according to an example not explicitly disclosing all the features of the claims.

At step <NUM>, a processor <NUM> of an electronic device <NUM> may receive a user input associated with an object. The user input may be a touch input by a stylus pen.

At step <NUM>, the processor <NUM> may acquire tilt information of the pen. For example, the processor <NUM> may detect the tilt information of the pen, based on electromagnetic waves that change according to a tilt of the pen. For example, the pen may have a sensor for detecting the tilt information thereof and acquire the tilt information through the sensor. Therefore, the processor <NUM> may receive the tilt information of the pen from the pen.

At step <NUM>, the processor <NUM> may generate an object, based on both the received user input and the acquired tilt information of the pen. For example, the processor <NUM> may determine a direction and slope of the object, based on the tilt information of the pen. Then, the processor <NUM> may display, on a display, the object to which the determined direction and slope are applied.

<FIG>, <FIG>, and <FIG> are illustrations of generating a 3D object by using pen tilt information according to an example not explicitly disclosing all the features of the claims.

Referring to <FIG>, a processor <NUM> of an electronic device <NUM> may establish a virtual coordinate system composed of X, Y and Z-axes in a user interface (e.g., an AR interface) displayed on a display <NUM>. For example, the X-axis may be set to correspond to the length A of the electronic device <NUM>, and the Y-axis may be set to correspond to the width B of the electronic device <NUM>. In addition, the Z-axis may be set to correspond to a vertical direction from the display <NUM>.

According to an embodiment, the processor <NUM> may display an object <NUM> (e.g., a text object "HELLO") on the display <NUM>. Then, the processor <NUM> may receive a user input by a pen <NUM> for the displayed object <NUM>, acquire tilt information of the pen <NUM>, and modify the object on the basis of the acquired tilt information.

When the tilt information indicates that the pen <NUM> has a tilt of <NUM>° from the Z-axis (that is, the pen is vertical to the display <NUM>), the processor <NUM> may display the object <NUM> without any tilt.

Referring to <FIG>, when the tilt information indicates that the pen <NUM> has a tilt of α° from the X-axis <NUM>, the processor <NUM> may display a text object <NUM> "HELLO" to have a tilt of α° from the X-axis <NUM>.

Referring to <FIG>, when the tilt information indicates that the pen <NUM> has a tilt of β° from the Y-axis <NUM>, the processor <NUM> may display a text object <NUM> "HELLO" to have a tilt of β° from the Y-axis <NUM>.

According to various embodiments, an electronic device includes a housing; a touch screen display disposed within the housing and exposed through at least a portion of the housing; a camera; a wireless communication circuit disposed within the housing; a position sensor; a motion sensor; a processor disposed within the housing and operatively connected to the display, the camera, the wireless communication circuit, the position sensor, and the motion sensor; and a memory disposed within the housing and operatively connected to the processor. The memory stores instructions causing, upon execution, the processor to capture an image by using the camera, to receive the captured image from the camera, to receive position data associated with the captured image from the position sensor, to receive orientation data associated with the captured image from the motion sensor, to receive a user input for adding an object to the captured image through the display, and to transmit the object, the captured image, the position data, and the orientation data to an external server through the wireless communication circuit.

The processor may receive the user input using a stylus pen.

The object includes text, and may further include at least one of a symbol, a geometric form, or an image.

The processor may receive a first user input of constructing a geometric form on the captured image, and receive a second user input of writing text at least partially overlapped with the geometric form on the captured image.

The processor may change a direction of the text in accordance with the geometric form, and transmit information about the changed direction to the server.

The processor may receive a third user input of moving the text in the captured image, and display the moved text on the display without displaying the geometric form.

The processor may identify three-dimensional (3D)-related information corresponding to the geometric form, and change the direction of the text, based on the identified 3D-related information.

The 3D-related information may include data on at least one of coordinates, angle, slope, width, length, height, or area, which corresponds to the geometric form.

The processor may receive a multi-touch input, and rotate the object at least in part, based on the received multi-touch input.

The multi-touch input may include a first touch input for determining a rotation axis of the object, and a second touch input for determining a rotation direction of the object.

The processor may receive tilt information of the stylus pen from the stylus pen, and rotate the object at least in part, based on the received tilt information.

The electronic device further includes a pressure sensor, and the processor receives pressure information associated with the user input from the pressure sensor, and generates the object, based on the received pressure information.

The processor may generate the object in accordance with the received user input, and store the generated object in the memory.

The processor may receive an image, position data associated with the image, and orientation data associated with the image from a second external electronic device including a display and a camera via the network interface, and transmit the object to the second external electronic device via the network interface such that the object is displayed on the display of the second external electronic device, wherein the orientation data is applied to the displayed object.

The electronic device may be one of various types of electronic devices. However, the electronic devices are not limited to those described above.

It should be appreciated that the present disclosure is not intended to be limited by the various embodiments described above or the terms used therein but includes various changes, equivalents, or replacements. With regard to the description of the accompanying drawings, similar reference numerals may be used to refer to similar or related elements. It is to be understood that a singular form of a noun corresponding to an item may include one or more of the item, unless the relevant context clearly indicates otherwise. As used herein, phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" may include all possible combinations of the items enumerated together in a corresponding one of the phrases. As used herein, such terms as "1st", "2nd", "first", and "second" may be used to simply distinguish a corresponding component from another component, but is not intended to limit the components in another aspect (e.g., importance or order). It is to be understood that if an element (e.g., a first element) is referred to, with or without the term "operatively" or "communicatively", as "coupled with", "coupled to", "connected with", or "connected to" another element (e.g., a second element), it indicates that the element may be coupled with the other element directly (e.g., wired), wirelessly, or via a third element.

As used herein, the term "module" may include a unit implemented in hardware, software, or firmware, and may be interchangeably used with other terms, for example, "logic", "logic block", "part", and "circuitry". The term "module" may indicate a single integral component, or a minimum unit or part thereof, adapted to perform one or more functions. For example, according to an embodiment, the term "module" may be implemented in a form of an application-specific integrated circuit (ASIC).

For example, a processor (e.g., the processor <NUM>) of the machine may invoke at least one of the one or more instructions stored in the storage medium, and execute it, with or without using one or more other components under the control of the processor. Wherein, the term "non-transitory" simply indicates that the storage medium is a tangible device, and does not include a signal (e.g., an electromagnetic wave), but this term does not differentiate between where data is semi-permanently stored in the storage medium and where the data is temporarily stored in the storage medium.

A method according to an embodiment of the present disclosure may be included and provided in a computer program product. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or be distributed (e.g., downloaded or uploaded) online via an application store (e.g., Play StoreTM), or between two user devices (e.g., smart phones) directly.

According to an embodiment, each component (e.g., a module or a program) of the above-described components may include a single entity or multiple entities. One or more of the above-described components may be omitted, or one or more other components may be added. In such a case, the integrated component may still perform one or more functions of each of the plurality of components in the same or similar manner as they are performed by a corresponding one of the plurality of components before the integration. Operations performed by the module, the program, or another component may be carried out sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations may be executed in a different order or omitted, or one or more other operations may be added.

Claim 1:
An electronic device (<NUM>), comprising:
a housing;
a touch screen display (<NUM>) disposed within the housing and exposed through at least a portion of the housing;
a camera (<NUM>);
a wireless communication circuit (<NUM>) disposed within the housing;
a position sensor;
a motion sensor;
a pressure sensor;
a processor (<NUM>) disposed within the housing and operatively connected to the display, the camera, the wireless communication circuit, the position sensor, the motion sensor and the pressure sensor; and
a memory (<NUM>) disposed within the housing and operatively connected to the processor,
wherein the memory is configured to store instructions causing, upon execution, the processor to:
capture (<NUM>) an image by using the camera,
receive (<NUM>) the captured image from the camera,
receive (<NUM>) position data associated with the captured image from the position sensor,
receive (<NUM>) orientation data associated with the captured image from the motion sensor,
receive (<NUM>, <NUM>) a first user input of constructing a geometric form on the captured image,
receive (<NUM>, <NUM>, <NUM>) a second user input of writing text at least partially overlapped with the geometric form on the captured image through the display, wherein the text comprises a plurality of letters and wherein the second user input is a touch input and includes pressure information associated with the touch input,
change (<NUM>) a direction of the text in accordance with the geometric form,
extract (<NUM>) the pressure information from the received second user input and find pressure levels associated with the second user input,
based on the pressure levels, determine depth information of the object to be added to the image, wherein the depth information is determined for each of the plurality of letters of the text based on the pressure information received for each of the plurality of letters of the text,
generate and display each letter of the plurality of letters of the text based on the determined depth information for each of the plurality of letters,
and
transmit (<NUM>, <NUM>, <NUM>) the object, the captured image, the position data, and the orientation data to an external server through the wireless communication circuit.