System and method for video call

Provided are a system and method for a video call. The system for a video call includes one or more terminals configured to transmit images obtained by capturing video call users, and a video call control server configured to generate a combined image by combining the images transmitted by the one or more terminals, and transmit the combined image to the terminals. Here, the respective images are rotated according to information on orientations sensed by the terminals, and combined into the combined image.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a National Phase of International Application No. PCT/KR2013/000703, filed on Jan. 29, 2013, which claims priority to and the benefit of Republic of Korea Patent Application Nos. 10-2012-0011184 filed on Feb. 3, 2012, 10-2012-0135149 filed on Nov. 27, 2012, and 10-2013-0009383, filed on Jan. 28, 2013, the disclosures of all of which are incorporated herein by reference in their entirety.

BACKGROUND

The present disclosure relates to video call technology, and, more particularly, to a system and method, for a video call, that display respective users in the same direction as a reference orientation or a reference posture in a video screen.

2. Discussion of Related Art

Recent developments in information and communications technology have allowed a video call as well as a voice call through a portable terminal. In a video call service, users capture their own images using cameras attached to portable terminals and then transmit the images to counterpart portable terminals so that they can talk with each other while looking at their counterparts.

A video call service can provide a one-to-one video call between a caller and a called party and a group video call. Here, group video calls are mainly made through a control device called a multipoint control unit (MCU). The MCU is a device that receives voice and images transmitted by respective portable terminals, combines the plurality of voices and images into one, and then transmits the combined voice and image data to the respective portable terminals.

FIG. 1is a diagram showing an existing video call system employing an MCU.

Referring toFIG. 1, in an existing video call system, since an MCU20combines images a, b, c, and d transmitted by respective portable terminals11,12,13, and14as they are, and then delivers a combined image I, to the respective portable terminals11to14. The images a to d in the combined image I may have different display orientations according to the various orientations being used by users of the respective terminals11to14during the video call.

For example, when the portable terminal11and the portable terminal14are rotated by 90 degrees clockwise with respect to a reference orientation, and capture images, and the portable terminal12and the portable terminal13capture images in the reference orientation, the images a and d captured by the portable terminal11and the portable terminal14each are combined by the MCU20in their then-active orientation, i.e., rotated by 90 degrees counterclockwise. The images b and c, however, captured by the portable terminal12and the portable terminal13are each combined by the MCU20in their then-active orientation, namely, the reference orientation.

In this case, when users of the respective portable terminals11to14look at the combined image I, the images a and d in the combined image I are shown in the orientation rotated by 90 degrees counterclockwise, and thus the viewers see a combined image that looks strange and is hard on the eyes.

SUMMARY

Aspects of the present disclosure are directed to providing a terminal and a video call control server that cause each video call user to be displayed as if oriented in a reference direction in a captured image, even when the terminal is rotated, and a system and method for a video call using the terminal and the video call control server.

According to an exemplary embodiment, there is provided a system for a video call, including: one or more terminals configured to transmit images obtained by capturing video call users; and a video call control server configured to generate a combined image by combining the images transmitted by the one or more terminals, and to transmit the combined image to the terminals. Here, one or more of the respective images are rotated, prior to transmitting the combined image, based on orientation information sensed by the one or more terminals.

According to another exemplary embodiment, there is provided a method for a video call, including: transmitting, at one or more terminals, respective images obtained by capturing video call users; generating a combined image, at a video call control server, by combining the respective images; and transmitting the combined image from the video call control server to the one or more terminals. Here, the respective images are rotated, prior to the transmitting of the combined image, based on orientation information sensed by the one or more terminals.

According to another exemplary embodiment, there is provided a video call control server, including: a communicator configured to receive respective orientation information and original respective images from one or more terminals; an image rotator configured to rotate the original respective images from at least one of the one or more terminals from a captured orientation to a reference orientation based on the respective orientation information of the at least one of the one or more terminals to provide rotated respective images; and a combined image generator configured to generate a combined image by combining the rotated respective images and any of the original respective images already in the reference orientation. Here, the communicator transmits the combined image to the one or more terminals. Also, one or more of the communicator, the image rotator, and the combined image generator are implemented by a hardware processor.

According to yet another exemplary embodiment, there is provided a video call control server, including: a communicator configured to receive captured video images from one or more terminals; an image analyzer configured to detect screen modes of the respective terminals from the received captured video images; a combined image generator configured to generate a combined image by combining the received captured video images based on the detected screen modes of the one or more terminals; and a hardware processor implementing one or more of the communicator, the image analyzer, and the combined image generator. Here, the communicator is further configured to transmit the combined image to the one or more terminals.

In another exemplary embodiment, there is provided a terminal used in a video call system, including: a capturing unit configured to capture video images; an orientation sensor configured to sense a terminal rotation state with respect to a previously set reference orientation; an image rotator configured to rotate the captured video images, according to the terminal rotation state, to provide rotated video images; a video call unit configured to transmit one of the captured video images and the rotated video images based on the terminal rotation state; and a hardware processor implementing at least one of the capturing unit, the orientation sensor, the image rotator, and the video call unit.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Hereinafter, detailed example embodiments will be described with reference to the accompanying drawings. However, the described embodiments are merely examples and not to be construed as limiting the scope of the present disclosure.

To avoid obscuring the key points, details familiar to those who work in this field are intentionally omitted. Terminology described below is defined considering functions in the present disclosure and may vary according to a user's or operator's intention or usual practice. Thus, the meanings of the terminology should be interpreted based on the overall context of the present specification.

The scope and spirit of the inventive concept are to be determined with reference to the appended claims, to that end, the following exemplary embodiments are provided only to efficiently describe the inventive concept to those of ordinary skill in the art.

A system for a video call according to an exemplary embodiment includes one or more terminals that transmit images obtained by capturing or photographing video call users, and a video call control server that generates a combined image by combining the images transmitted by the one or more terminals and transmitting the combined image to the terminals. Here, the respective images are rotated according to information on orientations sensed by the terminals, and combined into the combined image.

In an exemplary embodiment, the terminal may be configured to transmit the orientation information to the video call control server. In this case, the video call control server may rotate the images received from the terminals to a reference orientation according to the orientation information, and generate the combined image by combining the rotated images.

The terminals may transmit aspect ratio information on the terminals to the video call control server together with the orientation information. Also, when the orientations of the terminals are changed after the orientation information is transmitted, the terminals may transmit updated orientation information corresponding to the changed orientations to the video call control server.

In another exemplary embodiment, the terminals may sense their orientations, rotate captured images to a reference orientation according to information on the sensed orientations, and transmit the rotated images to the video call control server. Also, when the orientations of the terminals are changed after the orientation information is transmitted, the terminals may rotate captured images according to orientation information corresponding to the changed orientations, and transmit the rotated images to the video call control server.

In this case, the video call control server may check screen modes of the terminals using the images transmitted by the terminals, generate the combined image according to the screen modes, and transmit the combined image to the terminals. Also, the video call control server may check aspect ratios together with the screen modes of the terminals using the images received from the terminals, generate the combined image according to the screen modes and the aspect ratios, and transmit the combined image to the terminals.

A method for a video call according to exemplary embodiments includes transmitting, at one or more terminals, images obtained by capturing video call users, generating, at a video call control server, a combined image by combining the images transmitted by the one or more terminals, and transmitting, at the video call control server, the combined image to the terminals. The respective images constituting the combined image are rotated according to information on orientations sensed by the terminals, and combined.

In an exemplary embodiment, transmitting the images may further include transmitting the orientation information on the terminals to the video call control server. In this case, generating the combined image may include rotating the images received from the terminals to a reference orientation according to the received orientation information, and generating the combined image by combining the rotated images.

In addition, transmitting the images may further include transmitting aspect ratio information on the terminals to the video call control server together with the orientation information.

Meanwhile, when the orientations of the terminals are changed after transmission of the orientation information, the terminals may transmit orientation information corresponding to the changed orientations to the video call control server.

In another exemplary embodiment, transmitting the images may further include sensing orientations of the terminals, and rotating the captured images to a reference orientation according to information on the sensed orientations and transmitting the rotated images to the video call control server. In this case, when the orientations of the terminals are changed after the orientation information is transmitted, the terminals may rotate the captured images according to orientation information corresponding to the changed orientations and transmit the rotated images to the video call control server.

Generating the combined image may further include checking screen modes of the terminals using the images transmitted by the terminals, and generating the combined image according to the screen modes. Also, generating the combined image may further include checking aspect ratios of the terminals together with the screen modes using the images received from the terminals, and generating the combined image according to the screen modes and the aspect ratios.

A system for a video call configured as described above according to exemplary embodiments will be described in detail below.

FIG. 2is a diagram of a system for a video call according to a first exemplary embodiment.

Referring toFIG. 2, a system200for a video call according to this exemplary embodiment includes a plurality of terminals202and a video call control server204. Although four terminals202are shown in the drawing, the number of terminals202is not limited to four, and two or more terminals202may be used.

Each of the terminals202requests a video call from the video call control server204. In this embodiment and the following embodiments, the terminals202may include all types of terminals such as a portable terminal, a fixed terminal, a wireless terminal, a wired terminal, etc. capable of participating in a video call. For example, a cellular phone such as a smartphone, a personal digital assistant (PDA), a tablet personal computer (PC), a laptop computer, or a desktop computer may be a terminal in the present disclosure. As an example of the terminals202, a portable terminal in the form of a cellular phone is shown here, but the terminals202are not limited to portable terminals in the form of a cellular phone.

Each of the terminals202transmits orientation information about the terminal202itself to the video call control server204while requesting the video call from the video call control server204. Here, the orientation information is any type of information that can be used to determine how much the corresponding terminal202has been rotated with respect to a reference orientation.

FIG. 3is a diagram showing orientations of a terminal according to the first exemplary embodiment rotated with respect to a reference orientation. Referring toFIG. 3, each terminal202may be in a state as shown inFIG. 3(reference) in which the terminal202is in a reference orientation. Likewise, each terminal202may be in a state as shown inFIG. 3(a) in which the terminal202is rotated by 90 degrees clockwise with respect to a reference orientation (i.e., rotated by 270 degrees counterclockwise with respect to the reference orientation). Likewise, each terminal202may be in a state as shown inFIG. 3 (b)in which the terminal202is rotated by 180 degrees clockwise with respect to the reference orientation (rotated by 180 degrees counterclockwise with respect to the reference orientation). Finally, each terminal202may be in a state as shown inFIG. 3 (c)in which the terminal202is rotated by 270 degrees clockwise with respect to the reference orientation (rotated by 90 degrees counterclockwise with respect to the reference orientation).

Here, the rotation state of the terminal202is understood to vary from the reference orientation in multiples of 90 degrees (i.e., 0 degree, 90 degrees, 180 degrees, and 270 degrees), and this rotation state depends upon how much a user rotates the terminal202. Since it is practically impossible for a human user carrying the terminal202in one hand to consistently rotate the terminal202at an accurate angle, a threshold value for a rotation angle of the terminal202may be set in advance, and the rotation state may be changed according to the threshold value. For example, the rotation state may be set to the reference state when the rotation angle of the terminal202is −45 degrees to 45 degrees; rotated by 90 degrees clockwise when the rotation angle is 46 degrees to 135 degrees; rotated by 180 degrees clockwise when the rotation angle is 136 degrees to 225 degrees; and rotated by 270 degrees clockwise when the rotation angle is 226 degrees to 315 degrees.

The reference orientation of the terminal202may vary according to a setting, a communication service provider, etc., of the terminal202. For example, a state in which the screen of the terminal202is horizontally or vertically placed may be determined as the reference orientation.

In this way, each terminal202transmits its own orientation information, dependent on a state in which the terminal202is rotated with respect to the reference orientation, to the video call control server204. Meanwhile, when its own orientation is changed at a time after a terminal202has already transmitted its own orientation information to the video call control server204, the terminal202again transmits orientation information indicating the changed orientation to the video call control server204.

In addition, each terminal202may transmit its own reference aspect ratio information to the video call control server204. Here, the reference aspect ratio information denotes a width-to-height ratio of the corresponding terminal202in the reference orientation. For example, width-to-height aspect ratios of the terminals202may be 4:3 or 16:9 in the reference orientation, and each terminal202may transmit such reference aspect ratio information to the video call control server204. At this time, each terminal202may transmit its own orientation information and reference aspect ratio information using a session initiation protocol (SIP) message.

Each terminal202transmits an image obtained by capturing a user to the video call control server204. At this time, the image may be shown to be rotated by a predetermined angle with respect to a reference orientation according to a orientation of the terminal202when capturing the image. For example, when the terminal202is rotated by 90 degrees clockwise with respect to the reference orientation to capture the user, an image obtained by capturing the user is shown to be rotated by 90 degrees counterclockwise with respect to the reference orientation.

When an image transmitted from each terminal202has been rotated by the predetermined angle with respect to the reference orientation, the video call control server204rotates the image again to the reference orientation, and then generates a combined image by combining respective images.

Specifically, the video call control server204may check whether or not an image transmitted from each terminal202has been rotated by the predetermined angle with respect to the reference orientation using orientation information transmitted from the terminal202. When it is checked that the image transmitted by the terminal202has been rotated by the predetermined angle with respect to the reference orientation, the video call control server204rotates the image again to the reference orientation. For example, when the image transmitted by the terminal202has been rotated by 90 degrees counterclockwise with respect to the reference orientation, the video call control server204rotates the image by 90 degrees clockwise (or 270 degrees counterclockwise) to place the image in the reference orientation.

Then, all images transmitted by the respective terminals202are placed in the reference orientation. The video call control server204generates the combined image by combining the images all placed in the reference orientation, and then transmits the combined image to the respective terminals202.

At this time, when reference aspect ratio information has been transmitted from each terminal202, the video call control server204generates the combined image with an aspect ratio corresponding to the reference aspect ratio information, and then transmits the combined image to the terminals202. For example, when an aspect ratio of a terminal202is 4:3, the video call control server204generates the combined image with an aspect ratio of 4:3, and then transmits the combined image to the terminal202. Also, when an aspect ratio of a terminal202is 16:9, the video call control server204generates the combined image with an aspect ratio of 16:9, and then transmits the combined image to the terminal202.

At this time, the video call control server204may separately generate as many combined images according to each of aspect ratios of the respective terminals202, and then transmit the combined images according to the aspect ratios of the respective terminals202to the corresponding terminals202. For example, when the respective terminals202have two types of aspect ratios of 4:3 and 16:9, the video call control server204may separately generate a combined image having an aspect ratio of 4:3 and a combined image having an aspect ratio of 16:9, and then transmit the combined image having the aspect ratio of 4:3 to respective terminals202having the aspect ratio of 4:3 and the combined image having the aspect ratio of 16:9 to respective terminals202having the aspect ratio of 16:9.

In addition, according to an exemplary embodiment, the video call control server204checks orientation information on each terminal202, and then transmits a combined image with an aspect ratio corresponding to a current orientation of the terminal202to the terminal202. For example, when an aspect ratio of a terminal202is 4:3, and the terminal202is currently in the reference orientation, the video call control server204generates a combined image with an aspect ratio of 4:3 and transmits the combined image to the terminal202. Also, according to an exemplary embodiment, when an aspect ratio of a terminal202is 4:3, and the terminal202has been rotated by 90 degrees clockwise with respect to the reference orientation, the video call control server204generates a combined image with an aspect ratio of 3:4 and then transmits the combined image to the terminal202.

In an exemplary embodiment, each terminal202transmits its own orientation information to the video call control server204, and the video call control server204rotates an image rotated by a predetermined angle with respect to a reference orientation to the reference orientation again and then generates a combined image so that respective terminals202can receive the combined image in which respective images are placed in the reference orientation from the video call control server204even when the respective terminals202capture users in different orientations and transmit the captured images. In this case, the users of the respective terminals202can do a video call without having to view images of other users in an unnatural rotation.

FIG. 4is a diagram illustrating a process in which a video call control server rotates images using orientation information transmitted from respective terminals and then generates a combined image in the system for a video call according to the first exemplary embodiment.

Referring toFIG. 4, a terminal202-1transmits an image A captured in a reference orientation to the video call control server204, a terminal202-2transmits an image B captured in a state in which the terminal202-2has been rotated by 90 degrees clockwise with respect to the reference orientation to the video call control server204, a terminal202-3transmits an image C captured in a state in which the terminal202-3has been rotated by 180 degrees clockwise with respect to the reference orientation to the video call control server204, and a terminal202-4transmits an image D captured in a state in which the terminal202-4has been rotated by 270 degrees clockwise with respect to the reference orientation to the video call control server204. In this case, the image B, the image C, and the image D are shown to be rotated by 90 degrees, 180 degrees, and 270 degrees counterclockwise with respect to the reference orientation, respectively. Here, the respective terminals202-1,202-2,202-3and202-4may transmit their own orientation information and reference aspect ratio information to the video call control server204in advance of the images A, B, C and D.

Then, the video call control server204finds that the image B, the image C, and the image D have been rotated by 90 degrees, 180 degrees, and 270 degrees counterclockwise with respect to the reference orientation respectively from the orientation information received from the respective terminals202-1,202-2,202-3, and202-4. In this case, the video call control server204rotates the image B, the image C, and the image D by 90 degrees, 180 degrees, and 270 degrees clockwise respectively to place them in the reference orientation, generates a combined image by combining the images A, B, C, and D, and then transmits the combined image to the respective terminals202-1,202-2,202-3, and202-4.

At this time, assuming that an aspect ratio of the respective terminals202-1,202-2,202-3, and202-4is 16:9, a current aspect ratio of the terminals202-1and202-3is 16:9, and thus the video call control server204generates a combined image I′ with an aspect ratio of 16:9, and then transmits the combined image I′ to the terminals202-1and202-3. On the other hand, a current aspect ratio of the terminals202-2and202-4is 9:16, and thus the video call control server204generates a combined image I″ with an aspect ratio of 9:16, and then transmits the combined image I″ to the terminals202-2and202-4.

In the combined image I′ and the combined image I″, the respective images A, B, C, and D are shown to have the same aspect ratios as the original images. However, the present disclosure is not limited to this, and the respective images A, B, C, and D may be processed with at least one of a different aspect ratio and screen size than those of the original images and combined in the combined image I′ and the combined image I″. For example, when the respective images A, B, C, and D have the same aspect ratios as the original images in the combine image I′, there are empty spaces on the left and right of the images B and D in the combined image I′, and there are empty spaces above and under the images A and C in the combined image I″. To prevent the presence of empty spaces above and under each image or on the left and right of the image in the combined images I′ and I″, aspect ratios and screen sizes of images may be adjusted and combined. For example, in the combined image I′, the upper and lower sides of the images B and D may be cut off, and the sizes of the images B and D may be adjusted so that faces of video call users occupy the entire corresponding screen. Also, in the combined image I″, the left and right sides of the images A and C may be cut off. In addition, when a part of an image is cut off so as to be combined into a combined image, a face of a person in the image may be recognized so as not to cut off part of the face. Since image recognition technology is well known in the technical field to which the present disclosure pertains, detailed description thereof will be omitted.

FIG. 5is a block diagram showing a constitution of a video call control server according to the first exemplary embodiment. Referring toFIG. 5, the video call control server204includes a communicator500, a storage502, an image rotator504, and a combined image generator506.

The communicator500receives orientation information, and images obtained by capturing users of the corresponding terminals202, from the respective terminals202. The communicator500may also receive reference aspect ratio information from the respective terminals202. Also, the communicator500transmits a combined image, generated by the combined image generator506, to the respective terminals202.

The storage502stores the orientation information and captured images of the respective terminals202received by the communicator500. The storage502may also store the reference aspect ratio information on the respective terminals202received by the communicator500.

The image rotator504checks whether or not there is an image required to be rotated to a reference orientation, among the images transmitted by the respective terminals202, based on the orientation information on the respective terminals202. When there is an image required to be rotated to the reference orientation, the image rotator504rotates the image to place the image in the reference orientation.

The combined image generator506generates a combined image by combining images that are not required to be rotated to the reference orientation (i.e., images having been already placed in the reference orientation) among the images received from the respective terminals202and images that have been rotated to the reference orientation by the image rotator504among the images received from the respective terminals202. At this time, the combined image generator506may generate the combined image with an aspect ratio corresponding to the reference aspect ratio information and the orientation information transmitted by the respective terminals202.

FIG. 6is a flowchart illustrating a method for a video call according to the first exemplary embodiment.

Referring toFIG. 6, while requesting a video call from the video call control server204, the respective terminals202transmit their own orientation information to the video call control server204(S602). At this time, the respective terminals202may transmit their own reference aspect ratio information together with the orientation information. However, the reference aspect ratio information is not necessarily transmitted together with the orientation information, and may be transmitted separately from the orientation information, according to an exemplary embodiment.

Next, the respective terminals202transmit images obtained by capturing users to the video call control server204(S604). At this time, the images transmitted to the video call control server204may be shown to be rotated by a predetermined angle with respect to a reference orientation according to orientations of the respective terminals202when capturing.

Next, the video call control server204checks whether or not there is an image required to be rotated to the reference orientation among the images received from the respective terminals202using the orientation information received from the respective terminals202(S606).

When it is checked in S606that there is an image required to be rotated to the reference orientation, the video call control server204rotates the image to the reference orientation (S608). When images required to be rotated to the reference orientation are rotated to the reference orientation, all the images received from the respective terminals202are placed in the reference orientation.

Next, the video call control server204generates a combined image by combining the images placed in the reference orientation, and then transmits the combined image to the respective terminals202(S610). Here, when the reference aspect ratio information has been transmitted from the respective terminals202, the video call control server204generates the combined image with an aspect ratio corresponding to the reference aspect ratio information and the orientation information, and then transmits the combined image to the respective terminals202.

Meanwhile, when its own orientation is changed after each terminal202transmits its own orientation information to the video call control server204, the terminal202may transmit orientation information corresponding to the changed orientation to the video call control server204again.

FIG. 7is a diagram of a system for a video call according to a second exemplary embodiment.

Referring toFIG. 7, a system700for a video call according to the second exemplary embodiment includes a plurality of terminals702and a video call control server704. Although four terminals702are shown in the drawing, the number of terminals702is not limited to four, and two or more terminals702may be used.

Each of the terminals702transmits an image captured by the terminal702itself to the video call control server704while requesting a video call from the video call control server704. At this time, after checking its own orientation information, each of the terminals702rotates the captured image according to the orientation information, and transmits the rotated image to the video call control server704. Here, the orientation information denotes how much a current orientation of the terminal702has been rotated with respect to a previously set reference orientation.

FIG. 8is a diagram showing orientations of a terminal according to the second exemplary embodiment rotated by every 90 degrees clockwise with respect to a reference orientation and captured images rotated according to the orientations rotated with respect to the reference orientation.

Referring toFIG. 8 (a), a terminal702is in a reference orientation. InFIG. 8(b) the terminal702is rotated by 90 degrees clockwise with respect to the reference orientation (rotated by 270 degrees counterclockwise with respect to the reference orientation). InFIG. 8 (c), the terminal702is rotated by 180 degrees clockwise with respect to the reference orientation (rotated by 180 degrees counterclockwise with respect to the reference orientation). InFIG. 8 (d)the terminal702is rotated by 270 degrees clockwise with respect to the reference orientation (rotated by 90 degrees counterclockwise with respect to the reference orientation).

In other words, a rotation state of the terminal702changes from the reference orientation at multiples of 90 degrees (i.e., 0 degree, 90 degrees, 180 degrees, and 270 degrees), and varies according to how much a user rotates the terminal702. As described in the first exemplary embodiment, since it is practically impossible for a user to consistently rotate the terminal702by hand at an accurate angle, a threshold value of a rotation angle of the terminal702may be set in advance, and the rotation state may be changed according to the threshold value.

As shown in (a) ofFIG. 8, when the terminal702is in the reference orientation, the terminal702transmits a captured image to the video call control server704as it is. Here, the image is transmitted to the video call control server704in a horizontal screen mode. In exemplary embodiments, the horizontal screen mode denotes a case in which a horizontal screen length (or resolution) of the corresponding image is greater than a vertical screen length, and a vertical screen mode denotes a case in which a vertical screen length (or resolution) of the corresponding image is greater than a horizontal screen length. For example, an aspect ratio of the corresponding image may be set to 4:3 or 16:9 in the horizontal screen mode, and set to 3:4 or 9:16 in the vertical screen mode.

As shown in (b) ofFIG. 8, when the terminal702has been rotated by 90 degrees clockwise with respect to the reference orientation, an image captured by the terminal702is displayed to be rotated by 90 degrees counterclockwise. At this time, the terminal702rotates the captured image by 90 degrees clockwise and transmits the rotated image to the video call control server704so that a video call user is displayed in a reference direction in the captured image. The reference direction denotes a direction in which a video call user is displayed in a normal orientation in a captured image (e.g., a orientation in which the video call user's head is directed upward in the captured image). Here, the image transmitted to the video call control server704is transmitted in the vertical screen mode.

As shown in (c) ofFIG. 8, when the terminal702has been rotated by 180 degrees clockwise with respect to the reference orientation, an image captured by the terminal702is displayed to be rotated by 180 degrees counterclockwise. At this time, the terminal702rotates the captured image by 180 degrees clockwise and transmits the rotated image to the video call control server704so that the video call user is displayed in the reference direction in the captured image. Here, the image transmitted to the video call control server704is transmitted in the horizontal screen mode.

As shown in (d) ofFIG. 8, when the terminal702has been rotated by 270 degrees clockwise with respect to the reference orientation, an image captured by the terminal702is displayed to be rotated by 270 degrees counterclockwise. At this time, the terminal702rotates the captured image by 270 degrees clockwise and transmits the rotated image to the video call control server704so that the video call user is displayed in the reference direction in the captured image. Here, the image transmitted to the video call control server704is transmitted in the vertical screen mode.

As described above, each terminal702checks how much a current orientation of the terminal702itself has been rotated with respect to a reference orientation (i.e., checks orientation information), rotates a captured image according to the checked orientation information, and then transmits the rotated image to the video call control server704. Here, when each terminal702rotates a captured image so that a video call user is displayed in a reference direction in the captured image, even if the terminal702has been rotated with respect to the reference orientation, the video call user is displayed in the reference direction in the captured image.

When its own orientation is changed after each terminal702transmits a captured image, the terminal702may rotate a captured image according to the changed orientation using the same method as described above, and transmit the rotated image to the video call control server704. Then, the video call control server704may analyze the captured image rotated according to the changed orientation, and then generate a combined image corresponding to a current screen mode of the terminal702.

Here, since a state in which a terminal702is horizontally placed is set as a reference orientation, images in (a) and (c) ofFIG. 8are transmitted in the horizontal screen mode, and images in (b) and (d) ofFIG. 8are transmitted in the vertical screen mode. However, the reference orientation is not limited to the state in which a terminal702is horizontally placed.

Next, the video call control server704generates a combined image by combining the respective images transmitted from the respective terminals702. At this time, the video call control server704may generate the combined image according to a current screen mode (i.e., the horizontal screen mode or the vertical screen mode) of the corresponding terminal702.

Specifically, after checking the current screen mode (i.e., the horizontal screen mode or the vertical screen mode) of the corresponding terminal702using the respective images transmitted from the respective terminals702, the video call control server704generates a combined image according to the current screen mode of the terminal702and transmits the combined image to the terminal702. At this time, the video call control server704may transmit the combined image with an aspect ratio of the terminal702. In other words, the video call control server704may generate a combined image according to the current screen mode and the aspect ratio of the terminal702, and transmit the combined image to the terminal702.

For example, when a current screen mode of a terminal702is the horizontal screen mode, and an aspect ratio is 4:3, the video call control server704may generate a combined image with an aspect ratio of 4:3 in the horizontal screen mode, and transmit the combined image to the terminal702. Also, when a current screen mode of a terminal702is the vertical screen mode, and an aspect ratio is 16:9, the video call control server704may generate a combined image with an aspect ratio of 9:16 in the vertical screen mode, and transmit the combined image to the terminal702.

At this time, the video call control server704may generate as many combined images as there are types of aspect ratios of the respective terminals702, and then transmit the combined images according to current screen modes and aspect ratios of the respective terminals702to the corresponding terminals702. For example, when the respective terminals702have two types of aspect ratios of 4:3 and 16:9, the video call control server704may separately generate a combined image having an aspect ratio of 4:3, a combined image having an aspect ratio of 3:4, a combined image having an aspect ratio of 16:9, and a combined image having an aspect ratio of 9:16, and then transmit the combined images to the respective terminals702according to the current screen modes and the aspect ratios of the respective terminals702.

Every time an image is transmitted from each terminal702(or periodically), the video call control server704may analyze the image and check a current screen mode of the terminal702. At this time, when a current screen mode of a terminal702has been changed, the video call control server704may generate a combined image according to the changed screen mode, and transmit the combined image to the terminal702. For example, when a user rotates a terminal702from a horizontal orientation to a vertical orientation, the terminal702rotates an image according to the rotation direction and transmits the rotated image to the video call control server704, and the video call control server704checks that a screen mode of a received image has been changed from the horizontal screen mode to the vertical screen mode and may generate a combined image according to the changed screen mode.

In this exemplary embodiment, each terminal702rotates a captured image according to its own orientation information and transmits the rotated image to the video call control server704, and the video call control server704generates a combined image according to a current screen mode of the terminal702and transmits the combined image to the terminal702. Thus, even when respective terminals702capture their users in different orientations to perform a video call, it is possible to receive a combined image in which the video call users are displayed in a reference direction from the video call control server704. In this case, the users of the respective terminals702can enjoy the video call without having to view other users in an unnatural orientation.

FIG. 9is a diagram illustrating a process in which a video call control server according to the second exemplary embodiment generates a combined image using images transmitted from respective terminals.

Referring toFIG. 9, a first terminal702-1transmits an image captured in a reference orientation to the video call control server704as it is, a second terminal702-2rotates an image captured in a state in which the second terminal702-2has been rotated by 90 degrees clockwise with respect to the reference orientation by 90 degrees clockwise and then transmits the rotated image to the video call control server704, a third terminal702-3rotates an image captured in a state in which the third terminal702-3has been rotated by 180 degrees clockwise with respect to the reference orientation by 180 degrees clockwise and then transmits the rotated image to the video call control server704, and a fourth terminal702-4rotates an image captured in a state in which the fourth terminal702-4has been rotated by 270 degrees clockwise with respect to the reference orientation by 270 degrees clockwise and then transmits the rotated image to the video call control server704.

Here, a state in which the screen of a terminal702is horizontally placed is assumed to be the reference orientation. In this case, images A and C respectively transmitted from the first terminal702-1and the third terminal702-3are transmitted in the horizontal screen mode, and images B and D respectively transmitted from the second terminal702-2and the fourth terminal702-4are transmitted in the vertical screen mode. Also, an aspect ratio of the respective terminals702is assumed to be 16:9.

Then, the video call control server704may check current screen modes and aspect ratios of the first terminal702-1and the third terminal702-3using the images A and C respectively transmitted from the first terminal702-1and the third terminal702-3. Also, the video call control server704may check current screen modes and aspect ratios of the second terminal702-2and the fourth terminal702-4using the images B and D respectively transmitted from the second terminal702-2and the fourth terminal702-4.

In this case, the video call control server704combines the images A, B, C, and D to generate a combined image I′ in the horizontal screen mode with an aspect ratio of 16:9, and then transmits the combined image I′ to the first terminal702-1and the third terminal702-3. Also, video call control server704combines the images A, B, C, and D to generate a combined image I″ in the vertical screen mode with a aspect ratio of 9:16, and then transmits the combined image I″ to the second terminal702-2and the fourth terminal702-4.

As described in the first exemplary embodiment, the respective images A, B, C, and D may be processed with an aspect ratio and/or a screen size different from those of the original images, and combined in the combined image I′ and the combined image I″.

For example, as in the first exemplary embodiment, an aspect ratio or size may be adjusted to prevent the presence of empty spaces, or partial facial recognition may be performed to cut off a part of an image.

FIG. 10is a block diagram showing a constitution of a terminal according to the second exemplary embodiment.

Referring toFIG. 10, a terminal702according to the second exemplary embodiment includes a capturing unit1000, a orientation sensor1002, an image rotator1004, and a video call unit1006.

The capturing unit1000captures its front using a camera. For example, the capturing unit1000may capture a video call user during a video call.

The orientation sensor1002obtains information on a orientation of the terminal702. In other words, the orientation sensor1002senses how much the terminal702has been rotated with respect to a previously set reference orientation. The orientation sensor1002may include, for example, a gyro sensor, a gravity sensor, or so on.

The image rotator1004rotates the image captured by the capturing unit1000according to the orientation information on the terminal702. For example, the image rotator1004checks the orientation information obtained by the orientation sensor1002, and rotates the image captured by the capturing unit1000to display the video call user in a reference direction in the image when the terminal702has been rotated with respect to the reference orientation. In this case, even when the terminal702has been rotated with respect to the reference orientation, the video call user is displayed to be kept in the reference direction in the rotated image.

The image rotator1004delivers the rotated image or the image captured by the capturing unit1000to the video call unit1006. In other words, when the terminal702maintains the reference orientation, the image rotator1004delivers the image captured by the capturing unit1000to the video call unit1006as it is.

The video call unit1006requests a video call from the video call control server704. The video call unit1006transmits the image captured by the capturing unit1000or the image rotated by the image rotator1004to the video call control server704. The video call unit1006receives a combined image from the video call control server704.

FIG. 11is a block diagram showing a constitution of a video call control server according to the second exemplary embodiment. Referring toFIG. 11, the video call control server704includes a communicator1100, a storage1102, an image analyzer1104, and a combined image generator1106.

The communicator1100performs communication for a video call with respective terminals702. For example, the communicator1100receives captured images from the respective terminals702. Here, the captured images may be images rotated by image rotators1004. Also, the communicator1100transmits a combined image generated by the combined image generator1106to the respective terminals702.

The storage1102stores captured images of the respective terminals702received by the communicator1100. The storage1102may store the combined image generated by the combined image generator1106. In this case, the storage1102may be a buffer that temporarily stores the captured images of the respective terminals702and the combined image.

The image analyzer1104analyzes the captured images of the respective terminals702received by the communicator1100, thereby detecting current screen modes and aspect ratios of the respective terminals702. In other words, the image analyzer1104analyzes a horizontal screen length (or resolution) and a vertical screen length (or resolution) of a captured image of each terminal702, thereby detecting a current screen mode and an aspect ratio of the corresponding terminal702.

The combined image generator1106generates the combined image by combining the captured images received from the respective terminals702. The combined image generator1106may generate a combined image according to a current screen mode (horizontal screen mode or vertical screen mode) of each terminal702. At this time, the combined image generator1106may generate the combined image with an aspect ratio of the terminal702.

FIG. 12is a flowchart illustrating a method for a video call according to the second exemplary embodiment. Referring toFIG. 12, while requesting a video call from the video call control server704, the respective terminals702check their own orientation information (S1202). At this time, the respective terminals702may check their own orientation information using a gyro sensor, a gravity sensor, etc. installed therein.

Next, the respective terminals702rotate captured images according to their own orientation information, and transmit the rotated images to the video call control server704(S1204). At this time, the respective terminals702may rotate the captured images so that video call users are displayed in a previously set reference direction in the images.

Next, the video call control server704checks current screen modes and aspect ratios of the respective terminals702using the images transmitted from the respective terminals702(S1206).

Next, the video call control server704generates a combined image according to a current screen mode and an aspect ratio of each terminal702using an image transmitted from the terminal702, and then transmits the combined image to the terminal702(S1208).

Meanwhile, when a orientation of a given terminal702is changed during a video call, the terminal702may rotate a captured image according to the changed orientation, and then transmit the rotated image to the video call control server704. Then, after analyzing the captured image rotated according to the changed orientation, the video call control server704may generate a combined image corresponding to a current screen mode of the corresponding terminal702and transmit the combined image to the terminal702.

Meanwhile, exemplary embodiments may include a computer-readable recording medium including a program for performing the methods described herein on a computer. The computer-readable recording medium may separately include program commands, local data files, local data structures, etc. or include a combination of them. The medium may be specially designed and configured for the present disclosure, or known and available to those of ordinary skill in the field of computer software. Examples of the computer-readable recording medium include magnetic media, such as a hard disk, a floppy disk, and a magnetic tape, optical recording media, such as a CD-ROM and a DVD, magneto-optical media, such as a floptical disk, and hardware devices, such as a ROM, a RAM, and a flash memory, specially configured to store and perform program commands. Examples of the program commands may include high-level language codes executable by a computer using an interpreter, etc. as well as machine language codes made by compilers. A computer, such as the one mentioned above, is very familiar to those who practice in these technical arts. Therefore, the discussion herein has avoided obscuring the key features of the exemplary embodiments by purposefully omitting details concerning the manner in which a hardware processor of such a computer system uses the above-identified computer-readable codes and data storage device to carry out the various functions or implement the various units previously mentioned. Likewise, since a person familiar with this field understands that such functions and units may be implemented through various combinations of hardware and/or software, such implementation details are likewise omitted.

In exemplary embodiments, even when respective terminals capture users in different orientations and perform a video call, it is possible to receive a combined image of respective images in which video call users are all displayed in a reference direction from a video call control server. In this case, the users of the respective terminals can enjoy the video call without viewing other users in an unnatural orientation.

It will be apparent to those skilled in the art that various modifications can be made to the above-described exemplary embodiments without departing from the spirit or scope of the present disclosure. Thus, it is intended that the present disclosure covers all such modifications provided they come within the scope of the appended claims and their equivalents.

DESCRIPTION OF REFERENCE NUMBERS