There is provided an information processing device and an information processing method capable of giving a sense of realism as if a partner connected in video communication is present in one's sight. The information processing device including a communication unit configured to transmit and receive an image to and from a communication connection destination. Further, a control unit is configured to control a display range and a position of the image, on the basis of a display size and installation position information of a display device that displays the image and distance information of a subject appearing in the image that is acquired from the image.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a U.S. National Phase of International Patent Application No. PCT/JP2017/016435 filed on Apr. 25, 2017, which claims priority benefit of Japanese Patent Application No. JP 2016-151268 filed in the Japan Patent Office on Aug. 1, 2016. Each of the above-referenced applications is hereby incorporated herein by reference in its entirety.

TECHNICAL FIELD

BACKGROUND ART

In recent years, as communication technologies have developed, TV phone systems for having a conversation while looking at video of a partner at remote locations have been proposed. Specifically, for example, a display device, a camera, a microphone, and a speaker are provided in each place, and captured video and collected audio data in one place are output in real time from a display device and a speaker installed in the other place. Each user is located in front of the camera, and makes a conversation while looking at a figure of the partner appearing in the display device.

In addition, in regard to guiding a user's position with respect to a large-screen TV, for example, Patent Literature 1 below describes a method of guiding the user to an appropriate viewing position by performing facial recognition in order to prevent the user from viewing a TV screen at a position inappropriate for viewing by, for example, approaching the TV screen.

CITATION LIST

Patent Literature

DISCLOSURE OF INVENTION

Technical Problem

However, even if it is possible to make a conversation while looking at a figure of a partner in video communication, a sense of realism as if the partner is in one's sight cannot be obtained.

Hence, the present disclosure proposes an information processing device, an information processing method, and a program capable of giving a sense of realism as if a partner connected in video communication is present in one's sight.

Solution to Problem

According to the present disclosure, there is proposed an information processing device including: a communication unit configured to transmit and receive an image to and from a communication connection destination; and a control unit configured to control a display range and a position of the image, on the basis of a display size and installation position information of a display device that displays the image, and distance information of a subject appearing in the image that is acquired from the image.

According to the present disclosure, there is proposed an information processing method including, by a processor: transmitting and receiving an image to and from a communication connection destination; and controlling a display range and a position of the image, on the basis of a display size and installation position information of a display device that displays the image, and distance information of a subject appearing in the image that is acquired from the image.

According to the present disclosure, there is proposed a program causing a computer to function as: a communication unit configured to transmit and receive an image to and from a communication connection destination; and a control unit configured to control a display range and a position of the image, on the basis of a display size and installation position information of a display device that displays the image, and distance information of a subject appearing in the image that is acquired from the image.

Advantageous Effects of Invention

According to the present disclosure as described above, a sense of realism as if a partner connected in video communication is present in one's sight can be given.

MODE(S) FOR CARRYING OUT THE INVENTION

In addition, description will be given in the following order.

1. Overview of communication system according to embodiment of present disclosure

3. Operation processing

4. Other examples

5. Hardware configuration

1. Overview of Communication System According to Embodiment of Present Disclosure

An overview of a communication system1according to an embodiment of the present disclosure is described with reference toFIG. 1.FIG. 1illustrates an overall configuration of the communication system1according to an embodiment of the present disclosure.

As illustrated inFIG. 1, information processing devices10A,10B, and10C are connected via a network20. The information processing devices10A,10B, and10C are disposed respectively in space A, space B, and space C, and the spaces are connected by an information channel of video, sound, and the like, which makes it possible to feel as if the spaces are connected. For example, when a parent and a child live away from each other, their living spaces can be made to look connected to each other via a display device corresponding to a window or a door. In the case where living spaces are connected to each other, a user can live while roughly grasping a situation of a partner space (e.g., a situation of a child, a situation of a parent, or the like).

The information processing devices10A to10C perform synchronization processing of two-way communication, calculation and control of spatial distance, etc.; the calculation and control of separation distance, etc. may be performed by a processing server30. In addition, the communication system1includes the processing server30in the example illustrated inFIG. 1, but the present disclosure is not limited to this, and the processing server30does not need to be included.

BACKGROUND

Here, as described above, even if it is possible to make a conversation while looking at a figure of a partner in video communication, it is difficult to give a sense of realism as if the partner is in one's sight. To address this, it may be possible to express a sense of distance to a partner by controlling a display size, because the partner displayed small feels far, and the partner displayed large feels close, for example.

However, such a method is based on the premise that a person is confronting a camera, and a position of a display is not prescribed; hence, for example, in the case where a display60is disposed near the ground as illustrated inFIG. 2, a partner displayed large looks as if being in one's sight because a sense of distance becomes closer, but looks as if the partner is buried in the ground; thus, a very unnatural state occurs.

Hence, in the present embodiment, a display range of partner video is decided appropriately in accordance with a physical size and an installation position of a display; thus, a sense of realism as if a partner connected in video communication is present in one's sight can be given.

For example, in displaying video in which the whole body of a communication partner is seen, video of the communication partner is displayed in respective appropriate ranges in accordance with physical sizes and installation positions of displays15aand15binstalled on walls as illustrated inFIG. 3. The information processing device10performs control in a manner that the partner's face is located at an appropriate height from the ground, assuming that a position where the display15is installed is a virtual standing position Pv. The display15aillustrated on the left side ofFIG. 3has a lower side touching the ground; hence, a sense of realism can be given without unnaturalness even if the whole body of the communication partner is displayed.

On the other hand, the display15billustrated on the right side ofFIG. 3is installed at a position away from the ground; hence, displaying video of the whole body of the communication partner as it is results in a state where the communication partner is floating in air, makes life-size display impossible, and gives no sense of realism. Hence, the information processing device10controls a display range in a manner that the communication partner's face is disposed at an appropriate height h in the case of assuming that the communication partner has stood at the virtual standing position Pv, thereby making it possible to give a sense of realism as if the communication partner is present in one's sight.

In this manner, in the present embodiment, a life-size subject person can be displayed at an optimum position, without using distance information of the subject person measured with a special distance detector; thus, a sense of presence as if a connection partner is present there can be given.

The communication system1according to an embodiment of the present disclosure has been described. Now, a configuration of the information processing device10included in the communication system1and operation processing of the communication system1will be described in detail with reference to drawings.

2. Configuration of Information Processing Device

FIG. 4is a block diagram illustrating an example of a configuration of the information processing device10according to the present embodiment. As illustrated inFIG. 4, the information processing device10includes a control unit11, a camera12, a microphone13, a communication unit14, a display15, a speaker16, and a storage unit17.

The control unit11functions as an arithmetic processing device and a control device, and controls the overall operation of the information processing device10in accordance with a variety of programs. The control unit11is implemented, for example, by an electronic circuit such as a central processing unit (CPU) and a microprocessor. In addition, the control unit11may include a read only memory (ROM) that stores a program, an operation parameter and the like to be used, and a random access memory (RAM) that temporarily stores a parameter and the like varying as appropriate.

In addition, the information processing device10according to the present embodiment also functions as an image analysis unit111, a calculation unit112, an image generation unit113, a display control unit114, and a transmission control unit115.

The image analysis unit111analyzes video of a connection destination space received via the communication unit14. For example, the image analysis unit111performs object recognition, face detection, facial recognition, or the like.

The calculation unit112acquires various distance information of the communication partner appearing in video of the connection destination space, on the basis of an analysis result by the image analysis unit111. Calculation of distance information will be described in detail with reference toFIGS. 9 to 18.

The image generation unit113decides an appropriate range to perform life-size display on the display15, on the basis of various distance information of the communication partner calculated by the calculation unit112, information of a camera installed in the connection destination space (including specs of the camera and installation position information of the camera), and a physical size and an installation position (a height from the ground, or the like) of the display15, and performs clipping, enlargement, reduction, or the like of the range.

The display control unit114controls image display from the display15. For example, the display control unit114performs control to display, at an appropriate position, an image that is output from the image generation unit113and obtained by clipping an appropriate range. For example, the display control unit114adjusts a display position of a center position of the face in a manner that a person appearing in the image looks as if actually standing there, assuming that a position where the display15is installed is a virtual standing position, and performs life-size display.

The transmission control unit115performs control to transmit, from the communication unit14to another information processing device10installed in the connection destination space, a captured image of a space where the information processing device10is installed captured by the camera12and an audio signal of the space collected by the microphone13.

The camera12includes a lens system including an imaging lens, a diaphragm, a zoom lens, a focus lens, and the like, a drive system that causes the lens system to perform focus operation and zoom operation, a solid-state image sensor array that generates an imaging signal by photoelectrically converting imaging light obtained by the lens system, and the like. The solid-state image sensor array may be implemented by, for example, a charge coupled device (CCD) sensor array or a complementary metal oxide semiconductor (CMOS) sensor array. In addition, the camera12according to the present embodiment images the inside of a space, and acquires a captured image.

The microphone13collects sound in a space and environmental sound around the space (e.g., in a neighboring room, a hall, the outside of a house, etc.), and acquires audio data.

The communication unit14is a communication module for transmitting and receiving data to and from another device in a wired/wireless manner. The communication unit14wirelessly communicates with an external device directly or via a network access point in a scheme, for example, a wired local area network (LAN), a wireless LAN, Wireless Fidelity (Wi-Fi) (registered trademark), infrared communication, Bluetooth (registered trademark), short-range/contactless communication, and the like. The communication unit14according to the present embodiment connects to another information processing device10or the processing server30via the network20, and transmits and receives data.

The display15is a display device that displays video of a connection destination space received via the communication unit14. The display15may be, for example, a liquid crystal display (LCD), an organic electroluminescence (EL) display, or the like.

The speaker16is an output device that outputs audio data. For example, the speaker16outputs sound of a connection destination space received via the communication unit14.

The storage unit17stores programs and parameters for the control unit11described above to execute functions. In addition, the storage unit17includes a storage device including a storage medium, a recording device for recording data on the storage medium, a reader device for reading out data from the storage medium, a deletion device for deleting data recorded on the storage medium, and the like. Note that the storage unit17may be, for example, a storage medium, such as a nonvolatile memory such as flash ROM (or flash memory), electrically erasable programmable read-only memory (EEPROM), and erasable programmable ROM (EPROM), a magnetic disk such as a hard disk and a disk-type magnetic disk, an optical disc such as a compact disc (CD), a digital versatile disc recordable (DVD-R), and a Blu-Ray disc (BD; registered trademark), and a magneto optical (MO) disk.

3. Operation Processing

Now, operation processing of the communication system1according to the present embodiment will be described with reference toFIG. 5.FIG. 5is a flowchart illustrating operation processing of the communication system1according to the present embodiment. Here, as an example, description is given on the communication system1in which the information processing device10A installed in space A and the information processing device10B installed in space B are connected to connect space A and space B.

As illustrated inFIG. 5, first, the information processing device10B acquires camera information (step S103), and transmits camera parameters to the information processing device10A (step S106). Camera information includes specs of a camera and installation position information of a camera. Here, a positional relationship between a camera and a subject is described with reference toFIGS. 6 and 7.FIG. 6is a diagram for describing spec information and installation position information of the camera12B according to the present embodiment. The upper stage ofFIG. 6illustrates a side view of the camera12B installed on a wall and a subject (user B) that is imaged, and the lower stage ofFIG. 6illustrates a top view of the camera12B and user B. In addition,FIG. 7illustrates an image121captured by the camera12B. In this case, the following information is acquired as specs of the camera itself.

—Specs of Camera

Angle of view in direction horizontal to ground: φC

Angle of view in direction perpendicular to ground: θC

Number of pixels in direction horizontal to ground: XP

Number of pixels in direction perpendicular to ground: YP

In addition, in the case where the camera12B is installed in a manner that an imaging direction is parallel to the ground, the information processing device10B acquires a height HCGat which the camera12B is installed as installation information as illustrated inFIG. 6.

Next, the information processing device10A records received camera information in the storage unit17(step S109).

On the other hand, the information processing device10B performs imaging by the camera12B (step S112), and transmits a captured image (video) to the information processing device10B (step S115). Imaging and transmission are continuously performed in real time.

Next, the information processing device10A determines whether or not a face is included in the received video (step S118). Specifically, the image analysis unit111performs face detection.

Then, in the case where a face is included (Yes in step S118), the information processing device10A determines whether or not life-size display is necessary (step S121). Whether or not life-size display is necessary is, for example, determined depending on ON/OFF of a life-size display function on the basis of user setting, or determined on the basis of a user action or the like. For example, in the case where it can be determined that the face is seen but user B is not in a situation of making a conversation with user A, such as the case where user B has only passed in front of the camera12B during cleaning, the case where user B is watching TV while sitting in a sofa, or the case where user B is concentrating on study or work, the information processing device10A determines that life-size display is unnecessary. Such an action of user B may be recognized on the basis of an analysis result by the image analysis unit111, for example.

Next, in the case where it is determined that life-size display is necessary (Yes in step S121), the information processing device10A calculates various distance information of the subject, and decides a display range (steps S124to S130).

Specifically, the information processing device10A calculates a distance of the face of the subject (user B who is a communication partner) appearing in the video by the calculation unit112(step S124). Here, an example of distance calculation of a face is described with reference toFIGS. 8 to 10.FIG. 8is a diagram for describing a distance of a face. As illustrated inFIG. 8, in this specification, “distance of a face” is a distance DFfrom the camera12B to the face of user B. Such a distance DFof a face can be calculated in the following manner, for example.

FIG. 9is a diagram for describing facial information that can be acquired from the captured image121. As illustrated inFIG. 9, a horizontal direction of the image121is defined as an X axis, and a vertical direction is defined as a Y axis. The information processing device10A according to the present embodiment can perform face detection and facial recognition from the image121by the image analysis unit111, and obtain the following information.

—Information that can be Obtained in Face Detection and Facial Recognition

x coordinate of face center in captured image: XF

y coordinate of face center in captured image: YF

Width of face in captured image: WF

Height of face in captured image: HF

Age estimated from imaged face: AF

Sex estimated from imaged face: SF

Then, the calculation unit112of the information processing device10A refers to a distance conversion graph for each age or sex prepared in advance, including a size “HF/WF” of the acquired face, an age “AF”, and also a sex “SF” in some cases, and calculates a distance DFof the face from the camera12B.FIG. 10illustrates an example of a distance conversion graph for each age or sex. As illustrated inFIG. 10, the size of the imaged face is proportional to the distance from the camera to the face. The size of the face generally differs depending on age and sex; hence, as illustrated inFIG. 10, relationship curves of an adult male, an adult female, and a child are shown. Thus, the calculation unit112can calculate a distance of the subject from the camera from a size of a face and age or sex of the subject in an image. Note that the method for calculating a distance of a face described here is an example, and the present disclosure is not limited to this.

Then, the calculation unit112calculates a height of the face of the subject (step S112). Here, an example of calculation of a height of a face is described with reference toFIGS. 11 and 12.FIG. 11is a diagram for describing calculation of a height, from the ground, of a face of a subject in a captured image.FIG. 12is a diagram for describing distance information the can be acquired from a captured image in face height calculation.

The calculation unit112obtains a distance dCG0, from the camera12B, of a ground end G0on the nearest side of the captured image, on the basis of camera information acquired in step S109, according to the following formula 1.

In addition, the calculation unit112obtains a vertical actual length hp and a horizontal actual length wPof an image appearing in the camera12B at the distance DFfrom the camera12B, according to the following formulas 2 and 3, respectively.

Furthermore, when DF>dCG0(i.e., in the case where the distance from the camera12B to the subject is longer than the distance from the camera12B to the ground end G0on the nearest side of the captured image), the calculation unit112obtains a distance dFG0from G0to the person's face according to the following formula 4.
[Math. 4]
dFG0=DF−dCG0formula 4

Consequently, from a homothetic ratio, a length of hGPillustrated inFIG. 11is expressed by the following formula 5.

In addition, from coordinates of the face in the captured image, a length of hFPillustrated inFIG. 11is expressed by the following formula 6.

Consequently, the height of the face from the ground indicated by hFGinFIG. 11is obtained by the following formula 7.

As described above, according to the present embodiment, the information processing device10A can calculate an actual height of a face of the subject (user B who is a communication partner) from the ground on the basis of spec information and installation position information of the camera12B and an analysis result of an image captured by the camera12B.

Note that even if the subject is in a non-upright posture, a height hFGfrom the ground to the face can be obtained as long as a face is detected and the distance DFfrom the camera12B to the face is calculated. Here,FIGS. 13 and 14are diagrams for describing a case where a subject is not upright.FIG. 13is a diagram for describing the positional relationship between a camera and a subject in face height calculation in the case where the subject is not upright.FIG. 14is a diagram for describing distance information calculated from a captured image in face height calculation in the case where the subject is not upright.

In this case, the height hFGfrom the ground to the face illustrated inFIG. 14can be obtained similarly by using the above formulas 1 to 7.

In addition, in the case where the position of the face from the camera12B is at a distance equal to that of the ground end G0on the nearest side of the image captured by the camera12B, that is, in the case where the distance DFfrom the camera12B to the face=dCG0, dFG0=0 and hGP=0 are satisfied. Here,FIGS. 15 and 16are diagrams for describing a case where the position of the subject's face is at a distance equal to that of the ground end.FIG. 15is a diagram for describing the positional relationship between a camera and a subject in face height calculation in the case where the position of the subject's face is at a distance equal to that of the ground end.FIG. 16is a diagram for describing distance information calculated from a captured image in face height calculation in the case where the position of the subject's face is at a distance equal to that of the ground end. As illustrated inFIGS. 15 and 16, in the case where the subject is standing at a ground end on the nearest side of the image captured by the camera12B (distance DF=dCG0), dFG0=0 and hGP=0 are satisfied. Consequently, the calculation unit112calculates the height hFGfrom the ground to the face by using the following formula 8.

Furthermore, in the case where the position of the subject's face is closer to the camera12B than the ground end G0on the nearest side of the image captured by the camera12B is, the distance DFfrom the camera12B to the face <dCG0. Here,FIGS. 17 and 18are diagrams for describing a case where the position of the subject's face is closer to the camera12B than the ground end is.FIG. 17is a diagram for describing the positional relationship between a camera and a subject in face height calculation in the case where the position of the subject's face is closer to the camera12B than the ground end is.FIG. 18is a diagram for describing distance information calculated from a captured image in face height calculation in the case where the position of the subject's face is closer to the camera12B than the ground end is. As illustrated inFIGS. 17 and 18, in the case where the subject is standing on the nearer side than a ground end on the nearest side of the image captured by the camera12B (distance DF<dCG0), dFG0is expressed by the following formula 9.
[Math. 9]
dFG0=dCG0−DFformula 9

Consequently, hGPillustrated inFIGS. 17 and 18is expressed by the following formula 10.

Thus, the calculation unit112calculates the height hFGfrom the ground to the face by using the following formula 11.

As described above, no matter in what distance the subject is standing with respect to the camera12B, the height hFGfrom the ground to the face can be calculated.

Then, as illustrated inFIG. 5, the image generation unit113of the information processing device10A appropriately decides a range of display on the display15A from a received image, on the basis of the height hFGfrom the ground to the face calculated by the calculation unit112and a physical size and installation position information (a distance between a lower side of the display and the ground) of the display15A (step S130). Here, decision of the display range of display on the display15A is described with reference toFIGS. 19 and 20.FIG. 19is a diagram for describing a display range22in the image121received from the information processing device10B.FIG. 20is a diagram for describing actual dimensions of the display15A and a display image that is displayed.

Vertical and horizontal lengths (length in perpendicular direction: HD, length in horizontal direction of display: WD) of the display15A illustrated inFIG. 20and a distance HDGbetween the lower side and the ground are obtained in advance as size information and installation position information of the display15A.

On the basis of the size information and the installation position information, and the height hFGfrom the ground to the subject's face calculated in step S127, coordinates of an upper left pixel dT(xT, yT) and a lower right pixel dB(xB, yB) of the captured image are calculated and the display range22is decided, as illustrated inFIG. 19, in order to display an appropriate image at an appropriate position of the display15A (more specifically, to display the subject's face at the same height as the actual height from the ground).

Specifically, first, the image generation unit113calculates y coordinates of dTand dB. Here, since hFG(how tall the subject is) is given as a physical distance, a distance hTFbetween the center of the face and an upper side of the screen is obtained according to the following formula 12.
[Math. 12]
hTF=(HD+HDG)−hFGformula 12

Such a physical distance is

yn=YPhP*hTF
when expressed by the number of pixels of the captured image; hence, the y coordinate yTof dTis as follows.

In addition, similarly, the y coordinate yBof dBis as follows.

Next, x coordinates of dTand dBare obtained. Specifically, they are obtained respectively by the following formulas 13 and 14 in a manner that the face is located at the center of the display screen, for example.

As described above, the image generation unit113calculates coordinates of an upper left pixel dT(xT, yT) and a lower right pixel dB(xB, yB) of the captured image, and decides a range to be clipped of the captured image. By displaying a display image clipped in this manner on the display15A, user B who is the communication partner looks as if being actually present at the position where the display15A is installed (the virtual standing position Pv), as illustrated inFIG. 20.

Then, as illustrated inFIG. 5, it is determined whether or not a face is within the decided display range22(step S133).

In the case where a face is not within the display range, the information processing device10A transmits, to the information processing device10B, guiding information for guiding user B by sound, text, or the like in a manner that the face falls within the display range22(step S136). That is, as a result of deciding the display range so as to display the life-size subject in accordance with the size and installation position of the display15A, the partner's face does not fall within the display15A in some cases, as illustrated on the left side ofFIG. 21, for example. In this case, the information processing device10A can notify the imaging side that the face is not seen, and guide the subject to move to an appropriate imaging position as illustrated on the right side ofFIG. 21.

Next, on the basis of guiding information, the information processing device10B performs display of a guiding message, or the like on the display15B (step S139). The information processing device10A and the information processing device10B perform two-way communication, and the camera12and the display15are installed in each space; hence, the guiding message can be displayed on the display15B on the imaging side. Here, with reference toFIGS. 22 and 23, an example of guiding of the imaging side according to the present embodiment is described.FIGS. 22 and 23are diagrams for describing an example of guiding of the imaging side according to the present embodiment.

As illustrated in the upper stage ofFIG. 22, a guiding message31such as “Your face is outside display area. Please move down a little.” may be displayed on the display15B on the imaging side, for example. On the display15B on the imaging side, a captured image of user A on the partner side is displayed in real time by two-way communication. By the guiding message31being displayed on such a screen, user B can move in a manner that his/her own face enters a display area on the user A side. In addition, as illustrated in the lower stage of theFIG. 22, an image32of user B captured by the camera12B on the imaging side may be displayed in real time on the display15B on the imaging side, and a display area display33indicating an area displayed on the partner side may be displayed superimposed on the image32. Thus, a display area on the partner side (user A side) can be visualized to guide user B.

In addition, as illustrated in the upper stage ofFIG. 23, in a portion that does not obstruct conversation, such as a periphery of a screen, a frame34may be displayed for notification, for example, in blue when user B is at an appropriate position, and in red when user B is at an inappropriate position. In addition, as illustrated in the lower stage ofFIG. 23, a method of guiding by sound35such as “Your face is outside display area. Please move down a little.” is also possible. Note that instead of feedback by an audio message, sound effects or BGM different between when at an appropriate position and when at an inappropriate position can be played for notification. For example, sound may be made like “beeeeep” when entering an appropriate range, and sound may be kept being made like “beep, beep, beep” when in an inappropriate region.

Then, as illustrated inFIG. 5, when it is determined that a face is within the display range (Yes in step S133), the image generation unit113of the information processing device10determines whether or not an image of the decided display range is sufficient (step S136).

In the case where an image of the display range is lacking (No in step S136), the image generation unit113complements a lacking region (step S139). This is because as a result of deciding an appropriate display range as described above, a case may occur in which part of the display range of the display15A does not have image information. This will be specifically described with reference toFIG. 24.

FIG. 24is a diagram for describing image complementation. As illustrated on the left ofFIG. 24, as a result of adjusting a center position of the subject's face in an image126received from the partner side so as to perform life-size display, a portion36lacking in image information occurs in some cases in a range23of display on the display15A. For example, in the case where the subject gets too close to the camera, or the like, the lacking portion36may occur as illustrated on the left ofFIG. 24. In this case, the image generation unit113complements image information of the lacking portion as illustrated on the right ofFIG. 24. A method for generating a complementary image37is not particularly limited, and it can be generated by prediction from past images and a current image, for example.

Then, as illustrated inFIG. 5, the display control unit114displays an image of the decided display range on the display15A (step S142).

Note that in the case where a face is not included in the received image (No in step S118), or in the case where it is determined not to be necessary to perform life-size display (No in step S121), the display control unit114displays the received image as it is on the display15A (step S142).

The operation processing of the information processing device10according to the present embodiment has been described. Note that in the example illustrated inFIG. 5, one-way processing of transmitting a captured image from the information processing device10B to the information processing device10A is described for convenience in description; however, the communication system1according to the present embodiment is capable of two-way communication, and the processing described above may be performed similarly between the information processing devices10that are connected.

In addition, the above operation processing mainly describes transmission of a captured image (video), but audio information of the subject may also be transmitted together.

4. Other Examples

In addition to the embodiment described above, the information processing device10may clip a person portion from a captured image, for example, and display the person portion superimposed on a computer graphics (CG) space. This will be described with reference toFIG. 25.

FIG. 25is a diagram for describing life-size display in a CG space according to another example of the present disclosure. As illustrated inFIG. 25, in the case where a CG space is displayed on the display15A, a person (subject)24clipped from a captured image received from a communication destination is displayed in life-size display on the assumption that the person24is standing at the same place as a window25or a door in the CG space, for example.

Specifically, as in the embodiment described above, the information processing device10acquires an actual height of the person24from the ground to the face, from the received image and camera information on the imaging side, and life-size display is performed on the assumption that the same place as the window25or the door in the CG space is the virtual standing position Pv, as illustrated inFIG. 25.

<4-2. Delivery to Plurality of Locations>

In addition, as another example of the present disclosure, the information processing device10can deliver video to a plurality of locations (spaces).FIG. 26is a diagram for describing delivery to a plurality of spaces.

As illustrated inFIG. 26, video127captured in space B, for example, may be delivered to space A and space C by the information processing device10B. The display15A in space A and a display15C in space B differ in physical size and installation position as illustrated inFIG. 26, but processing (decision of a display range, complementation of a lacking image, etc.) of the video127is performed by each of the information processing devices10A and10C on the reception side (see steps S118to S142inFIG. 5); thus, life-size display can be performed appropriately in accordance with environment on the reception side.

5. Hardware Configuration

The embodiment of the present disclosure has been described. The processing performed by the information processing device10described above may be achieved by operating cooperatively software and hardware of an information processing device100(to be described later).

FIG. 27is an explanatory diagram illustrating a hardware configuration of the information processing device100according to the present disclosure. As illustrated inFIG. 27, the information processing device100includes a central processing unit (CPU)142, read only memory (ROM)144, random access memory (RAM)146, a bridge148, a bus150, an interface152, an input device154, an output device156, a storage device158, a drive160, a connection port162, and a communication device164.

The CPU142functions as an arithmetic processing device and a control device to enable operation of the image analysis unit111, the calculation unit112, the image generation unit113, the display control unit114, and the transmission control unit115of the information processing device10by operating cooperatively with various kinds of programs. In addition, the CPU142may be a microprocessor. The ROM144stores programs, operation parameters, or the like used by the CPU142. The RAM146transiently stores programs used when the CPU142is executed, parameters that change as appropriate when the CPU142is executed, or the like. The ROM144and the RAM146implement part of the storage unit17in the information processing system. The CPU142, the ROM144, and the RAM146are connected to each other through an internal bus including a CPU bus and the like.

The input device154includes an input means to which the user inputs information such as the camera12and the microphone13of the information processing device10, a mouse, a keyboard, a touchscreen, a button, a switch, and a lever, an input control circuit that generates an input signal on the basis of imaging by a camera or the user input and outputs the generated input signal to the CPU142, and the like. By operating the input device154, the user of the information processing device100can input various kinds of data into the information processing device100and instruct the information processing device100to perform a processing operation.

The output device156, for example, performs output from a device such as a liquid crystal display (LCD) device, an organic light emitting diode (OLED) device, or a lamp, as an example of the display15of the information processing device10. In addition, the output device156performs audio output from a speaker, a headphone, or the like, as an example of the speaker16of the information processing device10.

The storage device158is a data storage device. The storage device158may include a storage medium, a recording device that records data on the storage medium, a reading device that reads data from the storage medium, a deletion device that deletes data recorded on the storage medium, and the like. The storage device158stores various kinds of data or a program to be executed by the CPU142.

The drive160is a reader/writer for a storage medium, and is incorporated in or externally attached to the information processing device100. The drive160reads information recorded in a removable storage medium that is mounted, such as a magnetic disk, an optical disc, a magneto-optical disk, or semiconductor memory, and outputs the information to the RAM144. The drive160is also capable of writing information to the removable storage medium.

The connection port162is, for example, a bus used to connect to an information processing device or peripheral equipment outside the information processing device100. In addition, the connection port162may be a universal serial bus (USB).

The communication device164is, for example, a communication interface implemented by communication equipment for connection with a network, as an example of the communication unit14of the information processing device10. In addition, the communication device164may be a device supporting infrared communication, a communication device supporting a wireless local area network (LAN), a communication device supporting long term evolution (LTE), or a wired communication device that performs wired communication.

As described above, the communication system1according to the embodiment of the present disclosure can give a sense of realism as if a partner connected in video communication is present in one's sight.

Specifically, in capturing video, camera information (camera specs and installation position information) is transmitted to the partner side. On the reception side, a physical size and installation position information of a display are grasped in advance, an actual distance of a person is estimated from a size of the person's face appearing in received video, and a height of the person's face from the ground is calculated. Then, on the reception side, a display position of the person's face is adjusted assuming that a position of the display installed on a wall or the like is a virtual standing position, and video obtained by clipping an appropriate range is subjected to enlargement, reduction, complementation, or the like as needed, and displayed in life-size display.

For example, a computer program for causing hardware such as a central processing unit (CPU), read only memory (ROM), and random access memory (RAM) built in the information processing device10or the processing server30described above to exhibit functions of the information processing device10or the processing server30can also be produced. Furthermore, a computer-readable storage medium in which the computer program is stored is also provided.

In addition, the display range is decided by calculating the actual height of the face of the subject (communication partner) from the ground from the received image in the embodiment described above, but this is an example; in the present embodiment, an actual height of a predetermined body area of the subject from the ground may be calculated, and the display range may be decided so as to perform life-size display.

An information processing device including:

a communication unit configured to transmit and receive an image to and from a communication connection destination; and

a control unit configured to control a display range and a position of the image, on the basis of a display size and installation position information of a display device that displays the image, and distance information of a subject appearing in the image that is acquired from the image.

The information processing device according to (1),

in which the installation position information includes height information of the display device from a ground, and

the control unit calculates an actual height from the ground to a face as the distance information of the subject, and performs control so as to perform life-size display of the subject, assuming that the subject is standing at substantially a same position as the display device.

The information processing device according to (2), in which in a case of assuming that the subject is standing at substantially the same position as the display device, the control unit decides the display range of the image in a manner that the face of the subject is located at a same position as the actual height.

The information processing device according to (2) or (3), in which the control unit calculates the height from the ground to the face of the subject, on the basis of a distance from a camera at the communication connection destination to the subject and spec information of the camera, the distance being estimated on the basis of a size of the face appearing in the image and an age of the subject.

The information processing device according to (4), in which the control unit estimates the distance from the camera to the subject on the basis of a relationship curve that indicates a relationship between a size of a face appearing in an image and a distance from a camera to a face and differs depending on an age and/or a sex of the subject.

The information processing device according to (4), in which the spec information of the camera includes angle-of-view information in a direction perpendicular to the ground and angle-of-view information in a direction perpendicular to the ground.

The information processing device according to (3), in which in a case where the face is not within the decided display range, the control unit performs control so as to notify the communication connection destination that the face is outside the display range.

The information processing device according to (7), in which the control unit makes notification to guide the communication connection destination in a manner that the face enters the display range.

The information processing device according to (3), in which in a case where the decided display range includes a portion lacking in image information, the control unit complements an image of the lacking portion.

An information processing method including, by a processor:

transmitting and receiving an image to and from a communication connection destination; and

controlling a display range and a position of the image, on the basis of a display size and installation position information of a display device that displays the image, and distance information of a subject appearing in the image that is acquired from the image.

A program causing a computer to function as:

a communication unit configured to transmit and receive an image to and from a communication connection destination; and

a control unit configured to control a display range and a position of the image, on the basis of a display size and installation position information of a display device that displays the image, and distance information of a subject appearing in the image that is acquired from the image.

REFERENCE SIGNS LIST