Communication terminal, communication system, and communication method

A communication terminal shares data with another communication terminal. The communication terminal includes circuitry. The circuitry receives, from a communication management apparatus that manages the data shared with the another communication terminal, storage location information representing a storage location of background image data transmitted from the another communication terminal, and transmits a download request to an image storing apparatus corresponding to the storage location to request download of the background image data. The circuitry further starts downloading the background image data from the image storing apparatus and receives, from the communication management apparatus, stroke data of a stroke image rendered on the another communication terminal.

CROSS-REFERENCE TO RELATED APPLICATION

This patent application is based on and claims priority pursuant to 35 U.S.C. § 119(a) to Japanese Patent Application No. 2020-035268 filed on Mar. 2, 2020 in the Japan Patent Office, the entire disclosure of which is hereby incorporated by reference herein.

BACKGROUND

Technical Field

The present invention relates to a communication terminal, a communication system, and a communication method.

Description of the Related Art

There is a widely used communication system that enables communication between remote sites via a communication network such as the Internet. An example of such a communication system is a conference system that enables a remote conference. In a typical conference system, image data being held or displayed by a communication terminal used by one of two parties participating in the remote conference is transmitted, via the communication network, to a communication terminal used by the other party, thereby enabling the two parties to share the image data. With this system, the conference between the remote sites is held in a setting close to that of a conference held in the same physical space.

Further, in conferences or meetings held in organizations such as companies, educational institutions, and governmental institutions, a communication terminal such as an electronic whiteboard has been used which displays an image on a display so that a user renders a stroke image of a letter, character, number, or figure, for example, on the image. The stroke image is rendered based on stroke data including coordinate data. The communication terminal generates the stroke data by electronically converting content rendered on the display by a user moving an electronic pen or a finger over the display while keeping the electronic pen or the finger in contact with the display. When the stroke image is rendered on one communication terminal (a first communication terminal), stroke data for reproducing the stroke image is transmitted to the other communication terminal (a second communication terminal) via the communication network. Thereby, the other communication terminal displays the same stroke image.

The stroke data for reproducing the stroke image and the image data of a material being displayed or stored by the first communication terminal are thus transmitted to the second communication terminal via the communication network. Thereby, the second communication terminal is able to display the stroke image as well as a background image such as the image of a material or the image of a background, for example.

In recent years, however, the data of the background image such as the image of a material has been substantially increased in size owing to sophistication of application programs. For example, the data size of the background image data is substantially greater than that of the stroke data for reproducing the stroke image. Consequently, the background image data and the stroke data reach the second communication terminal at different times. That is, the stroke data may reach the second communication terminal before the background image data does. In this case, the second communication terminal displays the background image after the process of displaying the stroke image progresses to some extent. It is therefore after the display of the stroke image progresses to some extent that the user of the second communication terminal realizes the existence of the background image. Consequently, the user of the second communication terminal has difficulty in smooth communication with the user of the first communication terminal.

SUMMARY

In one embodiment of this invention, there is provided an improved communication terminal that shares data with another communication terminal. The communication terminal includes, for example, circuitry. The circuitry receives, from a communication management apparatus that manages the data shared with the another communication terminal, storage location information representing a storage location of background image data transmitted from the another communication terminal, and transmits a download request to an image storing apparatus corresponding to the storage location to request download of the background image data. The circuitry further starts downloading the background image data from the image storing apparatus and receives, from the communication management apparatus, stroke data of a stroke image rendered on the another communication terminal.

In one embodiment of this invention, there is provided an improved communication system that includes, for example, a first communication terminal and a second communication terminal that shares data with the first communication terminal. The first communication terminal includes first circuitry. The first circuitry transmits, to a communication management apparatus that manages the data shared with the second communication terminal, storage location information representing a storage location of background image data. The second communication terminal includes second circuitry. The second circuitry receives the storage location information from the communication management apparatus, and transmits a download request to an image storing apparatus corresponding to the storage location to request download of the background image data. The second circuitry further starts downloading the background image data from the image storing apparatus and receives, from the communication management apparatus, stroke data of a stroke image rendered on the first communication terminal.

In one embodiment of this invention, there is provided an improved communication method executed by a communication terminal that shares data with another communication terminal. The communication method includes, for example, receiving, from a communication management apparatus that manages the data shared with the another communication terminal, storage location information representing a storage location of background image data transmitted from the another communication terminal, transmitting a download request to an image storing apparatus corresponding to the storage position to request download of the background image data, and starting downloading the background image data from the image storing apparatus and receiving, from the communication management apparatus, stroke data of a stoke image rendered on the another communication terminal.

DETAILED DESCRIPTION

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present invention. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. In the drawings illustrating embodiments of the present invention, members or components having the same function or shape will be denoted with the same reference numerals to avoid redundant description.

An embodiment for implementing the present invention will be described below with reference to the drawings.

Schematic Configuration of Communication System

Communication Routes

With reference toFIG. 1, a description will be given of a communication system10of the embodiment that enables a video conference between a plurality of electronic whiteboards1aand1b, while allowing users to perform rendering on the electronic whiteboards1aand1b.

FIG. 1is a schematic diagram illustrating an example of communication routes in the communication system10of the embodiment. The video conference may also be referred to as the television (TV) conference or a remote conference, for example. The video conference, the TV conference, or the remote conference is an example of a session for sharing image data and stroke data between a plurality of terminals. For example, the session for sharing the image data and the stroke data may be a session other than the conference, such as distance learning, remote diagnosis, meeting, or simple conversation. Further, the session for sharing the image data and the stroke data may be used for unidirectional provision of information.

The communication system10includes the plurality of electronic whiteboards1aand1b, at least one relay apparatus3, a communication management apparatus5, and an image storing apparatus7. The electronic whiteboards1aand1bcommunicate therebetween image data and audio data for communication and content data such as image data and stroke data for sharing. The stroke data is data for reproducing (i.e., duplicating) a stroke image, and includes coordinate data, line width data, line color data, vector data, and data representing the writing pressure applied to render a stroke, for example. The stroke data is transmitted and received between the electronic whiteboards1aand1bas a serialized character string. Each of the electronic whiteboards1aand1btransmits and receives therebetween the image data and the audio data for communication to reproduce a site image and sound of the site at which the other electronic whiteboard1aor1bis located, thereby performing remote video communication.

With the electronic whiteboards1aand1btransmitting and receiving therebetween the image data of a background image for sharing, the users of the communication system10(i.e., the participants of the video conference) share the same background image. The background image is an image displayed on respective displays of the electronic whiteboards1aand1b. In the following description, the electronic whiteboards1aand1bmay be collectively referred to as the electronic whiteboards1. The background image includes the image of a material for a conference, for example, which is read from an electronic file and displayed on a display180(seeFIG. 2) of the electronic whiteboard1. The image data of the background image is transmitted and received in a file format such as the joint photographic experts group (JPEG) format. The electronic whiteboards1aand1bfurther transmit and receive therebetween the stroke data of the stroke image for sharing, thereby enabling the users of the communication system10(i.e., the participants of the video conference) to share the same stroke image. The stroke image is the image of a line, for example, which is rendered based on a handwritten stroke made by a user with an electronic pen, for example. The stroke image is displayed based on the stroke data, which represents points identifying coordinates on the display180.

The electronic whiteboards1included in the communication system10are not limited to the two electronic whiteboards1aand1b, and thus the communication system10may include three or more electronic whiteboards1. Each of the electronic whiteboards1is an example of a communication terminal having functions such as a communication function, a rendering function, and a display function. For example, the communication terminal may be a personal computer (PC), a smartphone, a tablet terminal, a smartwatch, a car navigation terminal, a gaming device, or a telepresence robot installed with an application program compatible with the communication system10. Further, the communication terminal may be medical equipment, in which case the background image may be the image of a patient.

FIG. 1illustrates the electronic whiteboards1aand1beach as an example of an electronic whiteboard with a video conference function. The site image, which is based on site image data, may be a video or still image.

An electronic whiteboard as a source of a video conference start request will be referred to as the starting terminal, and an electronic whiteboard as a destination to which the video conference start request is transmitted or relayed will be referred to as the destination terminal. InFIG. 1, the electronic whiteboard1aat a site A and the electronic whiteboard1bat a site B are illustrated as the starting terminal and the destination terminal, respectively. When the video conference start request is transmitted from the electronic whiteboard1b, however, the electronic whiteboard1bserves as the starting terminal, and the electronic whiteboard1aserves as the destination terminal. The electronic whiteboards1aand1bmay be used not only in communication between a plurality of offices and communication between different rooms in the same office but also in communication in the same room, communication between an indoor space and an outdoor space, and communication between an outdoor space and another outdoor space.

The relay apparatus3, which is implemented by a computer, performs a process of relaying the content data for communication between the electronic whiteboards1aand1b.

The communication management apparatus5, which is implemented by a computer, performs login authentication of the electronic whiteboards1aand1band centrally manages the respective communication states of the electronic whiteboards1aand1b, the communication state of the relay apparatus3, and a destination list. The communication management apparatus5further relays the stroke data for sharing between the electronic whiteboards1aand1b.

The image storing apparatus7, which is implemented by a computer, stores the image data of the background image for sharing uploaded from the electronic whiteboard1aand downloads the image data to the electronic whiteboard1b, and vice versa. That is, the image storing apparatus7stores the image data uploaded from the electronic whiteboard1band downloads the image data to the electronic whiteboard1a.

Each of the relay apparatus3, the communication management apparatus5, and the image storing apparatus7may be implemented by a single computer, or may be implemented by a plurality of computers to which units (i.e., functions or devices) of the apparatus are divided and allocated as desired. The communication management apparatus5and the image storing apparatus7form a server system6(seeFIG. 5) that manages the data shared by the electronic whiteboards1aand1b. The server system6may be implemented by a single computer including the units (i.e., functions or devices) of the communication management apparatus5and the image storing apparatus7. Further, instead of the communication management apparatus5, the image storing apparatus7may relay the stroke data for sharing between the electronic whiteboards1aand1b.

In the communication system10, a management information session SEI is established between the electronic whiteboards1aand1bto transmit and receive therebetween various management information via the communication management apparatus5. Further, four sessions are established between the electronic whiteboards1aand1bto transmit and receive therebetween four types of data: high-resolution site image data, intermediate-resolution site image data, low-resolution site image data, and audio data, via the relay apparatus3. InFIG. 1, the four sessions are collectively illustrated as an image and audio data session SED. The image and audio data session SED does not necessarily include four sessions, and may include three or less sessions or five or more sessions. Further, the communication session may be established directly between the starting terminal and the destination terminal without via the relay apparatus3. Further, in the communication system10, the communication management apparatus5may have the function of the relay apparatus3such that the image and audio data session SED is established between the electronic whiteboards1aand1bvia the communication management apparatus5. Further, in the communication system10, the stroke data is transmitted and received between the electronic whiteboards1aand1bvia the management information session SEI.

The image resolution of the site image data in the present embodiment will be described.

The low-resolution site image data, which forms a base image, includes 160 pixels in the horizontal direction by 120 pixels in the vertical direction, for example. The intermediate-resolution site image data includes 320 pixels in the horizontal direction by 240 pixels in the vertical direction, for example. The high-resolution site image data includes 640 pixels in the horizontal direction by 480 pixels in the vertical direction, for example. When the site image data routes through a narrow band path, low-quality image data including the low-resolution site image data for forming the base image is relayed. When the site image data routes through a relatively wide band path, intermediate-quality image data including the low-resolution site image data for forming the base image and the intermediate-resolution site image data is relayed. When the site image data routes through a substantially wide band path, high-quality image data including the low-resolution site image data for forming the base image, the intermediate-resolution site image data, and the high-resolution site image data is relayed. The audio data is smaller in data size than the site image data, and thus is relayed even with a narrow band path.

Use of Electronic Whiteboard

FIG. 2is a diagram illustrating use of the electronic whiteboard1of the embodiment. As illustrated inFIG. 2, the electronic whiteboard1includes a foot151, a support post152, a main unit153, and the display180. A lower side of the foot151is equipped with a plurality of casters, and an upper side of the foot151is equipped with the support post152. The main unit153is disposed on an upper side of the support post152, and has a front surface equipped with the display180. The main unit153includes therein a later-described central processing unit (CPU)101(seeFIG. 3). A user U inputs (i.e., renders) the stroke image of a letter or character, for example, on the display180with an electronic pen190.

Hardware Configurations

With reference toFIGS. 3 and 4, a description will be given of respective hardware configurations of apparatuses and terminals forming the communication system10. A component may be added to or removed from each of the hardware configurations inFIGS. 3 and 4as necessary.

Hardware Configuration of Electronic Whiteboard

FIG. 3is a diagram illustrating an example of the hardware configuration of the electronic whiteboard1of the embodiment. As illustrated inFIG. 3, the electronic whiteboard1includes the CPU101, a read only memory (ROM)102, a random access memory (RAM)103, a solid state drive (SSD)104, a network interface (I/F)105, and an external apparatus connection I/F106.

The CPU101controls an overall operation of the electronic whiteboard1. The ROM102is a nonvolatile memory that stores a program for the CPU101and a program used to drive the CPU101such as an initial program loader (IPL). The RAM103is a volatile memory used as a work area for the CPU101. The SSD104is a high-capacity memory that stores various data of a program for the electronic whiteboard1, for example. The network I/F105is a communication interface for connecting the electronic whiteboard1to a communication network100(seeFIG. 5) to perform communication. The external apparatus connection I/F106is an interface for connecting the electronic whiteboard1to various external apparatuses. The external apparatuses in this case include a universal serial bus (USB) memory130and other external apparatuses such as a microphone140, a speaker150, and a camera160, for example.

The electronic whiteboard1further includes a capture device111, a graphics processing unit (GPU)112, a display controller113, a contact sensor114, a sensor controller115, an electronic pen controller116, a near field communication circuit119, an antenna119afor the near field communication circuit119, a power switch122, and selection switches123.

The capture device111displays image data (i.e., image information) on a display of an external PC170, for example, as a still or video image. The GPU112is a semiconductor chip processor dedicated to graphics processing. The display controller113controls and manages screen display to output an image from the GPU112to the display180, for example. The contact sensor114detects the contact of the electronic pen190or a hand H of a user, for example, on the display180. The sensor controller115controls the processing of the contact sensor114. The contact sensor114performs input and detection of coordinates in accordance with an infrared blocking method. In this method of inputting and detecting coordinates, two light emitting and receiving devices disposed on opposite end portions of an upper area of the display180radiate a plurality of infrared rays parallel to the display180, and receive rays of light reflected by a reflecting member disposed around the display180and returning on optical paths of the rays radiated by the two light emitting and receiving devices. The contact sensor114outputs, to the sensor controller115, identifiers (IDs) of the infrared rays radiated by the two light emitting and receiving devices and blocked by an object. Then, the sensor controller115identifies the position of the coordinates corresponding to the position of contact of the object on the display180. The electronic pen controller116communicates with the electronic pen190to determine contact or non-contact of the head or end of the electronic pen190on the display180. The near field communication circuit119is a communication circuit conforming to a standard such as the near field communication (NFC) or Bluetooth (registered trademark) standard. The power switch122is a switch for switching on or off power supply to the electronic whiteboard1. The selection switches123are switches for adjusting parameters such as the brightness and the color tone of the image displayed on the display180, for example.

The electronic whiteboard1further includes a bus line110, which includes address buses and data buses to electrically connect the CPU101and the other components inFIG. 3to each other.

The contact sensor114is not limited to the infrared blocking method, and may employ a different type of detecting device, such as a capacitance touch panel that identifies the contact position by detecting a change in capacitance, a resistive touch panel that identifies the contact position by detecting a change in voltage of two resistance films facing each other, or an electromagnetic induction touch panel that identifies the contact position by detecting electromagnetic induction caused by contact of an object on a display. Further, the electronic pen controller116may determine contact or non-contact of a part of the electronic pen190held by the user or another part of the electronic pen190, as well as the head or end of the electronic pen190.

Hardware Configurations of Communication Management Apparatus, Relay Apparatus, and Image Storing Apparatus

FIG. 4is a diagram illustrating an example of the hardware configuration of each of the communication management apparatus5, the relay apparatus3, and the image storing apparatus7of the embodiment. A computer as an example of the communication management apparatus5includes a CPU501, a ROM502, a RAM503, a hard disk (HD)504, a hard disk drive (HDD) controller505, a display506, an external apparatus connection I/F508, a network IF509, a keyboard511, a pointing device512, a digital versatile disc rewritable (DVD-RW) drive514, a medium I/F516, and a bus line510.

The CPU501controls an overall operation of the communication management apparatus5. The ROM502is a nonvolatile memory that stores a program used to drive the CPU501such as the IPL. The RAM503is a volatile memory used as a work area for the CPU501. The HD504is a high-capacity memory that stores various data of a communication management program, for example. The HDD controller505controls writing and reading of various data to and from the HD504under the control of the CPU501. The display506displays various information such as a cursor, menus, windows, text, and images. The external apparatus connection I/F508is an interface for connecting the communication management apparatus5to various external apparatuses. The network I/F509is an interface for performing data communication via the communication network100such as the Internet. The keyboard511is an input device including a plurality of keys for inputting text, numerical values, and various instructions, for example. The pointing device512is an input device used to select and execute various instructions, select a processing target, and move the cursor, for example. The DVD-RW drive514controls writing (i.e., storage) and reading of data to and from a DVD-RW513. The DVD-RW513may be replaced by a DVD-recordable (DVD-R), for example. Further, the DVD-RW drive514may be replaced by a Blu-ray (registered trademark) drive that controls writing (i.e., storage) and reading of data to and from a disc such as a Blu-ray disc recordable erasable (BD-RE) or a compact disc rewritable (CD-RW) drive that controls writing (i.e., storage) and reading of data to and from a disc such as a CD-RW. The medium IF516controls writing (i.e., storage) and reading of data to and from a recording medium515such as a flash memory. The bus line510includes address buses and data buses to electrically connect the CPU501and the other components inFIG. 4to each other.

The relay apparatus3is implemented by a typical computer. As illustrated inFIG. 4, the relay apparatus3includes a CPU301, a ROM302, a RAM303, an HD304, an HDD controller305, a display306, an external apparatus connection I/F308, a network I/F309, a keyboard311, a pointing device312, a DVD-RW drive314, a medium I/F316, and a bus line310. These components are similar in configuration to the CPU501, the ROM502, the RAM503, the HD504, the HDD controller505, the display506, the external apparatus connection I/F508, the network I/F509, the keyboard511, the pointing device512, the DVD-RW drive514, the medium I/F516, and the bus line510of the communication management apparatus5, and thus description thereof will be omitted. In the relay apparatus3, however, the HD304stores a relay program.

The image storing apparatus7is implemented by a typical computer. As illustrated inFIG. 4, the image storing apparatus7includes a CPU701, a ROM702, a RAM703, an HD704, an HDD controller705, a display706, an external apparatus connection I/F708, a network I/F709, a keyboard711, a pointing device712, a DVD-RW drive714, a medium I/F716, and a bus line710. These components are similar in configuration to the CPU501, the ROM502, the RAM503, the HD504, the HDD controller505, the display506, the external apparatus connection I/F508, the network I/F50), the keyboard511, the pointing device512, the DVD-RW drive514, the medium I/F516, and the bus line510of the communication management apparatus5, and thus description thereof will be omitted. In the image storing apparatus7, however, the HD704stores an image storage program.

Each of the above-described programs may be distributed as recorded on a computer readable recording medium in an installable or executable file format. Examples of the recording medium include a compact disc recordable (CD-R), a DVD, a Blu-ray disc, and a secure digital (SD) card. The recording medium may be shipped to the market as a program product. For example, the electronic whiteboard1implements a communication method of the present invention by executing a program of the present invention.

General Arrangement of Communication System

A general arrangement of the communication system10will be described withFIG. 5.

FIG. 5is a diagram illustrating an example of the system configuration of the communication system10of the embodiment. InFIG. 5, the electronic whiteboard1aand the electronic whiteboard1bare located at the site A and the site B, respectively. The site A may be a Tokyo branch in Japan, and the site B may be a Beijing branch in China, for example. It is assumed here that the electronic whiteboard1ais used by a user UA1at the site A, and that the electronic whiteboard1bis used by users UB1and UB2at the site B.

The electronic whiteboards1aand1b, the relay apparatus3, the communication management apparatus5, and the image storing apparatus7transmit and receive data to and from each other via the communication network100such as the Internet or a local area network (LAN), for example. The communication via the communication network100is not limited to wired communication, and wireless communication conforming to a standard such as the wireless fidelity (Wi-Fi, registered trademark) standard may take place in some part of the communication network100.

Functional Configuration of Communication System

A functional configuration of the communication system10of the embodiment will be described withFIGS. 6A to 14.

FIGS. 6A and 6Bare diagrams illustrating an example of the functional configuration of the communication system10of the embodiment.FIGS. 6A and 6Billustrate parts of the terminals, apparatuses, and server system inFIG. 5related to later-described processes and operations.

Functional Configuration of Electronic Whiteboard

A functional configuration of each of the electronic whiteboards1aand1bwill be described withFIGS. 6A and 6B. Since the electronic whiteboards1aand1bhave similar functions, the following description will be given of the functions of the electronic whiteboard1with omission of the suffix a or b after each reference numeral.

The electronic whiteboard1includes a communication unit11, a receiving unit12, an image and audio processing unit13, a display control unit14, a determination unit15, an image processing unit16, a near field communication unit18, a display screen managing unit21, and a storing and reading unit19. Each of these units is a function or functional unit implemented when at least one of the components illustrated inFIG. 3operates in response to a command from the CPU101in accordance with the program deployed on the RAM103from the SSD104. The electronic whiteboard1further includes a storage unit1000implemented by the RAM103, the SSD104, or the USB memory130illustrated inFIG. 3. The storage unit1000stores a display screen management database (DB)1001and an operation information management DB1003.

The communication unit11is a function implemented by a command from the CPU101inFIG. 3and the network I/F105inFIG. 3to transmit and receive various data and information to and from another terminal, apparatus, or system via the communication network100. The communication unit11also functions as a starting unit, for example, to perform a process of starting communication with another electronic whiteboard1. The communication unit11further transmits and receives rendering screen data (e.g., the background image data and the stroke data) to be displayed on the display180to and from another electronic whiteboard1via the communication management apparatus5.

The receiving unit12is implemented by a command from the CPU101inFIG. 3and the contact sensor114and the electronic pen controller116inFIG. 3to receive various inputs made by the user with the electronic pen190, for example.

The image and audio processing unit13is implemented by a command from the CPU101inFIG. 3to perform a major process of the video conference function. For example, based on signals output from the microphone140and the camera160, the image and audio processing unit13performs digital processing such as encoding of the site image data and the audio data. Further, for example, based on the site image data and the audio data received by the communication unit11, the image and audio processing unit13generates an image signal and an audio signal. The image and audio processing unit13further performs a process of combining site image data items of different resolutions, for example.

The display control unit14is a function implemented by a command from the CPU101inFIG. 3and the display controller113inFIG. 3to perform control for outputting signals such as the image signal to the display180. For example, the display control unit14controls the display180to display a rendering screen based on the rendering screen data generated by the image processing unit16.

The determination unit15is a function implemented by a command from the CPU101inFIG. 3to make various determinations.

The image processing unit16is a function implemented by a command from the CPU101inFIG. 3and the capture device111inFIG. 3to perform a major process of an electronic whiteboard function. The image processing unit16generates, for example, the stroke data and the stroke image based on the stroke made with the electronic pen190, for example, and received by the receiving unit12. The image processing unit16further generates, for example, the stroke image based on the stroke data received by the communication unit11. The image processing unit16further generates the rendering screen data including the generated stroke image, for example.

The near field communication unit18is a function implemented by a command from the CPU101inFIG. 3and the near field communication circuit119and the antenna119ainFIG. 3to provide data to or acquire data from another terminal having a near field communication unit, such as an integrated circuit (IC) card or a smartphone, in near field wireless communication.

The display screen managing unit21is a function implemented by a command from the CPU101inFIG. 3to manage screen data forming a display screen shared with another electronic whiteboard1.

The storing and reading unit19is a function implemented by a command from the CPU101inFIG. 3to store various data in the storage unit1000and read therefrom various data. Each time the site image data and the audio data are received in the communication with another terminal, the site image data and audio data stored in the storage unit1000are overwritten with the newly received site image data and audio data. The display180displays an image based on the site image data before being overwritten with the newly-received site image data, and the speaker150outputs sound based on the audio data before being overwritten with the newly-received audio data. The storage unit1000further stores the stroke data of the stroke image rendered on the display180and the background image data of the background image displayed on the display180.

Display Screen Information

FIG. 7is a diagram schematically illustrating an example of display screen information of the embodiment.FIG. 7illustrates an example of the conceptual image of the information of the display screen for display on the electronic whiteboard1, which is stored in the display screen management DB1001.

In the example ofFIG. 7, the information of the display screen has a hierarchical structure with layers of information: “meeting,” “page” (i.e., pages P1, P2, P3, and P4), and “stroke” (i.e., strokes SR1, SR2, SR3, SR4, and SR5). The information “stroke” corresponds to the stroke data of the embodiment, and is managed under the corresponding page. The information “page” represents the stored image data of a single-page screen (i.e., the display screen) displayed on the display180. The information “meeting” represents the information of the meeting taking place on the electronic whiteboard1. For example, the stroke SR1includes a stroke ID identifying the stroke SR1(id=aaaaa), information identifying the page P4including the stroke SR1(parent=pag04), a global ID (gid) of the corresponding rendering data G1(gid=g048), and the rendering data G1. If an operation such as a moving operation is performed on the stroke image represented by the stroke SR1, for example, the stroke ID (id=aaaaa) is maintained, but the rendering data G1and the global ID identifying the rendering data G1(gid=g048) are updated. The rendering data G1, which is rendering data for reproducing the stroke SR1, includes coordinate data, line width data, line color data, and vector data, for example.

The electronic whiteboard1is capable of switching between pages. The information “page” corresponds to the pages of the electronic whiteboard1. For example, the page P1includes a page ID identifying the page P1(id=pag1), information identifying the meeting taking place on the electronic whiteboard1(parent=se01), a global ID identifying the page (i.e., page data) (gid=g01), and image data IM1. The image data IM1is the image data of the background image forming the page P1. That is, the image data IM1represents the background image displayed on the page P1. The page P2does not include image data. That is, the page2does not include a background image, i.e., the background of the page P2is blank.

The information “meeting” corresponds to the meeting taking place on the electronic whiteboard1, and is managed with a meeting ID identifying the meeting and a global ID identifying the meeting (i.e., meeting information) (gid=g2). When the electronic whiteboard1is in remote communication with another electronic whiteboard1, the meeting ID corresponds to a session ID identifying the corresponding session (id=se01).

For example, the image processing unit16of the electronic whiteboard1generates the stroke image rendered on the page P4with the rendering data items G1, G2, G3, G4, and G5included in the strokes SR1, SR2, SR3, SR4, and SR5, respectively. The global IDs appearing in the respective layers are assigned by the communication management apparatus5when the electronic whiteboard1shares data with another electronic whiteboard1. The hierarchical structure of the shared data illustrated inFIG. 7is shared by the plurality of electronic whiteboards1.

Operation Information

Operation information representing an operation performed on the display screen of the electronic whiteboard1will be described withFIG. 8.

FIG. 8is a diagram schematically illustrating an example of the operation information of the embodiment. The operation information illustrated inFIG. 8includes the information of the operation performed on the display screen displayed on the electronic whiteboard1. The operation information illustrated inFIG. 8includes a stroke ID (Id) for identifying the corresponding stroke data, a page ID (DistId) for identifying the page on which the stroke data is rendered, operation type information (Operation type) representing the type of the operation performed on the rendering screen, data type information (Data Type) representing the type of data as the operation target, and detailed data information (Data) representing the contents of the data as the operation target.

The operation type information represents the type of the operation performed by the user on a stroke or page in the display screen. If the operation target is the stroke, for example, the operation type information is described as “Add” when the stroke is rendered, described as “Remove” when the stroke is removed, and described as “Update” when the stroke is moved. Further, the data type information is described as “Stroke” when the operation target is the stroke, and described as “Page” when the operation target is the page. Further, when the operation target is the stroke, the detailed data information corresponds to the stroke data, and includes information such as the coordinate data, the line width data, and the line color data. The detailed data information corresponds to information “body” included in a later-described shared data management table (seeFIG. 14). The detailed data information includes information according to the type of the performed operation. In the detailed data information, information such as the coordinate data is updated in accordance with the operation type described in the operation type information.

The operation target is not limited to the data of a stroke or page, and may be the data of an object that is a collection of strokes. The object may be optical character reader (OCR)-processed text of strokes recognized as a letter, a character, or a symbol such as a numerical symbol, a system-generated character representing the date or time, a particular figure such as a triangle, a star, or a circle, an arrow, a line segment, or a line such as a Bezier curve, for example.

Functional Configuration of Relay Apparatus

Referring back toFIG. 6B, a description will be given of a functional configuration of the relay apparatus3.

The relay apparatus3includes a communication unit31(i.e., a forwarding unit), a determination unit35, and a storing and reading unit39. Each of these units is a function or functional unit implemented when at least one of the components illustrated inFIG. 4operates in response to a command from the CPU301in accordance with the relay program deployed on the RAM303from the HD304. The relay apparatus3further includes a storage unit3000implemented by the RAM303, the HD304, or a recording medium315illustrated inFIG. 4.

The communication unit31is a function implemented by a command from the CPU301inFIG. 4and the network I/F309inFIG. 4to transmit and receive various data and information to and from another terminal, apparatus, or system via the communication network100. The communication unit31also functions as a forwarding unit to forward the site image data and the audio data transmitted from a particular terminal to another terminal.

The determination unit35is a function implemented by a command from the CPU301inFIG. 4to make various determinations such as a determination on a data delay state.

The storing and reading unit39is a function implemented by a command from the CPU301inFIG. 4to store various data in the storage unit3000and read therefrom various data.

Functional Configuration of Communication Management Apparatus

A functional configuration of the communication management apparatus5will be described withFIG. 6A.

The communication management apparatus5includes a communication unit51, an authentication unit52, a determination unit53, a terminal managing unit54, a session managing unit55, a relay apparatus managing unit56, a shared data managing unit57, and a storing and reading unit59. Each of these units is a function or functional unit implemented when at least one of the components illustrated inFIG. 4operates in response to a command from the CPU501in accordance with the communication management program deployed on the RAM503from the HD504. The communication management apparatus5further includes a storage unit5000implemented by the RAM503, the HD504, or the recording medium515illustrated inFIG. 4. The storage unit5000stores an authentication management DB5001, a terminal management DB5002, a destination list management DB5003, a session management DB5004, a relay apparatus management DB5005, and a shared data management DB5006.

The communication unit51is a function implemented by a command from the CPU501inFIG. 4and the network I/F509inFIG. 4to transmit and receive various data and information to and from another terminal or apparatus via the communication network100. For example, the communication unit51receives stroke data transmitted to the electronic whiteboard1from another electronic whiteboard1, and transmits stroke data to another electronic whiteboard1.

The authentication unit52is a function implemented by a command from the CPU501inFIG. 4to perform login authentication for a login requesting terminal. If the communication unit51receives login request information, the authentication unit52executes the login authentication process for the login requesting terminal with reference to the authentication management DB5001.

The determination unit53is a function implemented by a command from the CPU501inFIG. 4to make various determinations.

The terminal managing unit54is a function implemented by a command from the CPU501inFIG. 4to manage various information of each of the electronic whiteboards1managed in the terminal management DB5002in accordance with the state of the electronic whiteboard1. The terminal managing unit54updates the information of the terminal management DB5002such as the operation state of the electronic whiteboard1, the time of receipt of information from the electronic whiteboard1, and the internet protocol (IP) address of the electronic whiteboard1, for example. The terminal managing unit54further manages the destination list management DB5003. In response to receipt of a request from one of the electronic whiteboards1, the terminal managing unit54provides the electronic whiteboard1with destination list information including the terminal IDs of destination candidate terminals managed in the destination list management DB5003.

The session managing unit55is a function implemented by a command from the CPU501inFIG. 4to manage the session held in the communication system10. For example, in response to receipt of start request information from one of the electronic whiteboards1, the session managing unit55generates the session ID for identifying the session. The start request information represents the request to start communication. Further, for example, the session managing unit55stores and manages various information of the session in the session management DB5004in association with the session ID.

The relay apparatus managing unit56is a function implemented by a command from the CPU501inFIG. 4. In the present embodiment, the communication system10includes one or more relay apparatuses3. When the communication system10includes a plurality of relay apparatuses3, the relay apparatus managing unit56selects one of the plurality of relay apparatuses3to be used for relaying the session. For example, the relay apparatus managing unit56manages the relay apparatus management DB5005, and selects one of the plurality of relay apparatuses3based on various information of the relay apparatuses3stored in the relay apparatus management DB5005. For example, based on the respective IP addresses of the relay apparatuses3stored in the relay apparatus management DB5005and the IP address of the starting terminal, the relay apparatus managing unit56selects one of the relay apparatuses3that is located near the starting terminal. Further, for example, the relay apparatus managing unit56selects one of the relay apparatuses3based on the respective maximum data transmission speeds of the relay apparatuses3stored in the relay apparatus management DB5005. In the embodiment, a desired method may be employed to select the relay apparatus3for use in relaying the session.

The shared data managing unit57is a function implemented by a command from the CPU501inFIG. 4to manage various data of the display screen transmitted and received between the electronic whiteboards1(i.e., communication terminals). For example, the shared data managing unit57receives the stroke data transmitted via the management information session SEI from one of the electronic whiteboards1participating in the session, and stores and manages the stroke data in the shared data management DB5006in association with the session ID.

The storing and reading unit59is a function implemented by a command from the CPU501inFIG. 4to store various data in the storage unit5000and read therefrom various data.

Authentication Management Table

FIG. 9is a conceptual diagram illustrating an example of an authentication management table of the embodiment. The authentication management DB5001in the storage unit5000is configured as the authentication management table as illustrated inFIG. 9. In the authentication management table, the respective terminal IDs of all electronic whiteboards1managed by the communication management apparatus5are managed in association with respective passwords. For example, the authentication management table illustrated inFIG. 9indicates that the electronic whiteboard1a(i.e., a communication terminal) has a terminal ID “01aa” and a password “aaaa.” The password is an example of authentication information. The authentication information also includes an access token.

Terminal Management Table

FIG. 10is a conceptual diagram illustrating an example of a terminal management table of the embodiment. The terminal management DB5002in the storage unit5000is configured as the terminal management table as illustrated inFIG. 10. In the terminal management table, the destination name, the operation state, the reception time, and the terminal IP address are managed in association with each other for each terminal ID. The terminal ID identifies the corresponding electronic whiteboard1(i.e., a communication terminal). The destination name represents the name of the electronic whiteboard1as the destination. The operation state represents the operation state of the electronic whiteboard1. The reception time represents the date and time of receipt of the login request information by the communication management apparatus5. The terminal IP address represents the IP address of the electronic whiteboard1(i.e., a communication terminal). For example, the terminal management table inFIG. 10indicates that the electronic whiteboard1awith the terminal ID “01aa” has a destination name “TERMINAL AA, TOKYO BRANCH, JAPAN,” an operation state “ON-LINE (COMMUNICATION AVAILABLE).” a reception time of “13:40 on Feb. 10, 2020” as the date and time of receipt of the login request information by the communication management apparatus5, and a terminal IP address “1.2.1.3” corresponding to the terminal ID “01aa.” The terminal ID, the destination name, and the terminal IP address are stored in the terminal management table when the electronic whiteboard1is previously registered to receive a service provided by the communication management apparatus5.

Destination List Management Table

FIG. 11is a conceptual diagram illustrating an example of a destination list management table of the embodiment. The destination list management DB5003in the storage unit5000is configured as the destination list management table as illustrated inFIG. 11. In the destination list management table, the respective terminal IDs of all destination candidate terminals are managed in association with the terminal ID of the starting terminal. The starting terminal is the electronic whiteboard1that transmits a communication start request. The destination candidate terminals are the electronic whiteboards1registered as the candidates for the destination terminal corresponding to the starting terminal. For example, the destination list management table illustrated inFIG. 11indicates that the destination candidate terminals to which the electronic whiteboard1awith the terminal ID “01aa” (i.e., the starting terminal) is able to transmit the communication start request are electronic whiteboards1including the electronic whiteboard1bwith a terminal ID “01ba.” The destination candidate terminals in the destination list management table are updated when a destination candidate terminal is added to or deleted from the destination list management table in response to receipt of an addition request or a deletion request from a given starting terminal to the communication management apparatus5.

The destination list is an example of destination information. In the destination information, the information of the destination candidate terminals such as the terminal IDs thereof may be arranged in a format other than the list format.

Session Management Table

FIG. 12is a conceptual diagram illustrating an example of a session management table of the embodiment. The session management DB5004in the storage unit5000is configured as the session management table as illustrated inFIG. 12. In the session management table, the relay apparatus ID, the starting terminal ID, the destination terminal ID, the delay time, and the delay information reception time are managed in association with each other for each session ID. The session ID identifies the session for implementing the communication between the relay apparatus3and the electronic white boards1(i.e., communication terminals). The relay apparatus ID is the apparatus ID of the relay apparatus3used in the session. The starting terminal ID is the terminal ID of the electronic whiteboard1serving as the starting terminal. The destination terminal ID is the terminal ID of the electronic white board1serving as the destination terminal. The delay time represents the delay time (milliseconds: ms) in the reception of the site image data by the destination terminal. The delay information reception time represents the date and time of receipt of delay information by the communication management apparatus5. The delay information, which represents the delay time, is transmitted from the destination terminal. For example, the session management table illustrated inFIG. 12indicates that the relay apparatus3with a relay apparatus ID “111a” relays the site image data and the audio data between an electronic white board1with a terminal ID “01aa” and an electronic white board1with a terminal ID “01db” in a communication session executed with a session ID “se01.” The session management table further indicates that the delay time of the site image data in the electronic white board1serving as the destination terminal is 200 ms at “13:41 on Feb. 10, 2020.”

Relay Apparatus Management Table

FIG. 13is a conceptual diagram illustrating an example of a relay apparatus management table of the embodiment. The relay apparatus management DB5005in the storage unit5000is configured as the relay apparatus management table as illustrated inFIG. 13. In the relay apparatus management table, the operation state of each relay apparatus3, the date and time of receipt of state information by the communication management apparatus5, the IP address of the relay apparatus3, and the maximum data transmission speed (megabits per second: Mbps) of the relay apparatus3are managed in association with each other for each relay apparatus ID. The relay apparatus ID represents the relay apparatus ID of the relay apparatus3. The state information includes the information of the operation state. For example, the relay apparatus management table illustrated inFIG. 13indicates that the operation state of the relay apparatus3with a relay apparatus ID “111a” is “ON-LINE,” and that the date and time of receipt of the state information by the communication management apparatus5is “13:00 on Feb. 10, 2020.” The relay apparatus management table further indicates that the relay apparatus3has an IP address “1.2.1.2” and a maximum data transmission speed of 100 Mbps.

Shared Data Management Table

FIG. 14is a conceptual diagram illustrating an example of a shared data management table of the embodiment. The shared data management DB5006in the storage unit5000is configured as the shared data management table as illustrated inFIG. 14. In the shared data management table, various data of the display screen shared and stored by the electronic whiteboards1(i.e., communication terminals) is stored and managed for each session ID for identifying the session for implementing the communication between the electronic whiteboards1and the relay apparatus3. The data of the shared data management table corresponds to the display screen information (seeFIG. 7) stored in the display screen management DB1001in each of the electronic whiteboards1.

The data of the shared data management table includes an ID for identifying the corresponding data item, a sequence number representing the generation order of the data item, information “body” representing the contents of the data item, and information “parent” for identifying higher-layer (i.e., parent) data of the data item. For example, in the case of meeting information representing a remote conference being held between a plurality of electronic whiteboards1, the shared data management table includes a meeting ID (i.e., session ID) for identifying the remote conference being held, information “children” representing the pages generated in the remote conference, and information “current page” representing the page forming the display screen currently displayed on the electronic whiteboards1. Further, in the case of page data, for example, the shared data management table includes the page ID for identifying the corresponding page (i.e., page data) and the uniform resource locator (URL) representing the storage location of the background image data forming the page.

Further, in the case of stroke data, for example, the shared data management table includes the stroke ID for identifying the stroke data generated by a stroke rendering event, the information representing the contents of the stroke data, and the information “parent” for identifying the page (i.e., page data) including the stroke. In this case, the information “body” representing the contents of the stroke data includes information such as the color of the rendered stroke, the width representing the thickness of the line of the rendered stroke, and the coordinates (x, y) of the vertices of the rendered stroke.

Herein, the stroke rendering refers to a process of inputting the rendering information performed by the user. For example, the stroke rendering is an event in which the user presses the electronic pen190against the display180, moves the electronic pen190while in contact with the display180, and releases the electronic pen190from the display180. The color of the stroke is expressed by numerical values each ranging from 0 to 25 to represent the corresponding color component in the red-green-blue-alpha (RGBA) data format. The width representing the thickness of the line of the rendered stroke is represented by the number of pixels. Each of the vertices of the rendered stroke is represented by X and Y coordinates. The vertices are connected together by a Bezier curve to form a line segment representing the stroke. With the shared data management table, the communication management apparatus5manages the various data of the display screen including the stroke data generated in a video conference using a particular communication session, for example.

Functional Configuration of Image Storing Apparatus

Referring back toFIG. 6B, a description will be given of a functional configuration of the image storing apparatus7.

The image storing apparatus7includes a communication unit71, a generation unit73, and a storing and reading unit79. Each of these units is a function or functional unit implemented when at least one of the components illustrated inFIG. 4operates in response to a command from the CPU701in accordance with the image storage program deployed on the RAM703from the HD704. The image storing apparatus7further includes a storage unit7000implemented by the RAM703, the HD704, or a recording medium715illustrated inFIG. 4.

The communication unit71is a function implemented by a command from the CPU701inFIG. 4and the network I/F709inFIG. 4to transmit and receive various data and information to and from another terminal or apparatus via the communication network100.

The generation unit73is a function implemented by a command from the CPU701inFIG. 4to generate the URL representing the storage location of the background image data.

The storing and reading unit79is a function implemented by a command from the CPU701inFIG. 4to store various data in the storage unit7000and read therefrom various data.

Processes and Operations of Embodiment

Processes and operations performed in the communication system10of the embodiment will be described withFIGS. 15 to 22D.

Preparatory Process for Remote Communication

A preparatory process for remote communication performed by the electronic whiteboard1aas the login requesting terminal will first be described withFIGS. 15 and 16. The electronic whiteboard1bperforms a login request process similar to that performed by the electronic whiteboard1a, and thus description thereof will be omitted.

FIG. 15is a sequence diagram illustrating an example of a preparatory process for starting remote communication between the electronic whiteboards1.FIG. 16is a diagram illustrating an example of a destination list screen displayed on one of the electronic whiteboards1.

When the power switch122of the electronic whiteboard1ais turned on, the receiving unit12areceives the power-on operation (step S11).

The communication unit11aof the electronic whiteboard1athen transmits the login request information to the communication management apparatus5via the communication network100(step S12). The login request information, which represents the request for login authentication, includes the terminal ID of the electronic whiteboard1aand the password. The terminal ID and the password are data read from the storage unit1000aand transmitted to the communication unit11aby the storing and reading unit19a. The terminal ID and the password transmitted to the communication unit11aare not necessarily read from the storage unit1000a, and may be input by the user with an input device such as a keyboard, or may be read from a recording medium connected to the electronic whiteboard1a, such as a subscriber identify module (SIM) card or an SD card. Then, the communication unit51of the communication management apparatus5receives the login request information transmitted from the electronic whiteboard1a.

Then, the authentication unit52of the communication management apparatus5performs a search through the authentication management table (seeFIG. 9) with search keys set to the terminal ID and the password included in the login request information received via the communication unit51. If the same terminal ID and password as those included in the login request information are managed in the authentication management table, the authentication unit52allows login of the electronic whiteboard1a(step S13), and the processes of step S14and the subsequent steps are executed.

If the login of the electronic whiteboard1ais allowed, the terminal managing unit54updates the information of the terminal management table (seeFIG. 10), specifically the information corresponding to the terminal ID “01aa” of the electronic whiteboard1a. For example, the terminal managing unit54changes the information of the operation state corresponding to the terminal ID “01aa” to “ON-LINE (COMMUNICATION AVAILABLE).” and updates the information of the reception time to the date and time of receipt of the login request information (step S14). The information of the terminal IP address in the terminal management table may be transmitted from the electronic whiteboard1aat step S12instead of being previously registered therein. Thereby, the operation state “ON-LINE (COMMUNICATION AVAILABLE).” the reception time “2020.2.10.13:40 (13:40 on Feb. 10, 2020),” and the terminal IP address “1.2.1.3” are managed in the terminal management table in association with the terminal ID “01aa,” as illustrated inFIG. 10, for example.

Then, the session managing unit55adds a new record to the session management table (seeFIG. 12) and manages the new record (step S15). In the new record, the terminal ID “01aa” of the electronic whiteboard1areceived at step S12is managed as the starting terminal ID. Then, the communication unit51transmits authentication result information to the electronic whiteboard1a(i.e., the source of the login request information) via the communication network100(step S16). The authentication result information represents an authentication result obtained through the process of step S13.

In response to receipt of the authentication result information indicating that the login of the electronic whiteboard1ais allowed, the communication unit11aof the electronic whiteboard1a(i.e., the login requesting terminal) transmits destination list request information to the communication management apparatus5via the communication network100(step S17). The destination list request information represents the request for the destination list. Then, the communication unit51of the communication management apparatus5receives the destination list request information.

Then, the terminal managing unit54performs a search through the destination list management table (seeFIG. 11) with a search key set to the terminal ID “01aa” of the electronic whiteboard1a(i.e., the login requesting terminal), to thereby read from the destination list management table the terminal IDs of the destination candidate terminals that are capable of communicating with the electronic whiteboard1a. The terminal managing unit54further reads, from the terminal management table (seeFIG. 10), the destination names corresponding to the terminal IDs of the destination candidate terminals (step S18). In the present example, the terminal managing unit54extracts the terminal IDs of the destination candidate terminals corresponding to the terminal ID “01aa” of the electronic whiteboard1aand the destination names corresponding to the terminal IDs.

Then, the communication unit51reads, via the storing and reading unit59, data such as the data of a destination list frame and the data of icons representing operation states from the storage unit5000, for example (step S19). The communication unit51further transmits the destination list information to the electronic whiteboard1a(step S20). The destination list information includes the read destination list frame and icons and the terminal IDs and the destination names extracted at step S18. That is, the destination list information includes the destination list frame, the icons, the terminal IDs, and the destination names. Then, in the electronic whiteboard1a, the communication unit11areceives the destination list information, and the storing and reading unit19astores the received destination list information in the storage unit1000a(step S21).

In the present embodiment, the communication management apparatus5thus centrally manages the destination list information of all electronic whiteboards1, instead of each of the electronic whiteboards1managing the destination list information. In the communication system10of the present embodiment, therefore, the communication management apparatus5centrally responds to changes such as the addition of a new electronic whiteboard1to the communication system10, the replacement of any of the electronic whiteboards1in the communication system10with a new electronic whiteboard1, and a change in the appearance of the destination list frame. Consequently, the electronic whiteboards1are saved from a process of changing the destination list information.

The terminal managing unit54further performs a search through the terminal management table (seeFIG. 10) with search keys set to the terminal IDs of the destination candidate terminals extracted at step S18, to thereby read from the terminal management table the respective operation states corresponding to the terminal IDs of the destination candidate terminals. Thereby, the terminal managing unit54acquires the respective operation states of the electronic whiteboards1corresponding to the terminal IDs of the destination candidate terminals (step S22).

The communication unit51then transmits terminal state information to the electronic whiteboard1avia the communication network100(step S23). The terminal state information includes the terminal IDs used as the search keys at step S22and the respective operation states of the destination candidate terminals corresponding to the terminal IDs.

Then, the storing and reading unit19aof the electronic whiteboard1asequentially stores, in the storage unit1000a, the items of the terminal state information received from the communication management apparatus5(step S24). The electronic whiteboard1athereby receives the terminal state information of the electronic whiteboards1as the destination candidate terminals, and thus acquires the current operation states of the destination candidate terminals, i.e., the other electronic whiteboards1such as the electronic whiteboard1bcapable of communicating with the electronic whiteboard1a.

Then, based on the destination list information and the terminal state information stored in the storage unit1000a, the display control unit14agenerates the destination list reflecting the states of the destination candidate terminals. Further, with the generated destination list, the display control unit14acontrols the display180of the electronic whiteboard1ato display a destination list screen800as illustrated inFIG. 16(step S25). The destination list screen800displays, for each of the destination candidate terminals, the icon representing the operation state of the destination candidate terminal and the terminal ID and the destination name of the destination candidate terminal. In the example ofFIG. 16, the icon representing the operation state of the destination candidate terminal is displayed as an “OFF-LINE” icon in the first row of the destination list screen800and as an “ON-LINE (COMMUNICATION AVAILABLE)” icon in the second row and a lower row of the destination list screen800.

In the communication management apparatus5, the terminal managing unit54performs a search through the destination list management table (seeFIG. 11) based on the terminal ID “01aa” of the electronic whiteboard1a, to thereby extract from the destination list management table the terminal IDs of the other electronic whiteboards1in which the electronic whiteboard1ais registered as a destination candidate terminal (step S26). In the destination list management table inFIG. 11, terminal IDs “01ba,” “01ca,” and “01da” are among the terminal IDs of the other electronic whiteboards1read in this step.

The terminal managing unit54then performs a search through the terminal management table (seeFIG. 10) based on the terminal ID “01aa” of the electronic whiteboard1a, to thereby acquire the operation state of the electronic whiteboard1afrom the terminal management table (step S27).

Then, the communication unit51transmits terminal state information to the electronic whiteboards1corresponding to the terminal IDs extracted at step S26and corresponding to the operation state “ON-LINE” in the terminal management table (seeFIG. 10) (step S28). The terminal state information in this case includes the terminal ID “01aa” of the electronic whiteboard1aand the operation state “ON-LINE” of the electronic whiteboard1aacquired at step S27. When transmitting the terminal state information to the other electronic whiteboards1such as the electronic whiteboard1b, the communication unit51refers to the IP addresses of the electronic whiteboards1managed in the terminal management table (seeFIG. 10) based on the terminal IDs of the electronic whiteboards1. Thereby, the communication unit51transmits the terminal ID “01aa” and the operation state “ON-LINE” of the electronic whiteboard1ato each of the other electronic whiteboards1capable of communicating with the electronic whiteboard1a, which is a destination candidate terminal for the other electronic whiteboards1. Consequently, each of the destination candidate terminals (i.e., the other electronic whiteboards1such as the electronic whiteboard1b) also displays, on the display180thereof, the state of the electronic whiteboard1a(i.e., a destination candidate terminal for the other electronic whiteboards1such as the electronic whiteboard1b) (step S29).

Process of Starting Remote Communication

With reference toFIG. 17, a description will be given of a process in which the electronic whiteboard1astarts remote communication with the electronic whiteboard1b.

FIG. 17is a sequence diagram illustrating an example of a process of starting remote communication. When the user UA1of the electronic whiteboard1a(i.e., the starting terminal) selects the electronic whiteboard1b(i.e., the destination terminal) by pressing a field in the destination list screen800ofFIG. 16corresponding to the terminal ID “01ba” of the electronic whiteboard1b, the receiving unit12areceives the user request to start the communication with the electronic whiteboard1b(i.e., the destination terminal) (step S31). Then, the communication unit11aof the electronic whiteboard1a(i.e., the starting terminal) transmits the start request information to the communication management apparatus5(step S32). The start request information, which represents the request to start the communication, includes the terminal ID “01aa” of the electronic whiteboard1a(i.e., the starting terminal) and the terminal ID “01ba” of the electronic whiteboard1b(i.e., the destination terminal). Then, the communication unit51of the communication management apparatus5receives the start request information and the IP address of the electronic whiteboard1a(i.e., the starting terminal) as the source of the start request information.

Then, the terminal managing unit54updates the terminal management table (seeFIG. 10) based on the terminal ID “01aa” of the electronic whiteboard1a(i.e., the starting terminal) and the terminal ID “01ba” of the electronic whiteboard1b(i.e., the destination terminal) included in the start request information (step S33). For example, based on the terminal ID “01aa” of the electronic whiteboard1a(i.e., the starting terminal) and the terminal ID “01ba” of the electronic whiteboard1b(i.e., the destination terminal) included in the start request information, the terminal managing unit54updates the respective operation states corresponding to the terminal IDs “01aa” and “01ba” in the terminal management table to “ON-LINE (COMMUNICATING).” The information of the electronic whiteboard1a(i.e., the starting terminal) and the electronic whiteboard1b(i.e., the destination terminal) managed in the terminal management table in this state indicates that the electronic whiteboards1aand1bhave not participated in a session yet but are in a communicating state. That is, the electronic whiteboard1a(i.e., the starting terminal) and the electronic whiteboard1b(i.e., the destination terminal) have not started a video conference yet, but are in the communicating state. Therefore, if a user of another electronic whiteboard1(e.g., a user of another electronic whiteboard1intending to join the communication) attempts to communicate with the electronic whiteboard1a(i.e., the starting terminal) or the electronic whiteboard1b(i.e., the destination terminal) in this state, the another electronic whiteboard1outputs a notification sound or displays a message to notify the user that the electronic whiteboard1aor1bis in the communicating state.

A description will be given of a process of executing a session to select the relay apparatus3to be used.

The session managing unit55of the communication management apparatus5first generates the session ID for identifying the session to be held between the electronic whiteboards1aand1b(step S34). It is assumed here that the session ID “se01” is generated.

The session managing unit55then stores and manages the session ID “se01” generated at step S34, the terminal ID “01aa” of the electronic whiteboard1a(i.e., the starting terminal), and the terminal ID “01ba” of the electronic whiteboard1b(i.e., the destination terminal) in the session management table (seeFIG. 12) in association with each other (step S35).

Then, the relay apparatus managing unit56selects the relay apparatus3that relays the session between the electronic whiteboard1a(i.e., the starting terminal) and the electronic whiteboard1b(i.e., the destination terminal) (step S36). For example, the relay apparatus managing unit56selects, from the relay apparatuses3with relay apparatus IDs corresponding to the operation state “ON-LINE” in the relay apparatus management table (seeFIG. 13), the relay apparatus3having an IP address close to the IP address of the electronic whiteboard1a(i.e., the starting terminal) in the terminal management table (seeFIG. 10). It is assumed in the following description that the relay apparatus3with the relay apparatus ID “111a” has been selected.

After the completion of the process of step S36to select the relay apparatus3, the communication unit51of the communication management apparatus5transmits relay apparatus selection information to the electronic whiteboard1a(i.e., the starting terminal) (step S37-1). The relay apparatus selection information includes the IP address of the relay apparatus3selected at step S36and the session ID “se01” generated at step S34. Then, the electronic whiteboard1a(i.e., the starting terminal) acquires the IP address of the communication management apparatus5as the source of the relay apparatus selection information.

The communication unit51of the communication management apparatus5further transmits relay apparatus selection information to the electronic whiteboard1b(i.e., the destination terminal) (step S37-2). The relay apparatus selection information includes the IP address of the relay apparatus3selected at step S36, the terminal ID “01aa” of the electronic whiteboard1a(i.e., the starting terminal), and the session ID “se01” generated at step S34. In the execution of the session with the session ID “se01,” therefore, the electronic whiteboard1b(i.e., the destination terminal) acquires the IP address of the communication management apparatus5as the source of the relay apparatus selection information.

Then, the communication unit11aof the electronic whiteboard1a(i.e., the starting terminal) transmits reception completion information to the communication management apparatus5(step S38-1). The reception completion information indicates that the relay apparatus selection information transmitted in the process of step S37-1has been received. The reception completion information includes the session ID transmitted in the process of step S37-1. Then, the communication management apparatus5acquires a message that the transmission of the relay apparatus selection information executed in a particular session with the session ID “se01” has been completed.

The electronic whiteboard1b(i.e., the destination terminal) similarly transmits, to the communication management apparatus5, reception completion information indicating that the relay apparatus selection information transmitted in the process of step S37-2has been received (step S38-2). In this case, too, the communication management apparatus5acquires a message that the transmission of the relay apparatus selection information executed in the particular session with the session ID “se01” has been completed.

As described above, the electronic whiteboards1aand1btransmit and receive the site image data and the audio data to and from each other via the relay apparatus3selected at step S36, to thereby have the video conference.

Process of Sharing Data between Electronic Whiteboards

A process of sharing data between the electronic whiteboards1will be described withFIGS. 18 to 22D.

FIGS. 18 to 20are sequence diagrams illustrating an example of the process of sharing data between the electronic whiteboards1in the communication system10of the embodiment.

The following description will be given of an example in which the electronic whiteboards1aand1bshare the background image data and the stroke data such that the background image displayed on the electronic whiteboard1aand the stroke image input to the electronic whiteboard1aare also displayed on the electronic whiteboard1b.

FIG. 21Ais a diagram illustrating an example of a screen displayed on the electronic whiteboard1a.FIG. 21Bis a diagram illustrating an example of a screen displayed on the electronic whiteboard1b.FIG. 21Cis a diagram illustrating another example of the screen displayed on the electronic whiteboard1a.FIG. 21Dis a diagram illustrating another example of the screen displayed on the electronic whiteboard1b.

In the electronic whiteboard1aat the site A, the display control unit14acontrols the display180to display the screen illustrated inFIG. 21A. The screen ofFIG. 21Adisplays a background image dA, an image v2of the site B, and a share button b1that is pressed to share the background image dA. If the user UA1of the electronic whiteboard1apresses the share button b1in this state, the receiving unit12areceives a background image sharing process requested by the user UA1(step S51). Then, the communication unit11atransmits an upload request to the image storing apparatus7to request upload of the image data of the background image dA (hereinafter referred to as the background image data) (step S52). In this case, the communication unit11atransmits the background image data to a predetermined URL for uploading data (e.g., http://qwe.jp/upload) Then, the communication unit71of the image storing apparatus7receives the upload request transmitted from the electronic whiteboard1a.

The storing and reading unit79of the image storing apparatus7stores the background image data received at step S52in the storage unit7000(step S53). Then, the generation unit73of the image storing apparatus7generates a URL representing the storage location of the stored background image data (step S54). The URL is an example of storage location information. The storage location information also includes a uniform resource identifier (URI). The execution order of steps S53and S54may be reversed. Further, steps S53and S54may be executed in parallel. Thereby, the electronic whiteboard1ais able to start uploading the background image data.

The communication unit71of the image storing apparatus7then transmits, to the electronic whiteboard1a, the URL of the background image data generated by the generation unit73(step S55). Then, the communication unit11aof the electronic whiteboard1areceives the URL of the background image data transmitted from the image storing apparatus7.

Then, the display screen managing unit21aof the electronic whiteboard1aupdates the page information stored in the display screen management DB1001a(step S56). In this case, the display screen managing unit21astores the page data (e.g., Page P1) corresponding to the background image data to be uploaded to the image storing apparatus7.

Then, the communication unit11atransmits an upload start notification to the communication management apparatus5to notify that the upload of the background image data has started (step S57). The upload start notification includes the page ID of the page corresponding to the background image data to be uploaded and the URL received at step S55. Then, the communication unit51of the communication management apparatus5receives the upload start notification transmitted from the electronic whiteboard1a.

Based on the various information received at step S57, the shared data managing unit57of the communication management apparatus5updates the page information stored in the shared data management DB5006(seeFIG. 14) (step S58). Specifically, the shared data managing unit57searches through the shared data management table for the information “body” associated with the page ID received at step S57, and describes the URL received at step S57in the information “body.” The shared data managing unit57further assigns a global ID (gid) to the page (i.e., page data) to be stored in the shared data management DB5006.

The communication unit51of the communication management apparatus5forwards the upload start notification received at step S57to the electronic whiteboard1b, which is having the remote conference with the electronic whiteboard1a(step S59). The communication unit111bof the electronic whiteboard1bthen receives the upload start notification. Then, the display control unit14bof the electronic whiteboard1bcontrols the display180of the electronic whiteboard1bto display the screen as illustrated inFIG. 21B(step S60). Before this step, the screen ofFIG. 21Bdisplays an image v1of the site A and a share button b2that is pressed to share data. In response to receipt of the upload start notification at step S59, the electronic whiteboard1boutputs data for displaying an hourglass icon c2that visualizes the passage of time taken to download the background image data.

The passage of time may be expressed by a still image of an hourglass or a moving image of an hourglass with moving sand. The hourglass icon c2is an example of schedule information indicating that image data is scheduled to be downloaded. Other examples of the schedule information include an icon other than the hourglass icon (e.g., a clock icon), a character (and/or a symbol) replacing the icon, and a combination of an icon and a character (and/or a symbol). Further, the schedule information may be notified by sound. In this case, the schedule information is output as sound from the speaker150by the image and audio processing unit13a, not by the display control unit14a.

Then, as illustrated inFIG. 19, the user UA1at the site A moves the electronic pen190or the hand H in contact with the display180of the electronic whiteboard1a, and the receiving unit12areceives the input of the stroke (i.e., trajectory) of the movement (step S61). Based on the input stroke, the image processing unit16athen generates stroke data (e.g., the data of coordinates (x, y)) for displaying a stroke image on the two-dimensional display180(step S62). Then, the display control unit14acontrols the display180of the electronic whiteboard1ato display the stroke image (step S63). Thereby, a stroke image stA is displayed, as illustrated inFIG. 21C.

Then, the communication unit11atransmits, to the communication management apparatus5, the stroke data generated at step S62to reproduce the stroke image (step S64). Then, the communication unit51of the communication management apparatus5receives the stroke data transmitted from the electronic whiteboard1a.

The shared data managing unit57of the communication management apparatus5then stores the stroke data received at step S64in the shared data management DB5006(seeFIG. 14) (step S65). In this case, the shared data managing unit57stores the items of the stroke data received by the communication unit51in the shared data management table sequentially (i.e., in the generation order of the items of the stroke data) in association with the meeting ID (i.e., the session ID) identifying the currently held remote conference, as illustrated inFIG. 14. The shared data managing unit57further assigns a global ID (gid) to the stroke data received at step S64.

Then, the communication unit51transmits the stroke data stored at step S65and the global ID (gid) assigned by the shared data managing unit57to the electronic whiteboard1a(step S66). Then, the electronic whiteboard1areceives the stroke data and the global ID (gid) transmitted from the communication management apparatus5. Thereby, the electronic whiteboard1ais notified that the stroke data has been received, and is also notified of the assigned global ID (gid).

The communication unit51further transmits the stroke data and global ID (gid) to the electronic whiteboard1b, with which the electronic whiteboard1ais having the remote conference (step S67). Then, the communication unit11bof the electronic whiteboard1breceives the stroke data and the global ID (gid) transmitted from the communication management apparatus5. The image processing unit16bof the electronic whiteboard1bthen generates the stroke image based on the stroke data received by the communication unit11b(step S68). Then, the display control unit14bof the electronic whiteboard1bcontrols the display180of the electronic whiteboard1bto display the stroke image generated at step S68(step S69). Thereby, the same stroke image stA as that displayed on the electronic whiteboard1a(i.e., the counterpart electronic whiteboard1of the electronic whiteboard1b) is displayed on the electronic whiteboard1b, as illustrated inFIG. 21D. The communication system10thus enables the stroke image rendered on the electronic whiteboard1ato be shared by the electronic whiteboard1b.

Then, as illustrated inFIG. 20, the communication unit71of the image storing apparatus7transmits an upload completion notification to the electronic whiteboard1ato notify that the upload of the background image data has been completed (step S71). The upload completion notification includes the URL representing the storage location of the uploaded background image data. Then, the communication unit11aof the electronic whiteboard1areceives the upload completion notification transmitted from the image storing apparatus7.

The communication unit11aof the electronic whiteboard1aforwards the upload completion notification received at step S71to the communication management apparatus5(step S72). Then, the communication unit51of the communication management apparatus5forwards the upload completion notification received at step S72to the electronic whiteboard1b(step S73). Then, the communication unit11bof the electronic whiteboard1breceives the upload completion notification transmitted from the communication management apparatus5.

The communication unit11bof the electronic whiteboard1btransmits a download request to the image storing apparatus7to request download of the background image data from the URL received at step S73, i.e., the URL representing the storage location in the image storing apparatus7(step S74). Then, the communication unit71of the image storing apparatus7receives the download request transmitted from the electronic whiteboard1b.

Then, based on the URL included in the download request received by the communication unit71, the storing and reading unit79of the image storing apparatus7reads the requested background image data from the storage unit7000(step S75). The communication unit71then transmits the requested background image data to the electronic whiteboard1bas the source of the download request. Then, the communication unit11bof the electronic whiteboard1bdownloads (i.e., receives) the background image data (step S76). Then, the display control unit14bof the electronic whiteboard1bcontrols the display180of the electronic whiteboard1bto display the background image based on the background image data downloaded (i.e., received) by the communication unit11b(step S77). Specifically, the display control unit14bdisplays, on the screen illustrated inFIG. 21D, the same background image dA as that illustrated inFIG. 21C, and stops displaying the hourglass icon c2.

Thereby, the electronic whiteboards1aand1bshare the stroke image and the background image displayed on the display180. The background image data of the background image displayed on one of the electronic whiteboards1aand1b(e.g., the electronic whiteboard1a) is uploaded to the image storing apparatus7to be downloadable to the other one of the electronic whiteboards1aand1b(e.g., the electronic whiteboard1b). Further, the stroke data of the stroke rendered on one of the electronic whiteboards1aand1b(e.g., the electronic whiteboard1a) is shared by the other one of the electronic whiteboards1aand1b(e.g., the electronic whiteboard1b) via the communication management apparatus5. The electronic whiteboards1aand1balso share, via the communication management apparatus5, the operation information (seeFIG. 8) of the operation such as generating the rendering screen (i.e., the page) to be displayed on the electronic whiteboards1aand1bor changing the rendering screen (i.e., the page). The process of steps S51to S60to start uploading the background image data to the image storing apparatus7from the electronic whiteboard1aand the stroke data sharing process of steps S61to S69may be reversed in order, or may be executed in parallel.

The rendering screen shared by the electronic whiteboards1aand1bwill be described withFIGS. 22A, 22B, 22C, and 22D.

A rendering screen200aillustrated inFIG. 22Adisplays a background image dl and a stroke image st1. A rendering screen200billustrated inFIG. 22Bdisplays a stroke image st2. A rendering screen200cillustrated inFIG. 22Cdisplays a background image d3and a stroke image st3. A rendering screen200dillustrated inFIG. 22Ddisplays a background image d4and a stroke image st4.

As described above, the electronic whiteboards1aand1bshare a plurality of rendering screens such as those illustrated inFIGS. 22A to 22D. Further, the electronic whiteboards1aand1bmanage, as rendering screen information (i.e., the display screen information) illustrated inFIG. 7, the screen data (i.e., the background image data and the stroke data) forming the rendering screen displayed on the display180. For example, the rendering screen200aillustrated inFIG. 22Ais a rendering screen corresponding to the screen data of the page P1, and the rendering screen200billustrated inFIG. 22Bis a rendering screen corresponding to the screen data of the page P2. Further, for example, the rendering screen200cillustrated inFIG. 22Cis a rendering screen corresponding to the screen data of the page P3, and the rendering screen200dillustrated inFIG. 22Dis a rendering screen corresponding to the screen data of the page P4. The rendering screens200ato200dillustrated inFIGS. 22A to 22Dare examples of the display screen displayed on the display180.

As described above, when the electronic whiteboard1atransmits the background image data and then the stroke data of the stroke image, the stroke data reaches the counterpart electronic whiteboard1bbefore the background image data reaches the electronic whiteboard1b, since the data size of the stroke data is substantially smaller than that of the image data. In the present embodiment, therefore, when the electronic whiteboard1auploads the image data, the communication management apparatus5receives the upload start notification from the electronic whiteboard1a, and forwards (i.e., transmits) the upload start notification to the electronic whiteboard1b, as in step S59inFIG. 18. Then, the electronic whiteboard1boutputs the data for displaying the hourglass icon c2, as illustrated inFIG. 21B, to visually notify that the background image data is going to be downloaded, as in step S60inFIG. 18. When the display of the stroke image is in progress, therefore, the users UB1and UB2of the electronic whiteboard1bat the site B realize that there is an incoming background image (e.g., the image of a material) to be transmitted later. With this realization, the users UB1and UB2are able to smoothly communicate with the user UA1of the electronic whiteboard1aat the site A.

First Modified Example of Embodiment

A process of the communication system10according to a first modified example of the embodiment will be described withFIG. 23.

In the communication system10of the first modified example, the electronic whiteboard1bstarts downloading the background image data from the image storing apparatus7before the upload from the electronic whiteboard1ais completed.FIG. 23is a sequence diagram illustrating an example of a data sharing process between the electronic whiteboards1in the communication system10according to the first modified example of the embodiment. The process of starting the upload of the background image data to the image storing apparatus7from the electronic whiteboard1aand the stroke data sharing process in the first modified example are similar to those illustrated inFIGS. 18 and 19, and thus description thereof will be omitted.

The communication unit11bof the electronic whiteboard1btransmits a download request to the image storing apparatus7to request download of the background image data from the URL transmitted from the communication management apparatus5, i.e., the URL representing the storage location in the image storing apparatus7(step S91). Then, the communication unit71of the image storing apparatus7receives the download request transmitted from the electronic whiteboard1b.

Then, based on the URL included in the download request received by the communication unit71, the storing and reading unit79of the image storing apparatus7reads the requested background image data from the storage unit7000(step S92). The communication unit71then transmits the read background image data to the electronic whiteboard1bas the source of the download request. Then, the communication unit11bof the electronic whiteboard1bstarts downloading (i.e., receiving) the background image data (step S93).

In parallel with the transmission of the background image data to the electronic whiteboard1b, the image storing apparatus7executes the process of uploading the background image data from the electronic whiteboard1a(i.e., step S52inFIG. 18) and the process of storing the background image data (i.e., step S53inFIG. 18). After the background image data has been uploaded to the image storing apparatus7, the communication unit71of the image storing apparatus7transmits the upload completion notification to the electronic whiteboard1ato notify that the upload of the background image data has been completed (step S94).

The electronic whiteboard1bcontinues to download the background image data from the image storing apparatus7. If the download process by the electronic whiteboard1bproceeds faster than the uploading process from the electronic whiteboard1a, the image storing apparatus7causes the electronic whiteboard1ato stand by, for example, and sequentially transmits the uploaded items of the background image data to the electronic whiteboard1b. That is, the image storing apparatus7transmits the background image data as necessary from the storage unit7000to the electronic whiteboard1b, while receiving the uploaded background image data from the electronic whiteboard1a.

When the download of the background image data from the image storing apparatus7is completed (step S95), the display control unit14bof the electronic whiteboard1bcontrols the display180of the electronic whiteboard1bto display the background image based on the downloaded (i.e., received) background image data (step S96).

As described above, the electronic whiteboard1bat the site B is able to start downloading the background image data without waiting for the completion of the upload of the background image data from the electronic whiteboard1aat the site A. Therefore, the time taken for the electronic whiteboard1bto share the background image data is reduced as compared with a case in which the electronic whiteboard1bstarts downloading the background image data after the upload of the background image data is completed. Consequently, the communication system10of the first modified example further contributes to the smooth communication between the user UA1of the electronic whiteboard1aand the users UB1and UB2of the electronic whiteboard1b.

Second Modified Example of Embodiment

A process of the communication system10according to a second modified example of the embodiment will be described withFIGS. 24 to 26. In the communication system10of the second modified example, the electronic whiteboard1astarts uploading the background image data to the image storing apparatus7in response to receipt of the download request from another electronic whiteboard1(e.g., the electronic whiteboard1b). That is, in the communication system10of the second modified example, the electronic whiteboard1adoes not upload the background image data to the image storing apparatus7unless the request to download the background image is transmitted to the electronic whiteboard1afrom the electronic whiteboard1b.

FIGS. 24 to 26are sequence diagrams illustrating an example of a data sharing process between the electronic whiteboards1in the communication system10of the second modified example of the embodiment. The following description will be given of an example in which the image data of the background image and the stroke data of the stroke image are shared by the electronic whiteboards1aand1bsuch that the background image displayed on the electronic whiteboard1aand the stroke image input to the electronic whiteboard1aare also displayed on the electronic whiteboard1b.

In the electronic whiteboard1aat the site A, the display control unit14acontrols the displays180to display the screen illustrated inFIG. 21A. If the user UA1of the electronic whiteboard1apresses the share button b1in this state, the receiving unit12areceives the background image sharing process requested by the user UA1(step S101). The communication unit11athen transmits, to the image storing apparatus7, request information that requests the URL representing the storage location of the background image data (step S102). Then, the communication unit71of the image storing apparatus7receives the request information transmitted from the electronic whiteboard1a.

Then, the generation unit73of the image storing apparatus7generates the URL representing the storage location of the background image data (step S103). The communication unit71then transmits, to the electronic whiteboard1a, the URL of the background image data generated by the generation unit73(step S104). Then, the communication unit11aof the electronic whiteboard1areceives the URL of the background image data.

Then, the display screen managing unit21aof the electronic whiteboard1aupdates the page information stored in the display screen management DB1001a(step S105). In this case, the display screen managing unit21astores the page data corresponding to the background image data to be uploaded to the image storing apparatus7.

Then, the communication unit11atransmits notification information to the communication management apparatus5to notify the communication management apparatus5of the URL of the background image data (step S106). The notification information includes the page ID of the page corresponding to the background image data to be uploaded and the URL received at step S104. Then, the communication unit51of the communication management apparatus5receives the notification information transmitted from the electronic whiteboard1a.

Based on the various information received at step S106, the shared data managing unit57of the communication management apparatus5updates the page information stored in the shared data management DB5006(step S107). Specifically, the shared data managing unit57searches through the shared data management table for the information “body” associated with the page ID received at step S106, and describes the URL received at step S106in the information “body.” The shared data managing unit57further assigns a global ID (gid) to the page (i.e., page data) to be stored in the shared data management DB5006.

The communication unit51of the communication management apparatus5forwards the notification information received at step S106to the electronic whiteboard1b(step S108). Then, the communication unit11bof the electronic whiteboard1breceives the notification information transmitted from the communication management apparatus5. The display control unit14bof the electronic whiteboard1bthen controls the display180of the electronic whiteboard1bto displays the screen as illustrated inFIG. 21B(step S109).

The stroke image sharing process of steps S111to S119inFIG. 25is similar to that of steps S61to S69inFIG. 19, and thus description thereof will be omitted. Then, as illustrated inFIG. 26, in response to receipt of a certain operation performed on the electronic whiteboard1bby the user UB1or UB2, the communication unit11bof the electronic whiteboard1btransmits, to the communication management apparatus5, a download request representing the request to download the background image data (step S121). The download request includes the URL representing the storage location of the background image data received at step S108. Then, the communication unit51of the communication management apparatus5forwards the download request received at step S121to the electronic whiteboard1a(step S122). Then, the communication unit11aof the electronic whiteboard1areceives the download request transmitted from the communication management apparatus5.

The communication unit11aof the electronic whiteboard1athen transmits, to the image storing apparatus7, an upload request representing the request to upload the background image data to the URL transmitted from the communication management apparatus5, i.e., the URL representing the storage location in the image storing apparatus7(step S123). Then, the communication unit71of the image storing apparatus7receives the upload request transmitted from the electronic whiteboard1a. The storing and reading unit79of the image storing apparatus7then stores the background image data received at step S123in the storage unit700(step S124). Then, the electronic whiteboard1astarts uploading the background image data to the URL. After the upload of the background image data is completed, the communication unit71of the image storing apparatus7transmits an upload completion notification to the electronic whiteboard1a(step S125). The following processes of steps S126to S131inFIG. 26are similar to those of steps S72to S77inFIG. 20, and thus description thereof will be omitted.

As described above, the electronic whiteboard1aat the site A does not upload the background image data unless receiving the download request from the electronic whiteboard1bat the site B. In the communication system10of the second modified example, therefore, the time of transmitting the background image data is adjusted based on the time of transmitting the download request from the user UB1or UB2at the site B in consideration of factors such as the progress of the remote conference and the band condition of the communication network100, for example.

As described above, an electronic whiteboard of an embodiment of the present invention is the electronic whiteboard1b(an example of a communication terminal) that shares data with the electronic whiteboard1a(an example of another communication terminal). The electronic whiteboard1breceives, from the communication management apparatus5that manages the data shared with the electronic whiteboard1a, the URL (an example of storage location information) representing the storage location of the background image data transmitted from the electronic whiteboard1a. The electronic whiteboard1bthen transmits the download request to the image storing apparatus7corresponding to the URL to request download of the background image data. The electronic whiteboard1bthen starts downloading the background image data from the image storing apparatus7, and receives, from the communication management apparatus5, the stroke data of the stroke image rendered on the electronic whiteboard1a. Thereby, the electronic whiteboard1bis capable of concurrently executing the sharing of the stroke image and the sharing of the background image with the electronic whiteboard1a, contributing to the smooth communication between the users UB1and UB2of the electronic whiteboard1band the user UA1of the electronic whiteboard1a.

Further, in response to receipt of the URL (an example of the storage location information) transmitted from the electronic whiteboard1a(an example of the another communication terminal), the electronic whiteboard1b(an example of the communication terminal) of the embodiment may control the display180(an example of a display) to display the hourglass icon c2(an example of schedule information) indicating that the background image data is scheduled to be downloaded. When the display of the stroke image is in progress, therefore, the users UB1and UB2of the electronic whiteboard1brealize that there is an incoming background image (e.g., the image of a material) to be transmitted later. With this realization, the users UB1and UB2of the electronic whiteboard1bare able to have smooth communication with the user UA1of the electronic whiteboard1a.

Further, in response to receipt of the URL (an example of the storage location information) transmitted from the electronic whiteboard1a(an example of the another communication terminal), the electronic whiteboard1b(an example of the communication terminal) of the embodiment may transmit the download request to the image storing apparatus7to request download of the background image data. Thereby, the electronic whiteboard1bstarts downloading the background image data without waiting for the completion of the upload of the background image data from the electronic whiteboard1a, reducing the time taken for the electronic whiteboard1bto share the background image data. Consequently, the electronic whiteboard1bfurther contributes to the smooth communication between the users UB1and UB2of the electronic whiteboard1band the user UA1of the electronic whiteboard1a.

Further, the electronic whiteboard1b(an example of the communication terminal) of the embodiment may receive, from the communication management apparatus5, the completion notification notifying the completion of the upload of the background image data. Then, in response to receipt of the completion notification, the electronic whiteboard1bmay transmit the download request to the image storing apparatus7to request download of the background image data. By adjusting the time of transmitting the download request, therefore, the user UB1or UB2of the electronic whiteboard1bis able to adjust the time of transmitting the background image data from the electronic whiteboard1a.

A communication system of an embodiment of the present invention is the communication system10including the electronic whiteboard1a(an example of a first communication terminal) and the electronic whiteboard1b(an example of a second communication terminal) that shares data with the electronic whiteboard1a. The electronic whiteboard1atransmits the URL (an example of storage location information) representing the storage location of the background image data to the communication management apparatus5that manages the data shared with the electronic whiteboard1b. Further, the electronic whiteboard1breceives the URL from the communication management apparatus5, and transmits the download request to the image storing apparatus7corresponding to the URL to request download of the background image data. Then, the electronic whiteboard1bstarts downloading the background image data from the image storing apparatus7, and receives, from the communication management apparatus5, the stroke data of the stroke image rendered on the electronic whiteboard1a. Thereby, the communication system10of the embodiment enables the sharing of the stroke image and the sharing of the background image to be concurrently executed between the electronic whiteboards1aand1b. Consequently, the communication system10contributes to smooth communication between the users UB1and UB2of the electronic whiteboard1band the user UA1of the electronic whiteboard1a.

The communication system10of the embodiment may further include the image storing apparatus7that stores the background image data shared by the electronic whiteboard1a(an example of the first communication terminal) and the electronic whiteboard1b(an example of the second communication terminal). The electronic whiteboard1amay receive the download request transmitted from the electronic whiteboard1b. In response to receipt of the download request, the electronic whiteboard1amay transmit the upload request to the image storing apparatus7corresponding to the URL (an example of the storage location information) representing the storage location of the background image data to request upload of the background image data. Further, the image storing apparatus7may receive the upload request transmitted from the electronic whiteboard1a, generate the URL based on the received upload request, and transmit the generated URL to the electronic whiteboard1a. Then, the electronic whiteboard1amay transmit, to the communication management apparatus5, the URL transmitted from the image storing apparatus7. The communication system10of the embodiment, therefore, enables the user UB1or UB2of the electronic whiteboard1b, which is going to download the background image data, to adjust the time of transmitting the background image data from the electronic whiteboard1a.

The various tables of the embodiment described above may be generated as a result of machine learning. The mutually associated data items in each of the tables may be categorized by machine learning to obviate the need for the tables. Herein, machine learning refers to a technology for causing a computer to acquire learning ability similar to human learning ability. According to the technology, the computer autonomously generates, from previously learned data, algorithms for making decisions such as data identification, and makes predictions by applying the algorithms to new data. The learning method for machine learning may be any of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning, or may be a learning method combining two or more of these learning methods. The learning method for machine learning is not limited to a particular method.

Each of the functions of the described embodiments may be implemented by one or more processing circuits or circuitry. Circuitry includes a programmed processor, as a processor includes circuitry. A processing circuit also includes devices such as an application specific integrated circuit (ASIC), digital signal processor (DSP), field programmable gate array (FPGA), and conventional circuit components arranged to perform the recited functions. Further, the above-described steps are not limited to the order disclosed herein.