Patent ID: 12192677

DETAILED DESCRIPTION

The following describes an image processing device22according to an embodiment of the present disclosure with reference to the drawings.FIG.1is a schematic diagram illustrating an overall configuration of a remote conference system10including the image processing device22according to an embodiment of the present disclosure. It should be noted that the image processing device22is not illustrated inFIG.1for simplification of the drawing.FIG.2is a system configuration diagram illustrating an overall configuration of the remote conference system10including the image processing device22according to the embodiment of the present disclosure.FIG.3illustrates an example of a screen50displayed on a monitor13of each remote conference client device11in the remote conference system10.

As illustrated inFIGS.1and2, the image processing device22is provided in the remote conference system10, for example. The remote conference system10is a communication conference system in which a plurality of participants perform one-way or two-way communication remotely. As illustrated inFIG.1, the participants operate remote conference client devices11from, for example, their homes, their workplaces (e.g., their desks or conference rooms at their workplaces), or other places to participate in a remote conference. As illustrated inFIGS.1and2, the remote conference system10includes a plurality of remote conference client devices11and a remote conference server unit20.

The remote conference client devices11are communication devices assigned to the respective participants for participating in the remote conference. The remote conference client device11communicates with the remote conference server unit20, and may be a smart phone, a tablet, a desktop computer, a laptop computer, or a television receiver, for example. The remote conference client devices11each mainly includes a communication unit12, a monitor13(an example of a “display device”), a camera14, a microphone15, and a speaker16.

In each of the remote conference client devices11, the communication unit12communicates with the remote conference server unit20. The communication unit12transmits image data and audio data generated in the remote conference client device11thereof to the remote conference server unit20. The communication unit12receives image data and audio data generated in the remote conference server unit20. The communication unit12outputs the received image data to the monitor13. The communication unit12outputs the received audio data to the speaker16.

The monitor13displays image data generated in the remote conference client device11and image data generated in the remote conference server unit20(image processing device22).

The camera14includes a lens and a light-receiving element which are not illustrated, for example. The camera14captures participants, for example, in the remote conference, and generates image data. The image data generated by the camera14is output to monitor13and communication unit12.

The participant uses the microphone15to speak in the remote conference. When the participant speaks to the microphone15, an audio signal is output to the speaker16and the communication unit12.

The speaker16outputs audio data generated in the remote conference client device11and audio data received by the communication unit12.

The remote conference server unit20is a host control processor for hosting the remote conferences. As illustrated inFIG.2, the remote conference server unit20mainly includes a communication device21, an image processing device22, and a control device29. The control device29controls the remote conference server unit20.

The communication device21is communicably connected to the communication unit12of each remote conference client device11. The communication device21transmits and receives image data and audio data to and from the communication unit12of each remote conference client device11. In other words, the communication device21is a distribution mean for distributing individual images and sounds from the remote conference server unit20to each remote conference client device11. Communication between the communication device21and the communication unit12may be wired communication or wireless communication.

The image processing device22generates image data of the screen50displayed on the monitor13of each remote conference client device11. The image processing device22mainly includes an image forming section23, a controller24, and storage25.

The image forming section23generates a screen (image) displayed on the monitor13of each remote conference client device11based on image data and the like transmitted from each remote conference client device11. The image forming section23forms a screen50as illustrated inFIGS.3A and3B, for example. Specifically, the screen50inFIG.3Ais formed by the image forming section23at a predetermined time in the remote conference. The screen50inFIG.3Bis formed by the image forming section23after formation of the first screen50A. Hereinafter, the screen50inFIG.3Amay be referred to as a first screen50A. The screen50inFIG.3Bmay be referred to as a second screen50B.

As illustrated inFIGS.3A and3B, the screen50mainly includes a participant display area51, a conference material display area52, and a capture history list display area53, for example.

The participant display area51is a region for displaying each participant participating in the remote conference. The conference material display area52is a region for displaying either or both a material and a slide used in the remote conference. The capture history list display area53is a region for displaying a list of screen captures of the screen50. The screen captures displayed in the capture history list display area53are formed by the image forming section23. Specifically, each of the screen capture displayed in the capture history list display area53is formed based on a snapshot of the screen50acquired by the acquiring section27. In the capture history list display area53, the screen captures are sequentially displayed in correspondence with snapshots of the screens50successively acquired by the acquiring section27, for example.

The controller24performs overall control on the image processing device22. The controller24includes a detecting section26, an acquiring section27, and a generating section28.

The detecting section26detects a difference between the screens50displayed on the monitor13of each remote conference client device11. Specifically, the detecting section26detects a difference between the screens50based on the image data generated by the image forming section23. For example, the detecting section26extracts image data generated by the image forming section23at predetermined time intervals, and compares the extracted image data to detect the difference between the screens50. Specifically, the detecting section26compares the pixel values of the extracted screen50, thereby detecting the difference between the screens50. More specifically, as illustrated inFIGS.3A and3B, the detecting section26extracts the image data of the first screen50A and the image data of the second screen50B, and compares the extracted image data of the first screen50A with the extracted image data of the second screen50B, thereby detecting the difference between the first screen50A and the second screen50B. The detecting section26compares the pixel values of the first screen50A and the pixel values of the second screen50B to detect the difference between the first screen50A and the second screen50B, for example.

The detecting section26calculates an amount of change of the screen50from the detected difference between the screens50. Specifically, the detecting section26calculates an amount of change between the first screen50A and the second screen50B from the detected difference between the first screen50A and the second screen50B. The detecting section26monitors the amount of change of the screen50that changes as the remote conference progresses, based on the calculated amount of change of the screen50.

The detecting section26is set to detect, in a predetermined area that is arbitrarily determined within the screen50, a change which is a reference for detecting the difference between the screens50. For example, the detecting section26may be set to detect the change in the entirety of the screen50illustrated inFIGS.3A and3B. Alternatively, the detecting section26may be set to detect the change only in the conference material display area52within the screen50illustrated inFIGS.3A and3B. In the remote conference, the participants are displayed in the participant display area51of the screen50. Erroneous detection of the change position by the detecting section26can be prevented by excluding the participant display area51from the change detection target.

A threshold of the amount of change of the screen50, which is to be detected as the difference between the screens50by the detecting section26, is arbitrarily determined. Here, the threshold of the amount of change of the screen50is arbitrarily determined by a user or a manufacturer. More specifically, the threshold is a value based on the amount of change of the screen50at a time when the screen50changes significantly due to page switching or topic change in the remote conference.

Moreover, the detecting section26is set to extract image data of the screen50at arbitrary intervals (time). For example, the detecting section26extracts the screen50at intervals of one minute after a remote conference start. In this case, the detecting section26extracts the image data of the first screen50A one minute after the remote conference start, and further extracts the pixel data of the second screen50B one minute after the image data of the first screen50A is extracted (2 minutes after the remote conference start).

The acquiring section27acquires a snapshot of the screen50displayed on the monitor13. The acquiring section27acquires snapshots of the screens50at predetermined intervals. Specifically, the acquiring section27acquires a snapshot of the first screen50A after a predetermined time has elapsed since the remote conference started. The acquiring section27further acquires a snapshot of the second screen50B after the predetermined time has elapsed since the snapshot of the first screen50A has been acquired.

In a case in which the detecting section26is set to detect the change in an arbitrarily determined area, the acquiring section27acquires, for example, a snapshot of the arbitrarily determined area within the screen50(snapshot of the specific area within the screen50). For example, in a case in which the detecting section26is set to detect the change only in the meeting material display area52within the screen50, the acquiring section27acquires a snapshot of only the meeting material display area52within the screen50. In this manner, the acquiring section27acquires a snapshot of the specific area of the screen50, thereby achieving efficient acquisition of the snapshot of the screen50. Moreover, the acquired snapshot of the screen50can be easily stored, distributed, and printed as a document. In addition, since only the document of the specific area (e.g., the meeting material display area52) is generated, the user can efficiently review the meeting and confirm points that the user wants to know. It should be noted that the acquiring section27may acquire a snapshot of the screen50as a whole even in a case in which the detecting section26is set to detect the change in an arbitrarily determined area, for example.

The generating section28generates a document of the snapshot of the screen50acquired by the acquiring section27. Specifically, when the amount of change between the first screen50A and the second screen50B calculated by the detecting section26is equal to or greater than the threshold, the generating section28generates a snapshot document of the first screen50A. When the amount of change of the screen50calculated by the detecting section26is equal to or greater than the threshold, the generating section28determines that the screen50displayed on the monitor13is significantly changing (changes from the first screen50A to the second screen50B) due to page switching or topic change during the remote conference, and generates a snapshot document of the latest screen which is the screen50(the first screen50A) immediately before the change. As a result, the user can easily review the remote conference by accessing the snapshot document of the screen50(the first screen50A) generated by the generating section28.

The document generated by the generating section28can be acquired from each remote conference client device11, and can be printed by a printer (not illustrated) or the like as a snapshot of the screen50. The document generated by the generating section28is associated with a snapshot of the screen50generated by the image forming section23. The user may select one of the snapshots of the screen50displayed on the monitor13to display the selected snapshot of the screen50in an enlarged manner, or to print the corresponding screen50.

The storage25is flash memory, for example. The storage25stores therein programs executed by the controller24, data generated by execution of the programs by the controller24, and data transmitted from each remote conference client device11, and the like. The storage25stores the snapshot document of the screen50generated by the generating section28. It should be noted that the participants of the remote conference can obtain the snapshot document of the screen50stored in the storage25from each remote conference client device11. As a result, the participants can access the snapshot document of the screen50in the remote conference to efficiently obtain a snapshot of the data displayed on the screen50connected in the remote conference, and can save, distribute, and print the snapshot document.

The storage25also stores therein a capture history of the screen50generated by the image forming section23. Examples of the capture history of the screen50include serial numbers of screen captures of the screen50assigned from the remote conference start, the elapsed time from the remote conference start, and the registration date and time of each screen capture of the screen50.

The following describes an operation of the image processing device22.FIG.4is a flow chart illustrating an operation flow of the image processing device22.

As illustrated inFIG.4, upon a start of the remote conference, the controller24of the image processing device22measures the time from the remote conference start (Step S1). If the controller24determines that a predetermined time (e.g., one minute) has elapsed since the remote conference started (Yes in Step S1), the controller24causes the detecting section26to extract image data of the first screen50A displayed on the monitor13after the predetermined time has elapsed from the remote conference start (Step S2). Moreover, the controller24causes the acquiring section27to acquire a snapshot of the first screen50A (Step S3). Upon the acquiring section27acquiring the snapshot of the first screen50A (Step S3), the controller24measures the time from the acquisition of the snapshot of the first screen50A (Step S4). When the controller24determines that the predetermined time (e.g., 1 minute) has elapsed since the acquiring section27acquired the snapshot of the first screen50A (Yes in Step S4), the detecting section26extracts image data of a second screen50B displayed on the monitor13(Step S5). Thereafter, the acquiring section27acquires a snapshot of the second screen50B (Step S6).

Upon the acquiring section27acquiring the snapshot of the second screen50B (Step S6), the controller24causes the detecting section26to compare the extracted image data of the first screen50A with the extracted image data of the second screen50B to detect a difference between the first screen50A and the second screen50B. Upon the detecting section26detecting the difference between the first screen50A and the second screen50B (Step S7), the controller24causes the detecting section26to calculate an amount of change between the first screen50A and the second screen50B based on the difference between the first screen50A and the second screen50B.

Upon the detecting section26calculating the amount of change between the first screen50A and the second screen50B (Step S8), the controller24determines whether or not the calculated amount of change is equal to or greater than a threshold (Step S9). If the controller24determines that the calculated amount of change is smaller than the threshold (No in Step S9), the controller24measures the time after extraction of the image data of the second screen50B (Step S4). In other words, the controller24determines that no page has been switched or no topic has changed in the remote conference, and measures the time until the next image data of the screen50is extracted.

If the controller24determines that the calculated amount of change is equal to or greater than the threshold (Yes in Step S9), the controller24causes the generating section28to generate a snapshot document of the first screen50A (Step S10). In other words, the controller24determines that the page has been switched or topic has changed in the remote conference, and generates a snapshot document of a latest screen which is the first screen50A immediately before the screen50has changed significantly. After the generating section28generates the snapshot document of the first screen50A (Step S10), the image forming section23forms a screen capture of the first screen50A (Step S11).

When the screen capture of the first screen50A is formed (Step11), the controller24determines whether or not the remote conference is over based on reception or non-reception of a signal from each remote conference client device11(Step S12). If the controller24determines that the remote conference is not over (No in Step S11), the controller24measures the time after extraction of the image data of the second screen50B (Step S4). Furthermore, each time the controller24calculates an amount of change of the screen50exceeding the preset threshold (Yes in Step S9), the generating section28generates a snapshot document of the first screen50A (Step S10), and the image forming section23forms a screen capture of the first screen50A (step S11). If the controller24determines that the remote conference is over (Yes in step S12) by contrast, the operation of the image processing device22ends.

As described above, according to the embodiment of the present disclosure, the image processing device acquires a snapshot of the screen50(the first screen50A) displayed on the monitor13at a time when the screen50changes significantly (changes from the first screen50A to the second screen50B), and generates a document of the acquired snapshot. Accordingly, the user can easily search for the screen at a time when the page is switched or topic changes in the remote conference, and the like based on the snapshot document. The user can also obtain information that cannot be obtained from any materials, such as the conference progress, occasional notes, and chat screens of the participants. As a result, the user can effectively review the remote conference. Moreover, even in a case in which the user has to leave for a while in the remote conference, the user can catch up the conference by accessing the snapshot history that the user has missed.

An embodiment of the present disclosure has been described above with reference to the drawings. However, the present disclosure is not limited to the above-described embodiment, and can be implemented in various aspects without departing from the gist of the present disclosure. The drawings mainly and schematically illustrate various constituent elements for ease of understanding. Aspects such as thickness, length, number, and intervals of each constituent element illustrated in the drawings may differ in practice for convenience of drawing preparation. Furthermore, aspects such as material, shape, and dimension of each constituent element illustrated in the above embodiment are merely examples and not particular limitations. Various changes can be made without substantially deviating from the configuration of the present disclosure.