Server apparatus, content display control system, and recording medium

Disclosed is a server apparatus for providing authentication information to associate a client terminal with a plurality of display apparatuses capable of displaying a content distributed from the client terminal. The server apparatus includes a storage configured to store authentication pattern information associated with information relating to operations of the client terminal; and an issuing unit configured to issue the authentication pattern information stored in the storage to a display apparatus on which an operation has been performed by the user, among the plurality of display apparatuses, in a mode of disabling issuing a same item of authentication pattern information in a period from issuing the authentication pattern information to receiving return of the authentication pattern information from the display apparatus.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority of the prior Japanese Patent Application No. 2016-082320, filed on Apr. 15, 2016, and Japanese Patent Application No. 2016-165654, filed on Aug. 26, 2016, the entire contents of which are incorporated herein by reference.

FIELD

The embodiments discussed herein relate to a server apparatus, and a content display control system.

BACKGROUND

An environment (location) has been considered for use in a potential technology to hold workshops by coordinating multiple large-sized display apparatuses disposed on a wall surface of a room or disposed on a desk with a mobile terminal such as a smartphone or the like possessed by each user. In such an environment, an aspect of the technology may be to display a screen (content) displayed on the mobile terminal possessed by each user at a desired position of a desired one of the display apparatuses. Under such an environment where multiple display apparatuses and multiple mobile terminals are present, the position of the mobile terminal needs to be linked with the position of the display apparatus by some method in order to display the screen of a certain mobile terminal at a desired position on the desired display apparatus.

FIGS. 1A to 1Care diagrams illustrating an example of linking a frame drawn on a display apparatus with a mobile terminal. As illustrated inFIG. 1A, a user draws a frame corresponding to a display area with a finger or a pen on one of the display apparatuses. Thereafter, as illustrated inFIG. 1B, the frame on the display apparatus and the mobile terminal are linked by shaking the mobile terminal. As a result, as illustrated inFIG. 1C, the screen of the mobile terminal is transferred to the display area of the display apparatus, and the screen of the mobile terminal is displayed on the display area of the display apparatus.

There are known in the art a technology of pairing a display apparatus and a mobile terminal for coordinating the display apparatus with the mobile terminal, or a technology of performing personal authentication at login by vocalization according to guidance (e.g., Patent Documents 1 and 2).

Patent Documents

SUMMARY

According to an aspect of embodiments, there is disclosed a server apparatus for providing authentication information to associate a client terminal with a plurality of display apparatuses capable of displaying a content distributed from the client terminal. The server apparatus includes a storage configured to store authentication pattern information associated with information relating to operations of the client terminal; and an issuing unit configured to issue the authentication pattern information stored in the storage to a display apparatus on which an operation has been performed by the user, among the plurality of display apparatuses, in a mode of disabling issuing a same item of authentication pattern information in a period from issuing the authentication pattern information to receiving return of the authentication pattern information from the display apparatus.

The object and advantages of the invention will be realized and attained means of the elements and combinations particularly pointed out in the appended claims.

DESCRIPTION OF EMBODIMENTS

The following may need to be considered with the above-described method of performing operations such as shaking subsequent to drawing of the frame.FIGS. 2A to 2Care diagrams illustrating an example in which a link between a frame drawn on a display apparatus and a mobile terminal fails. That is, as illustrated inFIG. 2A, any one of users draws a frame corresponding to a display area with a finger or a pen on any one of the display apparatuses. Thereafter, as illustrated inFIG. 2B, when two or more users shake their mobile terminals substantially at the same time, it may be difficult to determine which user's mobile terminal is to be linked with the frame (display area) on the display apparatus (seeFIG. 2C).

FIGS. 3A and 3Bare diagrams illustrating an example in which a link between a frame drawn on a display apparatus and a mobile terminal fails in a case where there are two or more display apparatuses. As illustrated inFIG. 3A, it is assumed that a user of a mobile terminal X draws a frame on a display apparatus A and a user of a mobile terminal Y draws a frame on a display apparatus B, almost at the same time. In each of the display apparatuses A and B, the operation such as the shaking of the mobile terminal is monitored for a predetermined listening period from the time at which the frame is recognized; however, the listening periods overlap as illustrated inFIG. 3B. Hence, when an operation is performed on each of the mobile terminals X and Y within the overlapping period, it may be difficult to identify which mobile terminal has performed which operation (shake, etc.).

These will not be solved by the technologies disclosed in the aforementioned related art documents.

The following illustrates preferred embodiments with reference the accompanying drawings.

Configuration

FIG. 4is a diagram illustrating a configuration example of a system according to an embodiment. InFIG. 4, the server apparatus1, multiple display apparatuses3, and multiple mobile terminals4are connected to one another so as to communicate with one another via a network2(wired or wireless) such as a LAN (Local Area Network) and the like. The server apparatus1has a function to lend an authentication motion pattern for use in identifying (authenticating) the mobile terminal4to the display apparatus3and receive the authentication motion pattern that has been used and returned from the display apparatus3. The server apparatus1also has a function to transfer terminal motion information for authentication transmitted from the mobile terminal4to the display apparatus3serving as a candidate (hereinafter also referred to as the “candidate display apparatus3”).

The display apparatus3has a large-sized display (including a liquid crystal display and the like as well as a projector), and also has a function of a PC (personal computer) that performs information processing. The mobile terminal4is an information apparatus such as a smartphone. The server apparatus1may be integrated with any one of the display apparatuses3.

In the following description, it is assumed that one screen displayed on the mobile terminal4is linked with one frame of any one of the display apparatuses3. Note that such a configuration may also be expanded in order to link one screen of the mobile terminal4with the frame of one or multiple display apparatuses3or to link multiple screens of the mobile terminal4to the frame of one or multiple display apparatuses3.

FIG. 5is a diagram illustrating a functional configuration example of the server apparatus1. InFIG. 5, the server apparatus1includes an authentication motion pattern manager11, a terminal motion information transferor12, an authentication motion pattern DB13, a pattern lending display list14, a terminal motion information transfer destination table15, and a display tilt angle table16.

When receiving an authentication motion pattern acquisition request from any one of the display apparatuses3via the network2, the authentication motion pattern manager11extracts one of the authentication motion patterns from the authentication motion pattern DB13, and transmits the extracted authentication motion pattern to the request source display apparatus3. In this case, all the information forming the authentication motion patterns may be transmitted; alternatively, in a case where the display apparatus3holds in advance information on the same content as the authentication motion pattern DB13, an ID for identifying the authentication motion pattern (a pattern ID) may be transmitted. The lent authentication motion pattern is managed such that the same authentication motion pattern will not be lent to another display apparatus redundantly (in an overlapped manner). In addition, the authentication motion pattern manager11records, in the pattern lending display list14, an ID (a display ID) of the display apparatus3that has lent the authentication motion pattern. Note that in a case where the authentication motion pattern manager11has lent the authentication motion pattern, the authentication motion pattern manager11transmits a terminal motion information transmission request to all the mobile terminals4present in a predetermined location (a predetermined range in which the multiple display apparatuses3may be used).

The authentication motion pattern manager11has a function to return the authentication motion pattern to the authentication motion pattern DB13upon receiving an authentication motion pattern returning request from the display apparatus3. The authentication motion pattern manager11deletes, from the pattern lending display list14, the ID (display ID) of the display apparatus3that has returned the authentication motion pattern while deleting a combination of the corresponding display ID and terminal ID from the terminal motion information transfer destination table15. Further, in a case where there are no lent authentication motion patterns left, the authentication motion pattern manager11transmits a terminal motion information transmission stop request to all mobile terminals4present in a predetermined location.

FIG. 6is a diagram illustrating an example of the data structure of the authentication motion pattern DB13, which includes items such as a “pattern ID”, “guidance information”, “acceleration information”, and the like. The “Pattern ID” is information that identifies the authentication motion pattern. Operations corresponding to the authentication motion pattern may, for example, include moving the mobile terminal4in a triangular shape, moving the mobile terminal4in a rectangular shape, moving the mobile terminal4to draw a circle, moving the mobile terminal4in any one of a right direction, a left direction, an upward direction and a downward direction, moving the mobile terminal4in left and right directions or in up and down directions, and the like.

The “guidance information” includes information such as moving images (animation), still images, text, voice, or the like that teaches how to operate in order to make the user understand what operation needs to be performed for each of the authentication motion patterns. The “acceleration information” is acceleration time-series data recorded by actually performing an operation in advance for of the authentication motion patterns in order to determine whether the operation actually performed by the user matches the operation defined by the authentication motion pattern.

Referring back toFIG. 5, the terminal motion information transferor12has a function to refer to the display tilt angle table16based on the tilt angle information of the mobile terminal4included in the terminal motion information to extract the candidate display apparatuses3in a case where the terminal motion information transferor12receives the terminal motion information from the mobile terminal4.FIGS. 7A and 7Bare diagrams illustrating an example of extraction by the display apparatus3in accordance with tilt angles of the mobile terminals4. As illustrated inFIG. 7A, acceleration information from the mobile terminals4having a tilt angle of 90 degrees is transferred to the vertically placed display apparatus3. As illustrated inFIG. 7B, acceleration information from the mobile terminals4having a tilt angle of 0 degrees is transferred to the horizontally placed display apparatus3. When moving the mobile terminal4according to the guidance, it is natural to align the display surface direction of the display apparatus3with the display surface direction of the mobile terminal4. Hence, the moving operation of the mobile terminal4is handled as an operation with respect to the display apparatus3only when the display surface directions of the display apparatus3and the mobile terminal4are matched.

Referring back toFIG. 5, the terminal motion information transferor12has a function to transfer acceleration information included in the terminal motion information from the mobile terminal4to the extract the candidate display apparatus3. Note that the tilt angle determined as being matched as a result is registered in the terminal motion information transfer destination table15to be reused.

FIGS. 8A to 8Care diagrams illustrating examples of data structures of various lists and tables. The pattern lending display list14lists display IDs. The terminal motion information transfer destination table15includes each terminal ID in association with one or more display ID (s). The display tilt angle table16includes a tilt angle set for each of the display IDs.

Next,FIG. 9is a diagram illustrating a functional configuration example of the display apparatus3. InFIG. 9, the display apparatus3includes an operation recognizer302, an authentication motion pattern request transmitter303, and an operation information transmitter304. The display apparatus3further includes an authentication motion pattern receiver305, a guidance display307, terminal motion information receiver309, an authentication processor311, an authentication result transmitter313, an authentication motion pattern returning unit314, and a screen receiver315.

The operation recognizer302has a function to recognize an operation based on coordinates of a contact position of a user's finger, a pen, or the like on the screen by a touch sensor301disposed on the front surface of a display308. For example, when a rectangular frame is drawn, the operation recognizer302recognizes a frame drawing operation. Note that it may be possible to use another mechanism to obtain the coordinates of the contact position by the user's finger or the pen on the screen instead of the touch sensor301. The authentication motion pattern request transmitter303has a function to transmit an authentication motion pattern request to the server apparatus1when the operation recognizer302recognizes (detects) the frame drawing operation.

The authentication motion pattern receiver305has a function to receive the authentication motion pattern from the server apparatus1to hold the received authentication motion pattern as an authentication motion pattern306. The guidance display307has a function to display guidance moving images or the like on the display screen of the display308, for example, within the frame that has been recognized, formatted and displayed, based on the guidance information included in the authentication motion pattern306.

The terminal motion information receiver309has a function to receive terminal motion information (acceleration information) from the mobile terminal4transferred from the server apparatus1to hold the received terminal motion information as terminal motion information310. The authentication processor311has a function to match the acceleration information of the authentication motion pattern306with the acceleration information of the terminal motion information310to determine whether these are the same authentication motion pattern. The authentication processor311determines that the terminal motion information310matches the authentication motion pattern306, the authentication processor311registers a frame (frame ID, XY coordinates) linked with the terminal ID in a frame-terminal correspondence table312. The authentication result transmitter313has a function to transmit the display ID to the corresponding mobile terminal4as an authentication result indicating that the authentication has been performed by the authentication processor311.

The authentication motion pattern returning unit314has a function to return to the server apparatus1the authentication motion pattern for which authentication has been completed in the authentication processor311or the authentication motion pattern for which authentication has not been performed due to the time being up. Upon returning the authentication motion pattern, it is not necessary to transmit all the information forming the authentication motion patterns, but the pattern ID may be specified. Upon returning the authentication motion pattern, the display ID that has used the authentication motion pattern and, when there is, the terminal ID of the linked mobile terminal4are also transmitted.

The screen receiver315has a function to receive an application screen from the linked mobile terminal4to display the received application screen on the display308. The operation information transmitter304has a function to transmit, to the mobile terminal4, the operation (enlargement/reduction, movement, page feed/return, point, etc., of the screen) of the application screen within the frame recognized by the operation recognizer302. The mobile terminal4that has received the operation information performs screen transition according to the operation information and transmits an update screen. The transmitted update screen is then received by the screen receiver315to thereby perform screen update of the display308.

Next,FIG. 10is a diagram illustrating a functional configuration example of the mobile terminal4. InFIG. 10, the mobile terminal4includes a terminal motion recognizer402, a tilt angle calculator403, a terminal motion information transmitter404, an authentication result receiver406, a screen linkage application408, a screen transmitter409, an operation information receiver410.

The terminal motion recognizer402has a function to recognize the motion or movement (mainly acceleration change) of the mobile terminal4itself from the signal of a built-in IMU (Inertial Measurement Unit) sensor401. The tilt angle calculator403has a function to calculate the tilt angle of the mobile terminal4similarly from the built-in IMU sensor401. The terminal motion information transmitter404has a function to transmit terminal motion information to the server apparatus1based on the motion or movement recognized by the terminal motion recognizer402and the tilt angle calculated by the tilt angle calculator403. Note that the terminal motion information transmitter404has a stop flag405and the terminal motion information transmitter404operates only when the stop flag405is OFF. The stop flag405is switched OFF when a terminal motion information transmission request is transmitted from the server apparatus1. When a terminal motion information transmission cancelling request is transmitted from the server apparatus1or when an authentication result that the linking has completed is obtained from the display apparatus3, the stop flag405is switched ON.

The authentication result receiver406has a function to receive, from, the display apparatus3, the authentication result that has been linked to hold the display ID included in the received authentication result as a display ID407. The screen linkage application408is a process entity of an application program for providing a screen to be displayed on the display apparatus3. The screen transmitter409has a function to transmit a screen of the screen linkage application408to the linked display apparatus3. The operation information receiver410has a function to receive operation information with respect to the screen from the display apparatus3to transfer the received operation information to the screen linkage application408.

FIG. 11is a diagram illustrating an example of a hardware configuration in association with information processing of the server apparatus1, the display apparatus3, and the mobile terminal4. InFIG. 11, the server apparatus1and the like include a CPU (Central Processing Unit)101, a ROM (Read Only Memory)102, and a RAM (Random Access Memory)103that are mutually connected via a bus107. The server apparatus1and the like further include a HDD (Hard Disk Drive)/SSD (Solid State Drive)104, a connection I/F (Interface)105, and a communication I/F106. The CPU101is configured to integrally control operations of the server apparatus1by executing programs stored in the ROM102or in one of the HDD24and the SSD104, using the RAM103as a work area. The connection I/F105is configured to serve as an interface with an apparatus connected to the server apparatus1and the like. The communication I/F106is an interface for performing communications with other information apparatuses via a network.

The functions of the respective apparatuses described with reference toFIGS. 5, 9, and 10are implemented by executing predetermined programs in the CPU101. Such programs may be acquired via a recording medium, may be acquired via a network, or may be embedded in a ROM. Data to be referred to or updated at the time of processing is held in the RAM103or the HDD/SSD104. In addition, in the display apparatus3, the display308configured to display a screen and the touch sensor301are connected to the connection I/F105. In the mobile terminal4, a display configured to display a screen, a touch sensor and a built-in IMU sensor401are connected to the connection I/F105.

Operations

FIG. 12is a sequence diagram illustrating a process example according to the embodiment. InFIG. 12, a user of the mobile terminal4selects a screen desired to be displayed on the display apparatus3from the screen linkage application408of the mobile terminal4(step S101), and traces a frame corresponding to a display area with a finger, a pen, and the like on the screen of the desired display apparatus3to draw the frame (step S102).FIG. 13Aillustrates a state in which the screen “A” is displayed on the mobile terminal4and the user draws a rectangular frame on the display apparatus3.

Referring back toFIG. 12, when the operation recognizer302of the display apparatus3detects that a frame has been drawn, the operation recognizer302displays the display area that has been formatted on the screen (step S103).FIG. 13Billustrates a state in which the display area is displayed on the screen of the display apparatus3.

Referring back toFIG. 12, the authentication motion pattern request transmitter303of the display apparatus3transmits an authentication motion pattern request to the server apparatus1with the display ID of the display apparatus3itself (step S104). In response to this request, the authentication motion pattern manager11of the server apparatus1extracts one of the authentication motion patterns from the authentication motion pattern DB13and transmits the extracted authentication motion pattern to the request source display apparatus3(step S105). The authentication motion pattern extracted so as not to lend the same authentication motion pattern redundantly (in an overlapped manner); for example, the same authentication motion pattern will not be lent by removing the authentication motion pattern redundantly (in an overlapped manner) at the end of the permutation of the authentication motion pattern from the permutation. In addition, the authentication motion pattern manager11transmits a terminal motion information transmission request to all the mobile terminals4being present in a predetermined location (a predetermined range in which multiple display apparatuses3may be used) (step S106).

The display apparatus3displays the guidance on the guidance display307based on the authentication motion pattern received by the authentication motion pattern receiver305(step S107).FIG. 13Cillustrates a state in which guidance for instructing a user to move the user's mobile terminal4in a clockwise direction is displayed in the display area of the display apparatus3.

Referring back toFIG. 12, when the user of the mobile terminal4moves the user's mobile terminal4according to the guidance of the authentication motion pattern (step S108), the terminal motion recognizer402recognizes the terminal motion (acceleration change), and the tilt angle calculator403calculates the tilt angle of the mobile terminal4. Subsequently, the terminal motion information transmitter404transmits the terminal motion information including those described above to the server apparatus1together with the terminal ID of the user's mobile terminal itself (step S109).FIG. 13Dillustrates a state in which the user moves the mobile terminal4in a clockwise direction according to the guidance.

Referring back toFIG. 12, the terminal motion information transferor12of the server apparatus1refers to the display tilt angle table16to specify the transfer destination display apparatus3based on the tilt angle of the terminal motion information received from the mobile terminal4(Step S110). Note that the terminal ID and the display ID that are linked based on the tilt angle are stored in the terminal motion information transfer destination table15, and hence, it is not necessary to determine the tilt angle every time the transfer destination display apparatus3is specified. Subsequently, the terminal motion information transferor12transmits the acceleration information of the terminal motion information to the specified display apparatus3together with the terminal ID (step S111).

When the terminal motion information receiver309of the display apparatus3receives the terminal motion information (acceleration information) the authentication processor311matches the received terminal motion information (the received acceleration information) with the acceleration information of the previously acquired authentication motion pattern (step S112). In this case, when the authentication processor311determines that the received terminal motion information (the received acceleration information) matches the acceleration information of the previously acquired authentication motion pattern (step S112), the authentication result transmitter313transmits the display ID to the mobile terminal4as an authentication result indicating that the authentication has been successful (step S113). The authentication motion pattern returning unit314returns the authentication motion pattern to the server apparatus1together with the pattern ID, the display ID, and the terminal ID (when there is any) (step S114). The authentication motion pattern manager11of the server apparatus1returns the returned authentication motion pattern to the authentication motion pattern DB13. For example, in a case where the authentication motion pattern is lent by removing the authentication motion pattern at the end of the permutation of the authentication motion pattern from the permutation, the returning of the authentication motion pattern is completed by adding the returned authentication motion pattern back to the end of the permutation again. Further, in a case where there are no lent authentication motion patterns left as a result of the returning of the authentication motion pattern, the authentication motion pattern manager11transmits a transmission stop request of the terminal motion information to all mobile terminals4present in a predetermined location (step S115).

The mobile terminal4that has received the authentication result indicating that the authentication has been successful from the authentication result receiver406starts transferring, by the screen transmitter409, the screen of the screen linkage application408to the display apparatus3to which the mobile terminal4is linked (step S116). Note that it is assumed that the transfer of the screen maybe voluntarily started by the mobile terminal4in this embodiment; however, the transfer of the screen may be requested by the display apparatus3such that the mobile terminal4may start transferring the screen in response to the request. Upon receiving the screen, the screen receiver315of the display apparatus3displays the received screen on the display308(step S117).FIG. 13Eillustrates a state in which the screen “A” of the mobile terminal4is displayed in the display area of the display apparatus3.

Referring back toFIG. 12, when any one of users subsequently performs an operation with respect to the area where the screen of the display apparatus3is displayed (step S118), the operation recognizer302of the display apparatus3recognizes the operation. The operation information transmitter304then transmits operation information to the corresponding mobile terminal4(step S119). When the operation information receiver410of the mobile terminal4receives the operation information, the screen linkage application408updates the screen according to the content of the operation, and the screen transmitter409transmits the updated screen to the display apparatus3(step S120). Upon receiving the updated screen, the screen receiver315of the display apparatus3displays the received updated screen on the display308(screen updated) (step S121).

The following illustrates detailed operations of the server apparatus1, the display apparatus3and the mobile terminal4.

FIG. 14is a flowchart illustrating a process example of the server apparatus1. InFIG. 14, when receiving an event from the outside (step S201), the server apparatus1determines whether the received event is one of an authentication motion pattern request (step S202), returning of the authentication motion pattern (step S203), and terminal motion information (step S204).

When the received event is the authentication motion pattern request (Yes in step S202), the authentication motion pattern manager11extracts one of the authentication motion patterns from the authentication motion pattern DB13, and disables lending the authentication motion pattern redundantly (in an overlapped manner) (step S205). Subsequently, the authentication motion pattern manager11registers the display ID of the request source display apparatus3in the pattern lending display list14(step S206), and transmits the extracted authentication motion pattern to the request source splay apparatus3(Step S207). Subsequently, the authentication motion pattern manager11issues a terminal motion information transmission request to all mobile terminals4within the location (step S208). The process then returns to a standby mode for receiving an event (step S201).

When the received event is the return of the authentication motion pattern (Yes in step S203), the authentication motion pattern manager11transmits the returned authentication motion pattern back to the authentication motion pattern DB13(step S209). The authentication motion pattern manager11subsequently deletes the corresponding display ID from the pattern lending display list14(step S210). The authentication motion pattern manager11then determines whether there is a terminal ID linked with the display ID in the terminal motion information transfer destination table15(step S211). When there is a linked terminal ID (Yes in step S211), this linked terminal ID is deleted from the terminal motion information transfer destination table15(step S212). When there is no linked terminal ID (No in step S211), the deletion of the terminal ID from the terminal motion information transfer destination table15is not performed.

The authentication motion pattern manager11determines whether the pattern lending display list14includes no display ID of the display apparatus3(whether the pattern lending display list14is empty) (step S213). When the pattern lending display list14includes no display ID of the display apparatus3(Yes in step S213), the authentication motion pattern manager11issues the terminal motion information transmission stop request to all the mobile terminals4within the location (Step S214). When the pattern lending display list14includes any display ID of the display apparatus3(No in step S213), the authentication motion pattern manager11will not issue the terminal motion information transmission stop request. The process then returns to a standby mode for receiving an event (step S201).

When the received event is the terminal motion information (Yes in step S204), the terminal motion information transferor12determines whether the transfer destination of the terminal ID of the transmission source is already included in the terminal motion information transfer destination table15(step S215). When the transfer destination of the terminal ID of the transmission source is not included in the terminal motion information transfer destination table15(No in step S215), the terminal motion information transferor12searches the display tilt angle table16for the display apparatus3having the tilt angle matching with the tilt angle of the mobile terminal4among the display apparatuses3included in the pattern lending display list14(step S216). The terminal ID and the display ID are added to the terminal motion information transfer destination table15in accordance with the search result (step S217). When the transfer destination of the terminal ID of the transmission source is already included in the terminal motion information transfer destination table15(Yes in step S215), search by the tilt angle will not be performed and the terminal ID and the display ID obtained in accordance with the search result will not be added to the terminal motion information transfer destination table15. The terminal motion information transferor12transmits the acceleration information of the terminal motion information and the terminal ID to the display apparatus3serving as the transfer destination (step S218). The process then returns to a standby mode for receiving an event (step S201).

FIG. 15is a flowchart illustrating a process example of the display apparatus3. InFIG. 15, the operation recognizer302of the display apparatus3performs touch detection (step S301). When the operation recognizer302has detected the touch (Yes in step S302), the operation recognizer302acquires a touched position to hold the acquired touched position as a touch trajectory (step S303). The operation recognizer302increments the touch trajectory until the operation recognizer302recognizes the end of the touch. When the operation recognizer302recognizes the end of the touch (Yes in step S304), the operation recognizer302recognizes an event for writing a frame from the touch trajectory (step S305). When the operation recognizer302recognizes the frame (Yes in step S306), the authentication motion pattern request transmitter303transmits an authentication motion pattern request to the server apparatus1(step S307).

Upon receiving the authentication motion pattern from the server apparatus1in response to the authentication motion pattern request (step S308), the authentication motion pattern receiver305displays guidance on the guidance display307based on guidance information included in the authentication motion pattern (step S309). The time of the internal timer is then set to “0” to start timekeeping (step S310). This timekeeping is used for determining whether the time is up in a case where authentication is incapable of being performed.

Subsequently, the terminal motion information receiver309checks whether the terminal motion information receiver309has received an incoming call of the terminal motion information from the server apparatus1(step S311), and when the terminal motion information receiver309has received the incoming call (Yes in step S312), the terminal motion information receiver309acquires the terminal motion information (step S313). The authentication processor311matches the acquired terminal motion information with the previously received authentication motion pattern (step S314). When the authentication processor311determines that the acquired terminal motion information matches the previously received authentication motion pattern as a result of the matching (Yes in step S315), the authentication processor311links the terminal ID with the frame (display area) to register the linked terminal ID in the frame-terminal correspondence table312(step S316). The authentication result transmitter313subsequently transmits the authentication result (display ID) to the corresponding mobile terminal4(step S317). The authentication motion pattern returning unit314returns the authentication motion pattern to the server apparatus1(step S318). The screen to be transmitted later from the mobile terminal4is displayed within a frame to which the screen is linked, transfer of the operation information performed on the screen to the mobile terminal4is started (step S319), and the process of detecting touch is processed again (step S301).

When the terminal motion information receiver309has not received an incoming call of the terminal motion information from the server apparatus1(No in step S312) or when the matching result indicates that the acquired terminal motion information does not match the previously received authentication motion pattern as a result of the matching (No in step S315), the time of the timer is incremented (step S320), and whether the timeout has occurred is determined (step S321). When the timeout has not occurred (No in step S321), the process of checking the incoming call is performed again (step S311). When the timeout has occurred (Yes in step S321), the process moves to the returning of the authentication motion pattern (step S318).

FIG. 16is a flowchart illustrating a process example of the mobile terminal4. InFIG. 16, when receiving an event from the outside (step S401), the mobile terminal4determines whether the received event is a terminal motion information transmission request, a terminal motion information transmission stop request, an authentication result (linkage), or screen operation information (steps S402to S405).

When the received event is the terminal motion information transmission request (Yes in step S402), the terminal motion information transmitter404turns OFF the stop flag405(step S406). Subsequently, the terminal motion information transmitter404determines whether the screen of the mobile terminal4itself has already been linked with the display apparatus3(step S407). When the mobile terminal4itself has already been linked with the display apparatus3(Yes in step S407), the process then returns to a standby mode for receiving an event (step S401). When the mobile terminal4itself has not been linked with the display apparatus3(No in step S407), the terminal motion information transmitter404starts monitoring the motion of the mobile terminal4itself (step S408). The process then returns to a standby mode for receiving an event (step S401).

In the above started motion monitoring, the terminal motion information transmitter404determines whether the motion amplitude (the amplitude of the acceleration change) is equal to or greater than a predetermined value (step S409). When the motion amplitude is equal to or greater than a predetermined value (Yes in step S409), the terminal motion information transmitter404starts transmitting the terminal motion information (step S410). Subsequently, the terminal motion information transmitter404determines whether the stop flag405is switched ON (step S411). When the stop flag405is not switched ON (No in step S411), the terminal motion information transmitter404continues transmitting the terminal motion information (step S410). When the stop flag405is switched ON (Yes in step S411), the terminal motion information transmitter404ends the transmission of the terminal motion information (step S412).

When the received event is the terminal motion information transmission stop request (Yes in step S403), the terminal motion information transmitter404turns ON the stop flag405(step S413). The process then returns to a standby mode for receiving an event (step S401).

When the received event is the authentication result (linkage) (Yes in step S404), the terminal motion information transmitter404turns ON the stop flag405(step S414), and the screen transmitter409starts the transfer of the screen (step S415). The process then returns to a standby mode for receiving an event (step S401).

When the received event is the screen operation information (Yes in step S405), the screen linkage application408transitions the application screen (step S416) and the screen transmitter409transfers the screen (step S417). The process then returns to a standby mode for receiving an event (step S401).

Note that the illustration is given above of the case where the matching of the authentication motion pattern is performed by the display apparatus3; however, the matching of the authentication motion pattern may be performed by the server apparatus1.

Modification

In the embodiment described above, the mobile terminal4itself is moved in accordance with a predetermined pattern, and the embodiment may be applied to the mobile terminal4having an inertial sensor (acceleration sensor, gyroscope, etc.) such as a smartphone. However, the embodiment is not applicable to a mobile terminal4having no inertial sensor such as a PC (Personal Computer). Hence, the following illustrates modification of the embodiment that may be applied to a mobile terminal4having no inertial sensor. Note that the modification may be combined with the above-described embodiment.

FIG. 17is a diagram illustrating a functional configuration example of the server apparatus1. InFIG. 17, the server apparatus1includes an authentication character string manager11′, an input character string information transferor12′, and an authentication character string allocating display list14′.

When receiving an authentication character string acquisition request from any one of the display apparatuses3via the network2, the authentication character string manager11′ has a function to issue an authentication character string that does not overlap the authentication character string currently being allocated and transmits the issued authentication character string to the request source display apparatus3. The allocated authentication character string is managed such that the allocated authentication character string will not be allocated to another display apparatus redundantly (in an overlapped manner). The authentication character string manager11′ records, in the authentication character string allocating display list14′, the ID (display ID) of the display apparatus3that has allocated the authentication character string. The authentication character string allocating display list14′ may have a data structure similar to a data structure of the aforementioned pattern lending display list14(FIG. 8A), or may hold the authentication character string that has been allocated in association with the display ID. Note that in a case where the authentication character string manager11′ has allocated the authentication character string, the authentication character string manager11′ transmits a transmission request of input character string information to all the mobile terminals4being present in a predetermined location (a predetermined range in which the multiple display apparatuses3may be used). The input character string information is character string information input by a user who has seen the authentication character string.

The authentication character string manager11′ has a function to perform a process of returning an authentication character string when receiving a return request of the authentication character string from the display apparatus3. That is, the authentication character string manager11′ deletes, from the authentication character string allocating display list14′, the ID (display ID) of the display apparatus3that has returned the authentication character string, and resets the authentication character string to be in a state of not being allocated. Further, in a case where there is no remaining authentication character string currently being allocated, the authentication character string manager11′ transmits a transmission stop request of the input character string information to all mobile terminals4being present in the predetermined location.

The input character string information transferor12′ has a function to transfer the input character string information to the display apparatus3with the terminal ID when receiving the input character string information from the mobile terminal4.

FIG. 18is a diagram illustrating a functional configuration example of the display apparatus3. InFIG. 18, the display apparatus3includes the operation recognizer302, an authentication character string request transmitter303′, and the operation information transmitter304. The display apparatus3further includes an authentication character string receiver305′, an authentication character string display307′, an input character string information receiver309′, the authentication processor311, the authentication result transmitter313, an authentication character string return unit314′, and the screen receiver315.

The operation recognizer302includes a function to recognize an operation based on coordinates of a contact position of a user's finger, a pen, or the like on the screen by the touch sensor301disposed on the front surface of the display308. For example, when a rectangular frame is drawn, the operation recognizer302recognizes a frame drawing operation. Note that it may be possible to use another mechanism to obtain the coordinates of the contact position by the user's finger or the pen on the screen instead of the touch sensor301. The authentication character string request transmitter303′ has a function to transmit an authentication character string request to the server apparatus1when the operation recognizes302recognizes (detects) the frame drawing operation.

The authentication character string receiver305′ has a function to receive the authentication character string from the server apparatus1to hold the received authentication character string as an authentication character string306′. The authentication character string display unit307′ has a function to display an authentication character string on the display screen of the display308, for example, within the recognized and formatted frame or adjacent to the frame, on the basis of the authentication character string306′.

The input character string information receiver309′ has a function to receive input character string information from the mobile terminal4transferred from the server apparatus1to hold the received input character string information as input character string information310′. The authentication processor311has a function to match the authentication character string306′ with the input character string information310′ and to determine whether the authentication character string306′ and the input character string information310′ are the same. When the authentication processor311determines that the authentication character string306′ matches the input character string information310′, the authentication processor311registers a frame (frame ID, XY coordinates) in association with the terminal ID in the frame-terminal correspondence table312. The authentication result transmitter313has a function to transmit the display ID to the corresponding mobile terminal4as an authentication result indicating that the authentication has been performed by the authentication processor311.

The authentication character string returning unit314′ has a function to return to the server apparatus1the authentication character string that has been authenticated by the authentication processor311or the authentication character string that has not been authenticated due to time-up. The authentication character string may be specified for requesting the returning of the authentication character string.

The screen receiver315has a function to receive an application screen from the mobile terminal4that has been linked to display the received application screen on the display308. The operation information transmitter304has a function to transmit, to the mobile terminal4, the operation (enlargement; reduction, movement, page feed/return, point, etc., of the screen) of the application screen within the frame recognized by the operation recognizer302. The mobile terminal4that has received the operation information performs screen transition according to the operation information and transmits an update screen. The transmitted update screen is then received by the screen receiver315to thereby perform screen update of the display308.

FIG. 19is a diagram illustrating an example of a functional configuration of the mobile terminal4. InFIG. 19, the mobile terminal4includes a character string input unit402′, an input character string information transmitter404′, the authentication result receiver406, the screen linkage application408, the screen transmitter409, and the operation information receiver410.

The character string input unit402′ has a function to input a character string from a user. The input character string information transmitter404′ has a function to transmit, to the server apparatus1, the character string information input by the character string input unit402′. Note that the terminal motion information transmitter404has the stop flag405, and the terminal motion information transmitter404′ operates only when the stop flag405is switched OFF. The stop flag405is switched OFF when an input character string information transmission request is transmitted from the server apparatus1. When a character string information stop request is transmitted from the server apparatus1or when an authentication result that the linking has completed is obtained from the display apparatus3, the stop flag405is switched ON.

The authentication result receiver406has a function to receive, from the display apparatus3, the authentication result that has been linked to hold the display ID included in the received authentication result as the display ID407. The screen linkage application408is a process entity of an application program for providing a screen to be displayed on the display apparatus3. The screen transmitter409has a function to transmit a screen of the screen linkage application408to the linked display apparatus3. The operation information receiver410has a function to receive operation information with respect to the screen from the display apparatus3to transfer the received operation information to the screen linkage application408.

FIG. 20is a sequence diagram illustrating a processing example of the embodiment. InFIG. 20, a user of the mobile terminal4selects a screen desired to be displayed on the display apparatus3from the screen linkage application408of the mobile terminal4(step S131), and traces a frame corresponding to a display area with a finger, a pen, and the like on the screen of the desired display apparatus3to draw the frame (step S132). FIG.21A illustrates a state in which the screen “A” is displayed on the mobile terminal4and the user draws a rectangular frame on the display apparatus3.

Referring back toFIG. 20, when the operation recognizer302of the display apparatus3detects that a frame has been drawn, the operation recognizer302displays the display area that has been formatted on the screen (step S133).FIG. 21Billustrates a state in which the display area is displayed on the screen of the display apparatus3.

Referring back toFIG. 20, the authentication character string request transmitter303′ of the display apparatus3transmits an authentication character string request to the server apparatus1with the display ID of the display apparatus3itself (step S134). In response to the authentication character string request, the authentication character string manager11′ of the server apparatus1sequentially or randomly issues an authentication character string that does not overlap the authentication character string currently being allocated, and transmits the issued authentication character string to the request source display apparatus3(step S135). For example, the authentication character string manager11′ issues an authentication character string “67” to transmit the issued authentication character string “67” to the display apparatus3. In addition, the authentication character string manager11′ also transmits an input character string information transmission request to all the mobile terminals4being present within a predetermined location. (a predetermined range in which multiple display apparatuses3may be used) (step S136). Upon receiving the input character string information transmission request, the mobile terminal4displays a character string input field on the screen.

The display apparatus3displays an authentication character string on the authentication character string display307′ based on the authentication character string received by the authentication character string receiver305′ (step S137).FIG. 21Cillustrates a state in which the authentication character string “67” is displayed in a display area of the display apparatus3and the character string input field is displayed on the screen of the mobile terminal4.

Referring back toFIG. 20, the user of the mobile terminal4inputs the same character string in the input field of the mobile terminal4according to the authentication character string displayed on the display apparatus3(step S138). The character string input section402′ inputs a character string, and the input character string information transmitter404′ transmits the input character string information to the server apparatus1together with the terminal ID of the mobile terminal itself (step S139).FIG. 21Dillustrates a state in which the user has input the character string “67” in the input field of the mobile terminal4.

Referring back toFIG. 20, the input character string information transferor12′ of the server apparatus1transfers the input character string information received from the mobile terminal4to the display apparatus3together with the terminal ID (step S140).

When the input character string information receiver309′ of the display apparatus3receives the input character string information and the terminal ID, the authentication processor311matches the received input character string information with the previously obtained authentication character string (step S141). In this case, when the authentication processor311determines that the received input character string information matches the previously obtained authentication character string, the authentication result transmitter313transmits the display ID to the mobile terminal4as an authentication result indicating that the authentication has been successful (step S142). The authentication character string returning unit314′ returns the authentication character string to the server apparatus1together with the authentication character string (step S143). That is, the authentication character string manager11′ of the server apparatus1deletes, from the authentication character string allocating display list14′, the ID (display ID) of the display apparatus3that has returned the authentication character string, and resets the authentication character string to be in a state of not being allocated. In addition, when all the authentication character strings being allocated have been returned and there are no remaining authentication character strings, the authentication character string manager11′ transmits an input character string information transmission stop request to all mobile terminals4being present in a predetermined location (step S144).

The mobile terminal4that has received the authentication result indicating that the authentication has been successful from the authentication result receiver406starts transferring, by the screen transmitter409, the screen of the screen linkage application408to the display apparatus3to which the mobile terminal4is linked (step S145). Note that it is assumed that the transfer of the screen maybe voluntarily started by the mobile terminal4in this embodiment; however, the transfer of the screen may be requested by the display apparatus3such that the mobile terminal4may start transferring the screen in response to the request. Upon receiving the screen, the screen receiver315of the display apparatus3displays the received screen on the display308(step S146).FIG. 21Eillustrates a state in which the screen “A” of the mobile terminal4is displayed in the display area of the display apparatus3.

Referring back toFIG. 20, when any one of users subsequently performs an operation with respect to the area where the screen of the display apparatus3is displayed (step S147), the operation recognizer302of the display apparatus3recognizes the operation. The operation information transmitter304then transmits operation information to the corresponding mobile terminal4(step S148). When the operation information receiver410of the mobile terminal4receives the operation information, the screen linkage application408updates the screen according to the content of the operation, and the screen transmitter409transmits the updated screen to the display apparatus3(step S149). Upon receiving the updated screen, the screen receiver315of the display apparatus3displays the received updated screen on the display308(screen updated) (step S150).

The following illustrates detailed operations of the server apparatus1, the display apparatus3and the mobile terminal4.

FIG. 22is a flowchart illustrating a process example of the server apparatus1. InFIG. 22, when receiving an event from the outside (step S221), the server apparatus1determines whether the received event is one of an authentication character string request (step S222), returning of the authentication character string (step S223), and input character string information (Step S224).

When the received event is the authentication character string request (Yes in step S222), the authentication character string manager11′ sequentially or randomly issues an authentication character string that does not currently overlap the authentication character string currently being allocated, and internally holds the issued authentication character string (step S225). Subsequently, the authentication character string manager11′ registers the requested display ID of the display apparatus3in the authentication character string allocating display list14′ (step S226), and transmits the issued authentication character string to the request source display apparatus3(Step S227). Subsequently, the authentication character string manager11′ issues an input character string information transmission request to all mobile terminals4within the location (step S228). The process then returns to a standby mode for receiving an event (step S221).

When the received event is returning of the authentication character string (Yes in step S223), the authentication character string manager11′ sets the returned authentication character string in a state of not being allocated, and deletes, from the authentication character string allocating display list14′, the corresponding display ID (step S229). Subsequently, the authentication character string manager11′ determines whether the authentication character string allocating display list14′ includes no authentication character string (whether the authentication character string allocating display list14′ is empty) (step S230). When the authentication character string allocating display list14′ includes no authentication character string (Yes in step S230), the authentication character string manager11′ issues an input character string information transmission stop request to all the mobile terminals4within the location (step S231). When the authentication character string allocating display list14′ includes any authentication character string (No in step S230), the authentication character string manager11′ will not issue the input character string information transmission stop request. The process then returns to a standby mode for receiving an event (step S221).

When the received event is input character string information (Yes in step S224), the input character string information transferor12′ transmits the input character string information and the terminal ID to the display apparatus3listed on the authentication character string allocating display list14′ (step S232). The process then returns to a standby mode for receiving an event (step S221).

FIG. 23is a flowchart illustrating a process example of the display apparatus3. InFIG. 23, the operation recognizer302of the display apparatus3performs touch detection (step S331). When the operation recognizer331has detected a touch (Yes in step S332), the operation recognizer332acquires a touched position to hold the acquired touched position as a touch trace (step S333). The operation recognizer302increments the touch trace until the operation recognizer302recognizes the end of the touch. When the operation recognizer302recognizes the end of the touch (Yes in step S334), the operation recognizer335recognizes a frame writing event from the touch trajectory (step S335). When the operation recognizer335recognizes the frame (Yes in step S336), the authentication character string request transmitter303′ transmits an authentication character string request to the server apparatus1(step S337).

Upon receiving, from the server apparatus1, the authentication character string in response to the authentication character string request (step S338), the authentication character string receiver305′ displays the authentication character string within the frame or in the vicinity of the frame (step S339). The time of the internal timer is then set to “0” to start timekeeping (step S340). This timekeeping is used for determining whether the time is up in a case where authentication is incapable of being performed.

Subsequently, the input character string information receiver309′ checks whether the input character string information receiver309′ has received an incoming call of the input character string information from the server apparatus1(step S341), and when the input character string information receiver309′ has received the incoming call (Yes in step S342), the input character string information receiver309′ acquires the input character string information (step S343). The authentication processor311matches the acquired input character string information with the previously received input character string (step S344). When the authentication processor311determines that the acquired input character string information matches the previously received input character string as a result of the matching (Yes in step S345), the authentication processor311links the terminal ID with the frame (display area) to register the linked terminal ID in the frame-terminal correspondence table312(step S346). The authentication result transmitter313subsequently transmits the authentication result (display ID) to the corresponding mobile terminal4(step S347). Subsequently, the authentication character string returning unit314′ returns the authentication character string to the server apparatus1(step S348). The screen to be transmitted later from the mobile terminal4is displayed within a frame to which the screen is linked, transfer of the operation information performed on the screen to the mobile terminal4is started (step S349), and the process then returns to touch detection (step S331).

When the input character string information receiver309′ has not received an incoming call of the input character string information from the server apparatus1(No in step S342), or when the authentication processor311determines that the acquired input character string information does not match the previously received input character string as a result of the matching (No in step S345), the timer is incremented (step S350), and whether the timeout has occurred is determined (step S351). When the timeout has not occurred (No in step S351), the process of checking the incoming call is performed again (step S341). When the timeout has occurred (Yes in step S351) the process moves to the returning of the input character string (step S348).

FIG. 24is a flowchart illustrating a process example of the mobile terminal4. InFIG. 24, when receiving an event from the outside (step S421), the mobile terminal4determines whether the received event is an input character string information transmission request, a terminal motion information transmission cancellation request, an authentication result (linkage), or screen operation information (steps S422to S425).

When the received event is the input character string information transmission request (Yes in step S422), the input character string information transmitter404′ turns OFF the stop flag405(step S426). Subsequently, the input character string information transmitter404′ determines whether the screen of the mobile terminal4itself has already been linked with the display apparatus3(step S427). When the mobile terminal4itself has already been linked with the display apparatus3(Yes in step S427), the process then returns to a standby mode for receiving an event (step S421). When the mobile terminal4itself has not been linked with the display apparatus3(No in step S427), the input character string information transmitter404′ displays an input field of a character string (step S428). The process then returns to a standby mode for receiving an event (step S421). Note that, when one mobile terminal4allows each of a plurality of applications to associate with a frame on the display apparatus3, it is not necessary to check whether the mobile terminal4itself has been linked with the display apparatus3.

After displaying an input field of the character string, the input character string information transmitter404′ determines whether a character string that has been input in the input field is finalized (step S429). When the input character string information transmitter404′ determines that the character string input in the input field finalized (Yes in step S429), the input character string information transmitter404′ transmits input character string information (step S430). Subsequently, the input character string information transmitter404′ determines whether the stop flag405is turned ON (step S431). When the stop flag405is not turned ON (No in step S431), the input character string information transmitter404′ stands ready to receive an input of the character string (step S429). When the stop flag405is turned ON (Yes in step S431), the input character string information transmitter404′ stops displaying the input field of the character string (step S432).

When the received event is the input character string information transmission stop request (Yes step S423), the input character string information transmitter404′ turns ON the stop flag405(step S433) and stops displaying the input field of the character string (step S434). The process then returns to a standby mode for receiving an event (step S421).

When the received event is the authentication result (linkage) (Yes in step S424), the input character string information transmitter404′ turns ON the stop flag405(step S435), and the screen transmitter409starts the transfer of the screen (step S436). The process then returns to a standby mode for receiving an event (step S421).

When the received event is screen operation information (Yes in step S425), the screen linkage application408transitions to an application screen (step S437) and the screen transmitter409transfers the screen (step S438). The process then returns to a standby mode for receiving an event (step S421).

Note that the illustration is given above of the case where the matching of the authentication character string is performed by the display apparatus3; however, the authentication character string may be performed by the server apparatus1.

Further, the embodiment given above is based on the example where the authentication character string is displayed on the display apparatus3and the character string corresponding to the authentication character string is input from the mobile terminal4; however, the embodiment is not limited to this example. The application ID for specifying an application may be input at the same time as inputting the authentication character string.FIG. 25is a diagram illustrating an example of inputting of a character string to the mobile terminal4. More specifically,FIG. 25illustrates a state in which a character string “67” corresponding to an authentication character string of the frame and the application ID “03” of the application have been input for being displayed within the frame. When the mobile terminal4is a PC or the like, windows of multiple applications may be displayed on the screen. In such a case, a window of which application is to be displayed on the display apparatus3needs to be selected. However, the selection of a window may be facilitated by allowing a user to input the application ID simultaneously with the authentication character string.

In this case, the input character string information transferor12′ (FIG. 17), the input character string information receiver309′ (FIG. 18), the character string input unit402′, and the input character string information transmitter404′ may have an expanded function to handle not only the character string corresponding to the authentication character string but also the application ID. The frame-terminal correspondence table312(seeFIG. 18) may also have an expanded function to associate the frames with the terminals and the applications.

Outline

One aspect of disclosed embodiments is to display a content on a display apparatus desired by a user.

The disclosed embodiments may provide a technology enabled to display a content on a display apparatus desired by a user.

ADVANTAGEOUS EFFECT

The disclosed embodiments may provide a technology capable of displaying content on a display apparatus desired by a user.

The preferred embodiments are described above. The embodiments of the present invention are illustrated with specific examples; however, the present invention is not limited to these examples, and various alterations or changes may be made without departing from the gist and the scope of the claims of the present invention. Specifically, the present invention shall not be construed as being limited to details of the specific examples and accompanying drawings thereof.