Abstract:
A method for interfacing a communication terminal ( 1 ) connected to a communication network ( 2 ) with object devices ( 4 ), comprising the steps of:—determining required services of an application executable from the communication terminal,—determining services provided by a plurality of objects devices connected to the communication network ( 2 ),—comparing the required services to the provided services so as to determine a missing service required by the application,—determining an upgradable object device among the plurality of object devices for a missing service, the upgradable object device belonging to an object type having a hardware capability ( 6 ) associated to said missing service,—searching a software resource repository ( 10 ) for a software module adapted so as to be downloaded to the upgradable object device to enable the upgradable object device to provide the missing service,—triggering a transfer of the software module from the software resource database to the upgradable object device.

Description:
FIELD OF THE INVENTION 
       [0001]    The invention relates to the field of the interaction between a communication terminal and physical objects through a communication network. 
       BACKGROUND 
       [0002]    The invention notably relates to the internet of objects and more precisely to the so-called “Web Of Things” which consists in transforming the physical objects in resources available through the web. The physical object may communicate through the Web. For instance, communication terminals, televisions, advertising boards, lamps, household appliances can communicate through the internet. 
         [0003]    One application of this architecture is, for a user, to interact through a communication terminal and through the internet with the physical objects. Such a terminal may be a personal computer, a notebook, a cellphone such as a “Smartphone”. 
         [0004]    Such mechanisms are notably described in works concerning the “Web of Things”. See for instance, the paper “Towards the Web of Things: Web Mashup for Embedded Devices” by D. Guinard and V. Trifa in Proc. 2nd Workshop on Mashups, Enterprises Mashups and Lightweight Composition on the Web (MEM&#39;09), 2009. 
         [0005]    Those works aim at allowing the consideration of physical objects as available resources for applications in order to enlarge the interaction possibilities between a user and physical objects through his communication terminal. 
       SUMMARY 
       [0006]    In an embodiment, the invention provides a method for interfacing a communication terminal connected to a communication network with object devices, comprising the steps of:
       determining required services of an application executable from the communication terminal,   determining services provided by a plurality of objects devices connected to the communication network,   comparing the required services to the provided services so as to determine a missing service required by the application,   determining an upgradable object device among the plurality of object devices for a missing service, the upgradable object device belonging to an object type having a hardware capability associated to said missing service,   searching a software resource repository for a software module adapted so as to be downloaded to the upgradable object device to enable the upgradable object device to provide the missing service,   triggering a transfer of the software module from the software resource database to the upgradable object device.       
 
         [0013]    According to embodiments, such device can comprise one or more of the features below. 
         [0014]    In embodiments of the method, the required services are determined on the basis of a description of the application including identifications of the services required by the application. 
         [0015]    In embodiments of the method, the method further comprises an initial step of generating the description of the application and the associated identifications of the services required by the application as a function of a user&#39;s selection of a service and/or a predefined application. 
         [0016]    In embodiments of the method, the generation of a description comprises a step of accessing a services database and/or predefined applications database, the selection of a service and/or an predefined application being made on the basis of the services and/or predefined applications included in the services and/or predefined applications database. 
         [0017]    In embodiments of the method, the generation of the description comprises a step of generating a logic description in the application description as a function of a logic determined by the user which describes the relations between services and/or predefined applications for the execution of the application. 
         [0018]    In embodiments of the method, the step of generating a description of the application comprises a step of identifying a service required by the predefined application as a function of the interface point of the internal application. 
         [0019]    In embodiments of the method, a software module is associated to the application in the initial step of generating description of the application and stored on the communication terminal, the searching of the software module being made in the communication terminal. 
         [0020]    In embodiments of the method, the step of determining the required services further comprises a step of accessing an object description comprising indications on the services provided by the object, the determination of the provided services being made on the basis of the indications on the provided services of the object description. 
         [0021]    In embodiments of the method, the determination of an upgradable object comprises: 
         [0000]    a step of accessing an object description of a plurality of object device, the object description comprising an indication on the type of a respective object,
 
a step of identifying a type of object device needed as a function of the missing service and
 
a step of selecting an object device as a function of the type of object needed and the indications on the type of object, the triggering of the transfer being made to the selected object device.
 
         [0022]    In embodiments of the method, the method further comprises: 
         [0000]    a step of determining a second set of object devices after the first set of object devices,
 
a step of determining an outdated object device as an object device which is comprised in the first set of object devices but absent in the second set of object devices, and
 
a step of triggering the uninstalling of the software module from the outdated object device.
 
         [0023]    In embodiments of the method, the software module search step is made in a software resource repository which is external to the communication terminal and connected to the communication network. 
         [0024]    In embodiments of the method, the search step is made in a software resource repository which is integrated to the communication terminal and the software module is associated to the application. 
         [0025]    In embodiments of the method, the method further comprises a step of determining a set of object devices connected to the communication network as a function of the communication terminal localization and wherein the determination of the services provided by a plurality of objects devices is made on said set of object devices. 
         [0026]    In an embodiment, the invention also provides a computer program comprising computer-executable instructions that perform the above-mentioned method when executed. 
         [0027]    The invention originates from the observation that the objects cannot easily change or evolve to integrate new services, especially as a function of the user e.g. at the user&#39;s initiative. For instance, when an application cannot match the services it requires with the objects in its environment, it cannot apply its logic to the objects and therefore cannot fulfill its tasks. Therefore, when the objects present in the user&#39;s environment change, for instance when the user is moving from a place to another place, the continuity of the service may not be provided for the application as some of the services may become unavailable. 
         [0028]    Therefore, an idea on which the invention is based is to provide a method which consists in detecting in the objects of the users environment the missing services which are needed to launch an application, and automatically update the objects so as the previously missing services become available in order to insure the operability of the application. Therefore, the method allows to dynamically deal with changes in the user&#39;s environment by providing missing required services to the objects so as to make an application operational. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0029]    These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter, by way of example, with reference to the drawings. 
           [0030]      FIG. 1  is a schematic representation of a general architecture of a system in which an embodiment of the invention may be implemented. 
           [0031]      FIGS. 2 to 4  are schematic representations of successive points in time each time in two distinctive places when a method according to an embodiment of the invention is employed on a communication terminal. 
           [0032]      FIG. 5  is a flowchart illustrating the method employed in  FIGS. 2 to 4 . 
       
    
    
     DETAILED DESCRIPTION OF THE EMBODIMENTS 
       [0033]      FIG. 1  illustrates a communication terminal  1  controlled by a user  13  and connected to a network  2 . More precisely, the communication terminal  1  comprises a communication interface  3  which allows it to communicate with the network  2 . The communication terminal  1  may be a mobile terminal such as a mobile phone i.e. a smartphone or a notebook, and the communication interface  3  may then be a radio interface, i.e. Wifi, 3G, LTE, etc. 
         [0034]    A plurality of physical objects  4 , especially a television  14  and a lamp  15 , are connected to the communication network  2 . For that purpose, the physical objects  4  also comprise communication interfaces  5  connected to the network  2  through an interface device which is built directly in the physical objects  4 . Alternatively, an interface device may form a gateway between a plurality of physical objects  4  and the communication network  2 . 
         [0035]    The physical objects  4  may be of different nature. For instance, the physical objects  4  may be devices such as advertisement boards, televisions, lamps, household appliances, communication terminals, fridges, cameras, media drives, information displays etc. These physical objects  4  may be present in the user&#39;s home but also in public environments or other locations, for instance the user&#39;s workplace. 
         [0036]    The communication network  2  is generally a composite network that may comprise different kinds of networks e.g. a private wired network, a public network, a radio network etc. 
         [0037]    The physical objects  4  offer a plurality of services  8  via their network interface  5 . A service  8  is a function offered by a physical object  4  which is available through the “web of things” and therefore may be used or activated through the user&#39;s terminal  1  or by another physical object  4 . For instance, the Television  14  may offer several services  8  comprising changing of the channel, changing the luminance, changing the volume, displaying a picture or a videostream, etc. With this panel of services it is possible to remotely control the Television. 
         [0038]    Similarly, the lamp  15  provides services. For instance, the lamp provides a switch-on/switch-off service. Furthermore, the lamp  15  comprises hardware  6  able to change the light and the color of the lamp  15 . The switch-on/switch-off service is enabled by the capability of the lamp  15  to change the light. More precisely, a switch-on/switch-off module  17  enables turning the lamp  15  on or off based on instructions emitted by the user&#39;s terminal  1 . However, even if the lamp  15  already integrates a switch-on/switch-off module providing a switch-on/switch-off service, the lamp  15  may further evolve to propose new services. 
         [0039]    For that purpose, each physical object comprises an extension module  7 . This extension module  7  comprises an information storage in which a software module may be stored. When stored on a physical object, a software module allows the physical object  4  to provide a new service. The new service is accessible to the user&#39;s terminal  1  through the interfacing between the user&#39;s terminal  1  and an interface point of the software module. 
         [0040]    For instance, the software module may be a software module allowing the interaction between an external object, such as the user terminal  1 , with the hardware  6  of a respective physical object  4 . For that purpose, on a first end the software module comprises an interface points which allows receiving and/or emitting a datastream from or to an interface point of the user&#39;s terminal  1 , or from or to an interface point of another physical object  4 . On the other end the software module provides commands and/or receives data from the hardware  6  of the physical object  4 . 
         [0041]    A specific example of a new service that may be implemented in the lamp  15  is a blink service. This blink service makes the lamp  15  blink at a specific sequence of “on” and “off”. Therefore, a software module may be installed on the information storage of the lamp  15 , and provide the blink service on demand. This service allows the lamp to provide new capabilities or functionalities associated to its hardware. However, another kind of service may also be data stored in the object  6 . 
         [0042]    The communication terminal  1  comprises a user interface which allows the user to execute and/or create applications stored in a storage device of the communication terminal  1 . 
         [0043]    Each application needs one or more service  8  provided by the physical objects  4  to fulfill its tasks. Therefore, when the application is executed or active, the user&#39;s terminal  1  communicates with the physical objects  4  and employs the offered services notably through the software modules of the physical objects  4 . For instance, when executed, a “turn on the lamp” application requires the use of the switch-on/switch-off module of the lamp. 
         [0044]    However, the services required by the application may not be initially available on the physical objects  4  present in the user&#39;s environment. 
         [0045]    Indeed, the required services may not have been expected in the manufacturing phase of the physical object  4 . Therefore, in order to interface the application with the surrounding physical objects  4  so as to be able to execute said application, the communication terminal  1  is able to initiate the implementation of an additional software module in one of the appropriate physical object  4  in which the required service is lacking. 
         [0046]    As a specific example, the user&#39;s terminal comprises an “ambilight” application. When executed in an environment with the lamp  15  and the television  14 , the ambilight application makes the lamp  15  communicating with the television  14 , in a transparent way, for calculating the dominant color of the picture of the television  15  when the user watch a movie on this television  15 . The lamp  15  changes then the color of the user environment according to the dominant color. However, the above mentioned lamp  15  does not initially offer a service that provides a functionality of changing the light color as a function of the screen dominant color, as well as a functionality that provides a physical object to physical object communication which will initiate the communication the dominant color from the television  14  to the lamp  15 . 
         [0047]    Therefore, the communication terminal  1  automatically implements a software module in the lamp  15  which will enable to provide this ambilight service to the ambilight application. More precisely the software module will communicate with the television  14  in order to receive a datastream comprising the dominant color. With the dominant color, the software module will give instructions to the hardware  6  to change the color of the lamp  15  accordingly. 
         [0048]    This implementation may be made at the moment of the execution of the application. Alternatively, the implementation may be achieved as a background protocol which continuously upgrades the physical object  4  so as to enable the execution of the application in the user&#39;s terminal  1  in its environment. The automatic implementation of the required services in the physical objects  4  enables a continuity of the operation of an active application when the user&#39;s environment changes. 
         [0049]    In order to manage and create applications, the communication terminal  1  comprises an Activity Creation Module  20  (AC-M) and an Activity Management Module  21  (AM-M). 
         [0050]    The Activity Creation Module  20  allows the user to create extended applications, also called oriented activities that will use one or more object&#39;s services and may also use applications when executed. The creation of such an extended application allows the user to organize several applications and services in one same extended application. This extended application may be executed as a background application on an extended period of time. 
         [0051]    More precisely, an extended application is a meta-data package which may contain applications references, a logic description and at least one service extension. 
         [0052]    The logic description specifies the logical relations between the applications and the services extensions in the extended application. 
         [0053]    A service extension makes references to a service to be used in the surrounding physical objects  4  and may be for instance associated to an application programming interface (API) which corresponds to a software interface which will allow the application to communicate with the physical objects  4  through the interface points. 
         [0054]    In other words, the extended application corresponds to a services profile which indicates the necessary services in order to execute the application and comprises a logic that organizes the services and applications. The service extensions are identified by their semantic description. 
         [0055]    The user  13  creates the extended application by adding one or several applications and/or one or several services extension. 
         [0056]    The service extensions may be found by queering an extension database  10  and the applications may be retrieved in an application database  9  which may be external to the user&#39;s terminal  1 . When an application is retrieved from the application database  9 , services extensions are added to the oriented activity and linked to the application according to the needs of the application i.e. the interface point of the respective application. More precisely, these needs may be indicated by a semantic description of the respective application, and the appropriate service extensions may be identified by the user&#39;s terminal from the extension database  10  according to these needs. 
         [0057]    During the creation of the oriented activity, the user  13  may orchestrate the execution of the called applications and services by configuring a logical scheme that will organize the services and the application as expected by the user during its execution. 
         [0058]    For instance, in  FIG. 3 , an application  53  comprises three service extensions F1, F2 and F3, and one application A1 which are interrelated by logical relations I1, I2, I3. 
         [0059]    The Activity Management Module  21  enables the user  13  to access and execute the extended applications. Therefore, the Activity Management Module  21  allows the user to manually execute an extended application or executes automatically the extended application as a function of the current time. In other words, the extended application may be automatically activated by the communication terminal  1  when the current time is in a specified timeslot predefined by the user through the Activity management module  21 . Moreover, several extended application may be active at a same time. 
         [0060]    The Activity Management Module  21  retrieves the extended applications which are for instance created by the user in an internal database  12  of the communication terminal  1  or accesses it by querying a remote database  11 . In the case of a remote database  11 , when the application is selected, the application is downloaded and stored on the communication terminal  1 . 
         [0061]    In order to control the execution of the extended applications, the communication terminal  1  further comprises an execution module  22  (EXE-M) which ensures the execution and the organization of the applications and the services according to the services extensions and of the extended application and according to the respective control logic of the extended applications. 
         [0062]    For the purpose of upgrading the appropriate physical objects  4  of the user environment so as to enable the execution of the extended applications, the communication terminal  1  comprises a object overview module  18  (OFO-M), and a missing services collector module  19  (MFC-M). 
         [0063]    The object overview module  18  analyses the environment of the user  13  in order to list a set of available physical objects  4  in this environment and determine the services offered by these physical objects  4 . 
         [0064]    An external database may reference the available physical objects  4 . This database can associate the objects  4  and their geographical position. For instance, the database can contain data associating the identifier of the physical objects  4  with their characteristics, state, hardware capabilities, provided services, type of object and geographical position. These data are stored as a description associated to each physical object  4 . The description is for instance in XML language and may also be called a semantic description. Notably, the geographical position may be latitude and longitude coordinates. The set of physical objects  4  may be determined dynamically by the localization of the communication terminal  1 . In other words, the computing means in the terminal  1  may continuously refresh, through a background protocol the set of physical objects  4  as a function of the localization of the communication terminal  1 . 
         [0065]    The object overview module  18  determines the missing services in the set of physical objects  4  according to a list of offered services in the environment determined by the Object overview module  18 . 
         [0066]    The missing services collector module  19  then implements software modules enabling the provision of the required services on the appropriate physical objects  4 . 
         [0067]    The software modules are provided from the service extension database  10  or from a local database. 
         [0068]      FIGS. 2 to 4  illustrate the state of the physical objects  4  and communication terminals  1  and  52  in several successive moments in two separate places. During these successive moments, the operation&#39;s continuity of extended applications  53  and  54  is enabled by the automatic implementation of software modules. 
         [0069]      FIG. 2  corresponds to an initial state in the two different places. Each place corresponds to a logical identification of a set of neighboring physical objects  4 : 
         [0070]    A first place  50  corresponding to the user&#39;s living room comprises three different physical objects  4 : a video camera O1, a Television O2 and a Hi-Fi system O3. 
         [0071]    A second place  51  corresponding to the user&#39;s kitchen comprises two different physical objects  4 : a baby phone O4 and a screen on a fridge O5. 
         [0072]    In this initial state a set of services are already available from the several physical objects  4  present in the two places and are illustrated by the schematized circles. For instance, the television O2 offers two different services schematized by two different patterns, the X pattern corresponding to a ShowVideoStream services  55 . 
         [0073]    In  FIG. 3 , a user U1 arrives at the first place  50  and activates a video call application  53 , which is an extended application, by means of the communication terminal  1 . The video call application  53  makes a video call using the surrounding physical objects  4  with an automatic capture of the minutes, and the displaying of the videocall on a screen as a function of the behavior of the user  13 . Especially, when the user looks into the direction of the television O2, the video call is displayed on the television O2. This application  52  contains three services extensions F1, F2, F3 that indicate the services to be provided by the surrounding physical objects  4  that will affect their behavior or state. 
         [0074]    More precisely, F1 corresponds to a speechToText function that requires the provision of the minutes as a function of the soundtrack. F2 corresponds to a “presence sensor” function and may be fulfilled by an eyeTracking service allowing a physical object recording a video to monitor the attention of the user and the direction of his field of view. F3 indicates a function requiring the video call to be broadcasted to a screen of a physical object  4 . 
         [0075]    The service extensions F1, F2 and F3 are preferably associated in the extended application semantic description to semantic values. The object overview module  18  of the user&#39;s terminal  1  analyses the surrounding physical objects  4  in order to determine if the objects provide services matching the service extensions of the application or equivalents services by comparing the semantic values of the service extensions and the services provided by the surrounding objects. 
         [0076]    An association of a service extension and a provided service can be made when the corresponding semantic value of the provided services and required services present an important semantic correlation. 
         [0077]    The semantic values may be the identifier of the service or complementary keywords. Therefore, the correlation may be made directly on the identifier of the service or complementary keywords associated to the service. Similarly, the correlation can be based on the identifier of the service extension or on keywords associated to the service extension. For instance, the association can be made if the service extension and the provided services present an identical Keyword or identifier. 
         [0078]    However, other methods may be employed and can take into account the fact that a semantic value may be more generalized than another. For instance the service may have a “display picture” semantic value and the application may have a “display photograph” semantic description. Such correlations may be then detected by the user&#39;s terminal  1  which then associates of the application with the respective physical object  4 . A correlation may also be detected by the user&#39;s terminal when two semantic values are synonyms. 
         [0079]    It may also be required to make a finer correlation. For that purpose, the computation means of the communication terminal  1  may use an ontology. This ontology may be accessible on the web, or can be local. The ontology may evolve to integrate new services, new evolutions and new devices. The use of an ontology allows to correlate similar concepts: for instance “scintillate” may be correlated with “blink”. 
         [0080]    Turning back to the example, only the television O2 provides one of the required services, which is the ShowVideoStream service  55 . As two services are missing in the environment corresponding to the first place  50 , the missing services  56  and  57  are retrieved from a database, in order to guarantee the execution of the application. Alternatively, the software modules are directly packaged into the activity as a software package. 
         [0081]    The appropriate software modules corresponding to missing services  56  and  57  are installed onto the appropriated physical objects  4  by the missing services collector module  19  so as to offer the required services: speechToText service  56  is installed onto the Hifi System O3 and an EyeTracking service  57  is installed onto the video camera O1. 
         [0082]    The execution module  22  then executes the video call application  53  according its control logic. 
         [0083]    In  FIG. 4 , the user U1 has moved to the second place  51  while its video call application  53  is still active. The user&#39;s environment and more precisely the surrounding objects have completely changed. 
         [0084]    Therefore, the object overview module  18  and the missing services collector module  19  fulfill the automatic update of the surrounding objects O4 and O5. The object overview module  18  identifies that the service extension F2 is already available by object O5. 
         [0085]    The missing services collector module  19  installs the missing services F1 and F3 onto the two objects O4 and O5. The application can therefore continue its tasks and therefore the continuity of the application is insured. 
         [0086]    Meanwhile, a second user U2 arrived at the first place  50  and uses a application  54  that comprises the service extensions F2, F4 and F5. The service extension F2 requires the service  57 , the service extension F4 requires the service  59  and the service extension F5 requires the service  58 . Therefore, the object overview module  18  identifies that the service required by service extensions F5 and F2 are already available on the objects O1 and O2. However, the service required by the service extension F4 is absent from the surrounding physical object O1, O2 and O3. The missing services collector module  19  installs the missing service  59  onto the object O1. Therefore, the application  54  takes advantage of the previously installed software modules gained through the video call application  53  of the first user U1 as it uses the service  57  added by the communication terminal  1  for the video call application  53 . 
         [0087]    In embodiments the installed software modules can be automatically uninstalled from an updated object of a place after the user has left the place. For that purpose the communication terminal  1  determines outdated physical objects which are not in his environment anymore and on which a software was installed, and then triggers the uninstall of the software module. Alternatively the physical object  4  may automatically uninstall the software module after a certain time. 
         [0088]    A method for upgrading the physical objects according to the needs of the application will now be described with reference to  FIG. 5 . 
         [0089]    In a first step  30 , the object overview module  18  analyses the user environment in order to find the objects in the user&#39;s environment. In a first test  31 , the object overview module  18  verifies if objects are present in the environment. If no object is present the process is stopped. 
         [0090]    If objects are detected by the object overview module, the object overview module generates in a resources listing step  32  a set of objects according to the user environment. The object overview module  18  further lists the available services provided by the set of objects. For that purpose, the object overview module  18  analyses the semantic descriptions of the set of objects. 
         [0091]    In a required services listing step, the object overview module  18  generates a list of required services according to the application executed by the user. For that purpose, each application is associated to a semantic description which describes notably the services which need to be interfaced with the application. The object overview module  18  compares the semantic descriptions of the provided services and the semantic description of the required services. 
         [0092]    If all required services are provided by the objects, the application is launched by the execution module in a step  40 , otherwise the object overview module  18  makes a list of the services which do not have a match in the objects of the user environment. 
         [0093]    In a further step  36 , the object overview module  18  determines the types of objects on which may be implemented the missing features by generating a table associating the missing services with the types of objects that may implement the missing services. For that purpose, the object overview module  18  retrieves missing service requirements for each missing service. The missing service requirement may be retrieved from the extension database  10  or may be directly indicated in the service extension description. The object overview module  18  analyses the semantic description of the physical objects  4  in order to identify the hardware capabilities of the physical objects  4 . The hardware capabilities of each object  4  are then compared with the hardware requirements of each of the missing services so as to determine if the physical object  4  belongs to a type compatible with a missing service. 
         [0094]    If no matching physical object may be associated to a missing service, the method stops. Else, the test  37  is followed by a step of installing, on the matching physical object, an appropriate software module  38  which is able to use the hardware capabilities so as to provide the missing service. 
         [0095]    In other embodiments, the objects overview module may identify the drivers references, or the product and manufacturer identifier of a physical object in the semantic description in order to determine if the physical object and the missing service are compatible. The data about a physical object which is analyzed may be initially stored in the external database referencing the physical objects  4  or in a data storage in said physical object  4  itself. 
         [0096]    For that purpose the services collector module  19  retrieves the software module adapted to provide the missing service with the matching object from a software database which may be an external software database on internal to the communication terminal  1 . Alternatively packaged software modules which are already associated to the application may be installed. The communication terminal initiates the downloading from the database to the physical object  4  of the software module and its installation on the physical object. 
         [0097]    In a further test  39 , the services collector module  19  verifies if the software module is successfully installed. If the software module is successfully installed, the application may be launched by the execution module  22  in a next step  40 , therefore activating or using the services of the physical objects  4 . 
         [0098]    The invention is not limited to the described embodiments. The appended claims are to be construed as embodying all modification and alternative constructions that may be occurred to one skilled in the art, which fairly fall within the basic teaching here, set forth. The use of the verb “to comprise” or “to include” and its conjugations does not exclude the presence of elements or steps other than those stated in a claim. Furthermore, the use of the article “a” or “an” preceding an element or step does not exclude the presence of a plurality of such elements or steps. The invention may be implemented by means of hardware as well as software. The same item of hardware may represent several “means”. 
         [0099]    In the claims, any reference signs placed between parentheses shall not be construed as limiting the scope of the claims.