Mutual interactivity between mobile devices based on position and orientation

A method for initiating a gesture-based mutual interaction scheme between a first and second mobile device, comprising: associating a gesture-based mutual interaction scheme between the first and second mobile device that associates a position scheme with a respective action, where the position scheme relates to any of: an absolute or relative orientation, and an absolute or relative trajectory; acquiring a first position property of the first mobile device and a second position property of the second mobile device; determining that each of the first and second position properties comply with the position scheme; triggering an execution of an action on the second mobile device, where the action is associated with the position scheme that the first position property complies with; where the second mobile device conditions the execution of the action triggered by the first mobile device on the compliance of the second position property with the position scheme.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a 35 U.S.C. § 371 national stage application of PCT/IL2018/051324 filed Dec. 3, 2018, entitled “Mutual Interactivity Between Mobile Devices Based on Position and Orientation”, which claims priority to Application No. IL 256288 filed Dec. 7, 2017, both of which are incorporated herein in their entirety.

FIELD OF THE DISCLOSED TECHNIQUE

The disclosed technique relates to remote interaction between mobile devices in general, and to orientation and position dependent remote interaction, in particular.

BACKGROUND OF THE DISCLOSED TECHNIQUE

Mobile devices enable individuals to interact using platforms such as Android®, and OS®, each platform having respective different, and often proprietary applications with different interfaces. Typical communication platforms include email, voice/audio communications, and social networking. Creating a link between two or more devices for any of these communications platforms typically requires use of a touch screen, button, or voice command. Additionally, connecting with a prospective contact requires identifying the prospective contact in an address book, or other contact list.

U.S. Pat. No. 8,433,244 B2 to Liu et al, entitled “Orientation based Control of Mobile Device”, directs to controlling a feature of a non-navigation related application, such as the arrangement of menu items, function controlled by a quick launch key, based on the orientation of the device. The controls may be context-specific such that the orientation is applied to control the feature only in certain contexts. Context may be implied from various factors such as time and date information, the location of the device, and the proximity of the device to other devices.

U.S. Pat. No. 7,379,563 B2 to Shamaie et al, entitled “Tracking Bimanual Movements”, directs to recognizing and tracking bimanual movements in the presence of occlusion. The tracking of two separate hands is acquired before the occlusion, and reacquired after the occlusion. Occlusion may be attributed to hand gestures, i.e. changing a hand shape may occlude one or more fingers. Alternatively, occlusion may be caused by one hand blocking the other.

U.S. Pat. No. 7,058,204 B2 to Hildreth et al, entitled ‘Multiple Camera Control System’, directs to tracking an object using two different viewpoints obtained using two cameras. An absolute position of the object is computed from the difference between each viewpoint and a background.

SUMMARY OF THE PRESENT DISCLOSED TECHNIQUE

It is an object of the disclosed technique to provide a novel method and system for initiating at least one mutual interaction scheme between a first mobile device and at least a second mobile device.

In accordance with the disclosed technique, there is thus provided a method comprising the procedures of: associating at least one mutual interaction scheme between the first mobile device and the at least second mobile device, the mutual interaction scheme associating at least one position scheme with at least one respective action, the at least one position scheme relating to at least one of: an absolute orientation, a relative orientation, an absolute position, a relative position, an absolute trajectory, and a relative trajectory; acquiring a position property of the first mobile device; determining that the acquired position property of the first mobile device complies with at least one of the at least one position scheme of the mutual interaction scheme; and triggering an execution of one of the at least one respective action on the at least second mobile device, the respective action triggered on the second mobile device associated with the at least one position scheme with which the position property of the first mobile device complies, in accordance with the mutual interaction scheme.

In some embodiments, the method further comprises executing on the first mobile device, one of the one or more respective actions in response to the determined compliance of the acquired position property of the first mobile device.

In some embodiments, the acquired position property of the first mobile device relates to at least one of: an absolute orientation, a relative orientation, an absolute position, a relative position, an absolute trajectory, and a relative trajectory of the first mobile device.

In some embodiments, the acquiring the position property of the first mobile device, and the determining that the acquired position property of the first mobile device complies with one of the at least one position scheme of the mutual interaction scheme is performed by the first mobile device.

In some embodiments, the method further comprises executing, by the at least second mobile device, the one of the one or more respective actions triggered by the first mobile device.

In some embodiments, the method further comprises the procedures of: acquiring at least a position property of the at least second mobile device; determining that the acquired position property of the at least second mobile device complies with at least one of the at least one position schemes of the mutual interaction scheme.

In some embodiments, the acquired position property of the at least second mobile device relates to at least one of: an absolute orientation, a relative orientation, an absolute position, a relative position, an absolute trajectory, and a relative trajectory of the at least second mobile device.

In some embodiments, the acquisition of the position property of the at least second mobile device, and the determining of the compliance of the position property of the at least second mobile device is performed by the at least second mobile device.

In some embodiments, the method further comprises notifying the first mobile device of the compliance of the position property of the at least second mobile device.

In some embodiments, the acquisition of the position property of the at least second mobile device, and the determining of the compliance of the position property of the at least second mobile device is performed by the first mobile device.

In some embodiments, the method further comprises conditioning the executing, by the at least second mobile device, the one of the one or more respective actions triggered by the first mobile device, on the determined compliance of the position property of the at least second mobile device.

In some embodiments, the method further comprises: triggering an execution of one of the one or more respective actions on the first mobile device, the respective action triggered on the first mobile device associated with the at least one position scheme with which the position property of the at least second mobile device complies, in accordance with the mutual interaction scheme.

In some embodiments, the method further comprises: executing, by the first mobile device, the one of the one or more respective actions triggered by the at least second mobile device.

In some embodiments, one of the one or more respective actions comprises indicating the determined compliance of the acquired position property of the first mobile device.

In some embodiments, the one of the one or more respective actions comprises indicating the determined compliance of the position property of the at least second mobile device.

In some embodiments, the one or more respective actions comprises any of: registering and unregistering any of the first mobile device and the at least second mobile device to the mutual interaction scheme, in response to the determined compliance of the acquired position property of the first mobile device.

In some embodiments, the one or more respective actions comprises any of: registering and unregistering any of the first mobile device and the at least second mobile device to the mutual interaction scheme, in response to the determined compliance of the position property of the at least second mobile device.

In some embodiments, the acquired position property of the first mobile device corresponds to a distance between the first mobile device and the at least second mobile device, wherein the at least one position scheme of the mutual interaction scheme correspond to the acquired distance, wherein determining further comprises determining that the acquired distance complies with the position scheme of the mutual interaction scheme corresponding to the acquired distance.

In some embodiments, the at least one respective action associated with the position scheme of the mutual interaction scheme corresponding to the acquired distance comprises indicating the acquired distance at any of the first mobile device and the at least second mobile device.

In some embodiments, the position property of the first mobile device corresponds to a distance and orientation of the first mobile device with respect to an audio receiver, and wherein the position property of the at least second mobile device corresponds to a distance and orientation of the at least second mobile device with respect to the audio receiver, wherein the at least one position scheme of the mutual interaction scheme corresponds to an audio range with respect to the audio receiver, wherein determining further comprises determining that the position property of the first mobile device and the position property of the at least second mobile device all comply with the audio range, the method further comprising: synchronizing an internal clock of the at least second mobile device with an internal clock of the first mobile device, computing a phase shift for each of the at least second mobile device such that an audio file simultaneously transmitted by the first mobile device and each of the at least second mobile device constructively interferes at the audio receiver, and rendering the audio file by the first device, and rendering the audio file by each of the at least second mobile device according to each respective phase shift.

In some embodiments, the method further comprises performing the synchronizing, computing and rendering steps with respect to a plurality of mobile devices, grouping a first portion of the plurality of mobile device as a left speaker cluster, grouping a second portion of the plurality of mobile devices as a right speaker cluster, and rendering the audio file by the mobiles devices grouped as the left speaker cluster to emulate a left speaker, rendering the audio file by the mobiles devices grouped as the right speaker cluster to emulate a right speaker, thereby emulating a stereo loudspeaker at the audio receiver.

In some embodiments, the method further comprises mutually notifying each of the first mobile device and the at least second mobile device of the respective compliances of the acquired position properties of the first mobile device and the at least second mobile device with the at least one position scheme of the mutual interaction scheme, wherein executing the respective action comprises invoking a multi-perspective imaging application at each of the first mobile device and the at least second mobile device, wherein acquiring the respective position properties of the first mobile device and the at least second mobile device comprises capturing an image of an object with any of: a camera configured with the first mobile device and a camera configured with the at least second mobile device, wherein the acquired position properties of the first mobile device and the at least second acquired position property are with respect to the object, wherein the at least one position scheme of the mutual interaction scheme with which the position property of the first mobile device complies, corresponds to an optical range of the camera configured with the first mobile device with respect to the object, and wherein the at least one position scheme of the mutual interaction scheme with which the position property of the at least second mobile device complies, corresponds to an optical range of the camera configured with the at least second mobile device with respect to the object.

In some embodiments, invoking the multi-perspective imaging application comprises: capturing at least one image of the object, simultaneous with capturing the image, acquiring associated metadata, the associated metadata comprising position and orientation properties associated with the captured image, and a time stamp associated with the captured image, and providing the captured image and the associated metadata to an image processor.

In some embodiments, the method further comprises: receiving from the first mobile device and the at least second mobile device, the multiple captured images with the associated metadata, and processing the multiple captured images using the associated metadata to perform any of: creating a panoramic image of the object, creating a multi-perspective image of the object, and tracking the object.

In some embodiments, the method further comprises: depositing a visible trace, wherein acquiring the first position property comprises acquiring a trajectory corresponding to the visible trace, wherein determining that the acquired first position property complies with at least one of the at least one position scheme of the mutual interaction scheme comprises determining that the acquired trajectory corresponds to the deposited visible trace, and wherein triggering the at least second mobile device to execute the associated action comprises triggering the at least second mobile device to display a bit map corresponding to the visible trace.

In some embodiments, the method further comprises displaying the bit map corresponding to the visible trace at the at least second mobile device, and storing the bit map at a memory of the at least second mobile device.

In some embodiments, the triggered one of the at least one respective action on the at least second mobile device comprises exchanging a security credential between the first mobile device and the at least second mobile device.

In accordance with another aspect of the disclosed technique there is provided a mutually interactive system, comprising: a first mobile device comprising: an inertial measurement unit configured to acquire a position property of the first mobile device, a transceiver; and at least a second mobile device, each comprising: a transceiver configured to communicate with the transceiver of the first mobile device, the first and the at least a second mobile devices associated with at least one mutual interaction scheme associating at least one position scheme with at least one respective action, the at least one position scheme relating to at least one of: an absolute orientation, a relative orientation, an absolute position, a relative position, an absolute trajectory, and a relative trajectory, wherein the first mobile device is configured to: determine that the acquired position property of the first mobile device complies with one of the at least one position schemes of the mutual interaction scheme, trigger, via the respective transceivers of first mobile device and the at least second mobile device, the processor of the at least second mobile device to execute one of the at least one respective action, the triggered action associated with the one of the at least one position schemes with which the first acquired position property complies, and wherein the at least second mobile device is configured to execute the triggered action.

In some embodiments, the first device is configured to execute one of the one or more respective actions.

In some embodiments, the at least a second mobile device each further comprises an inertial measurement configured to acquire a position property of the at least mobile device.

In some embodiments, the at least second mobile device is further configured to determine that the acquired position property of the at least second mobile device complies with at least one of the at least one position schemes of the mutual interaction scheme.

In some embodiments, the at least second mobile device is configured to notify the first mobile device of the compliance of the acquired position property of the at least second mobile device, and provide the acquired position property of the at least second mobile device via the respective transceivers of the first device and the at least second mobile device.

In some embodiments, the respective acquired position property of the first mobile device and the at least second mobile device relates to at least one of: an absolute orientation, a relative orientation, an absolute position, a relative position, an absolute trajectory, and a relative trajectory of the respective first mobile device and the at least second mobile device.

In some embodiments, any of the first mobile device and the second mobile device further comprise a camera configured to acquire any of: the position property of the first mobile device, and a position property of the at least second mobile device, and wherein the first mobile device is further configured to determine that the acquired position property of the at least second mobile device complies with at least one of the at least one position schemes of the mutual interaction scheme, and wherein the at least second mobile device is further configured to determine that the acquired position property of the first mobile device complies with at least one of the at least one position schemes of the mutual interaction scheme.

In some embodiments, the at least second mobile device is further configured to condition the execution by the at least second mobile device of the one of the one or more respective actions triggered by the first mobile device, on the determined compliance of the acquired position property of the at least second mobile device.

In some embodiments, the at least second mobile device is further configured to trigger an execution of one of the one or more respective actions on the first mobile device, the respective action triggered on the first mobile device associated with the at least one position scheme with which the position property of the least second mobile device complies, in accordance with the mutual interaction scheme.

In some embodiments, one of the one or more respective actions comprises indicating the determined compliance of the acquired position property of the first mobile device.

In some embodiments, one of the one or more respective actions comprises indicating the determined compliance of the acquired position property of the at least second mobile device.

In some embodiments, one of the one or more respective actions comprises any of: registering and unregistering any of the first mobile device and the at least second mobile device to the mutual interaction scheme in response to the determined compliance of the acquired orientation of the first mobile device.

In some embodiments, one of the one or more respective actions comprises any of: registering and unregistering any of the first mobile device and the at least second mobile device to the mutual interaction scheme in response to the determined compliance of the acquired orientation of the at least second mobile device.

In some embodiments, the acquired position property of the first mobile device corresponds to a distance between the first mobile device and the at least second mobile device, wherein the at least one position scheme of the mutual interaction scheme corresponds to the acquired distance, wherein determining further comprises determining that the acquired distance complies with the position scheme of the mutual interaction scheme corresponding to the acquired distance.

In some embodiments, the first mobile device and the at least second mobile device are each configured to indicate the acquired distance in response to the determined compliance of acquired distance with the position scheme corresponding to the distance.

In some embodiments, the first mobile device is operative as an audio transmitter and further comprises a speaker, and wherein the at least second mobile device is operative as an audio receiver and further comprises a speaker, wherein the at least one position scheme of the mutual interaction scheme corresponds to an audio range with respect to an audio receiver, wherein the first acquired position property of the first mobile device corresponds to a distance and orientation of the first mobile device with respect to the audio receiver, and wherein the first mobile device is further configured to: receive at least a second position property corresponding to a distance and orientation of the at least second mobile device with respect to the first mobile device, determine that the acquired first position property and the at least second position property both comply with the audio range respective of the audio receiver, synchronizing an internal clock of the at least second mobile device with an internal clock of the first mobile device, computing a phase shift for each of the at least second mobile device such that an audio file simultaneously transmitted by the first mobile device and each of the at least second mobile device constructively interferes at the audio receiver, and rendering the audio file by the first device, and rendering the audio file by each of the at least second mobile device according to each respective phase shift.

In some embodiments, the acquired position properties of the first mobile device and the at least second mobile device are with respect to an object, wherein the at least one position scheme of the mutual interaction scheme with which the position property of the first mobile device complies, corresponds to an optical range of the camera configured with the first mobile device with respect to the object, and wherein the at least one position scheme of the mutual interaction scheme with which the position property of the at least second mobile device complies, corresponds to an optical range of the camera configured with the at least second mobile device with respect to the object, wherein the at least one respective action configured to be executed by the at least second mobile device is a multi-perspective imaging application, and wherein the first mobile device is configured to execute the multi-perspective imaging application.

In some embodiments, the system further comprises: an image processor, wherein executing the multi-perspective imaging application comprises: capturing at least one image of the object, simultaneous with capturing the image, acquiring associated metadata, the associated metadata comprising position and orientation properties associated with the captured image, and a time stamp associated with the captured image, and providing the captured image and the associated metadata to the image processor, wherein the image processor is configured to: receive from the first mobile device and the at least second mobile device, the multiple captured images with the associated metadata, and process the multiple captured images using the associated metadata to perform any of: creating a panoramic image of the object, creating a multi-perspective image of the object, and tracking the object, and provide the result of the processing step to any of the first and the at least second mobile device.

In some embodiments, the first mobile device is configured deposit a visible trace, and wherein the at least second mobile device is configured to: display a bit map corresponding to the visible trace at a user interface of the at least second mobile device, and store the bit map at a memory of the at least second mobile device, wherein determining that the acquired position property of the first mobile device complies with at least one of the at least one position scheme of the mutual interaction scheme comprises determining that the acquired position property of the first mobile device corresponds to the deposited visible trace, and wherein triggering the at least second mobile device to execute the associated action comprises triggering the at least second mobile device to display the bit map corresponding to the visible trace.

In some embodiments, the triggered one of the at least one respective action on the at least second mobile device comprises exchanging a security credential between the first mobile device and the at least second mobile device.

In accordance with another aspect of the disclosed technique there is provided a method for initiating at least one mutual interaction scheme between a mobile device and a computer, the method comprising the procedures of: associating at least one mutual interaction scheme between the mobile device and the computer, the mutual interaction scheme associating at least one position scheme with at least one respective action, the at least one position scheme relating to at least one of: an absolute orientation, a relative orientation, an absolute position, a relative position, an absolute trajectory, and a relative trajectory of the mobile device; acquiring a position property of the mobile device; determining that the acquired position property of the mobile device complies with at least one of the at least one position scheme of the mutual interaction scheme; and triggering an execution of one of the at least one respective action on a display of the computer, the respective action triggered on the display associated with the at least one position scheme with which the position property of the mobile device complies, in accordance with the mutual interaction scheme, wherein the at least one respective action comprises virtually coupling the mobile device with a three dimensional object displayed on the display, and manipulating a rendition of the three dimensional object on the display to correspond to the acquired position property of the mobile device.

In accordance with another aspect of the disclosed technique there is provided a mutually interactive system, comprising: a mobile device comprising: an inertial measurement unit configured to acquire a position property of the first mobile device, a transceiver; and a computer comprising: a display, a processor, and a transceiver configured to communicate with the transceiver of the mobile device, the mobile device and the computer associated with at least one mutual interaction scheme associating at least one position scheme with at least one respective action, the at least one position scheme relating to at least one of: an absolute orientation, a relative orientation, an absolute position, a relative position, an absolute trajectory, and a relative trajectory of the mobile device, wherein the mobile device is configured to: determine that the acquired position property of the mobile device complies with one of the at least one position schemes of the mutual interaction scheme, trigger, via the respective transceivers of mobile device and the computer, the processor of the computer to execute one of the at least one respective action, the triggered action associated with the one of the at least one position schemes with which the acquired position property complies, wherein the at least one respective action comprises virtually coupling the mobile device with a three dimensional object displayed on the display, and manipulating a rendition of the three dimensional object on the display to correspond to the acquired position property of the mobile device.

DETAILED DESCRIPTION OF THE EMBODIMENTS

The disclosed technique overcomes the disadvantages of the prior art by providing a system and method for implementing a position-and-orientation-dependent mutual interaction scheme between two or more mobile devices. The mutual interaction scheme defines at least one position scheme in association with at least one respective action. A first mobile device acquires a position property of the first mobile device, and compares the acquired position property to the position schemes of the mutual interaction scheme. If the acquired position property complies with one of the position schemes, the first mobile device triggers the execution of the respective action associated with the complied-with position scheme, on a second mobile device. Both the position scheme of the mutual interaction scheme, and the acquired position property of the mobile device are understood herein to relate to at least one of an absolute or relative orientation, position and trajectory of either one of the mobile devices. Similarly, the second device acquires a position property of the second mobile device, and compares the acquired position property of the second mobile device with the position schemes of the mutual interaction scheme. If the acquired position property of the second mobile device complies with one of the position schemes of the mutual interaction scheme, the second mobile device triggers the execution of the action associated with the complied-with position scheme on the first mobile device.

In this manner, the first mobile device can remotely influence the execution of one or more actions by the second mobile device, in response to manipulations of the respective spatial position and orientation of the first mobile device. Similarly, the second mobile device can remotely influence the execution of one or more actions by the first mobile device, in response to manipulations of the respective spatial position and orientation of the second mobile device. As a result, the two mobile device mutual interact in response to manipulating their respective spatial positions and orientations, precluding the need for touch-based interaction, such as typing, touching, swiping, and like. The mutual interaction scheme may be extended to three or more mobile devices, allowing multiple mobile devices to interact based on manipulating the respective positions, orientations and trajectories of the multiple mobile devices. Furthermore, the acquisition of the position properties and triggering of the respective actions is performed throughout in real-time, allowing the users of the respective mobile devices to interact in real-time, in response to manipulating the position and orientation of their respective mobile devices.

Reference is now made toFIGS. 1A-1Bwhich illustrate an exemplary implementation of two mobile devices150A and150B interacting via a mutual interaction scheme in accordance with an embodiment of the disclosed technique. Each of mobile devices150A and150B include hardware componentry and software for detecting and acquiring the respective positions, orientations and motion of mobile devices150A and150B. Additionally, each of Mobile devices150A and150B include a visual display152A and152B, and a respective transceiver154A and154B. These and other components of mobile devices150A and150B will be described in greater detail herein below in conjunction withFIGS. 2A-2G. Additionally, each of mobile devices150A and150B is registered to a mutually interactive application that associates mobile devices150A and150B with a mutual interaction scheme. In the example brought forth inFIGS. 1A and 1B, the mutual interaction scheme includes a rule associating a trajectory, acquired by one mobile device, with displaying the acquired trajectory at the other mobile device.

With reference toFIG. 1A, Alice, the user of mobile device150A, manipulates mobile device150A in a circular trajectory. Mobile device150A acquires the circular trajectory, and compares the acquired circular trajectory with the position schemes of the mutual interaction scheme. Mobile device150A determines that the acquired circular trajectory complies with the rule described above (i.e., displaying the acquired circular trajectory on mobile device150B). Accordingly, mobile device150A triggers mobile device150B to display the trajectory acquired by mobile device150A, on visual display152B of mobile device150B. With reference toFIG. 1B, mobile device150B displays on visual display152B an icon (star) associated with mobile device150A, moving in a circular trajectory corresponding to the circular trajectory acquired by mobile device150A.

Referring back toFIG. 1A, in a similar manner, Bob, the user of mobile device150B, manipulates mobile device150B in a triangular trajectory. Mobile device150B acquires the triangular trajectory, and compares the acquired triangular trajectory with the position scheme of the mutual interaction scheme. Mobile device150B determines that the acquired triangular trajectory complies with the rule described above. Accordingly, mobile device150B triggers mobile device150A to display the trajectory acquired by mobile device150B on visual display150A of mobile device150A. Referring back toFIG. 1B, mobile device150A displays on display152A of mobile device150A, an icon (a ‘+’ sign) associated with mobile device150B, moving in a triangular trajectory, corresponding to the triangular trajectory acquired by mobile device150B.

The term ‘mobile device’ refers herein below to any kind of computing device, which is intended to be operated while carried by an agent, such as a human user, an animal, an android, a vehicle, and the like. The vehicle can be an aircraft, a vessel, a ground vehicle (e.g., a car, a motorcycle, a bicycle, a Segway, a monocycle and the like). The mobile device is operative to compute and communicate with other devices via wireless means, without requiring cables, wires, and the like. Thus, the mobile device can be manipulated in space by the agent, such manipulations including at least one of spatial translations and rotations.

The disclosed technique shall be exemplified herein with reference to two devices. However, the disclosed technique may be implemented three devices as further explained below. Reference is now made toFIGS. 2A and 2B, which are schematic illustrations of mobile devices, generally referenced100A and100B respectively, constructed and operative in accordance with another embodiment of the disclosed techniques. Mobile devices100A and100B are provided by way of example, as a personal communication device (e.g., a mobile phone), and are representative of each of mobile devices150A and150B described above with respect toFIGS. 1A-1B.

Mobile devices100A and100B each include at least one respective processor134A and134B. Processors134A and134B may be any of a CPU, GPU, or APU, digital signal processor (DSP), and the like.

Each of mobile devices100A and100B additionally include a respective transceiver104A and104B. Each of transceivers104A and104B includes one or more of: a respective medium range RF transceiver (receiver and transmitter)106A and106B (e.g., WIFI), a respective short range transceiver (receiver and transmitter)108A and108B (e.g., Bluetooth), and a respective cellular communication transceiver (receiver and transmitter)114A and114B (e.g., GSM, LTE, WIMAX). Transceivers104A and104B are each operative to send and receive radio frequency (RF) signals relating to data and executable instructions.

Each of mobile devices100A and100B additionally includes a respective 3D accelerometer unit116A and116B, a respective gyroscope118A and118B, and a respective compass128A and128B, collectively referred to herein as MEMs. MEMs116A,118A, and128A are integrated within a single motion tracking component130A, such as a 9-axis inertial measurement unit (IMU) that provides real-time spatial tracking of at least one of translational and rotational motion of mobile device100A. Similarly, MEMs116B,118B, and128B are integrated within IMU130B, providing real-time spatial tracking of at least one of translational and rotational motion of mobile device100B. Each of mobile devices100A and100B additionally includes at least one respective memory store132A and132B.

Each of mobile devices100A and100B may additionally include any of the following optional components: respective user interfaces102A and102B; respective GPS receivers110A and1108; respective indoor GPS receivers112A and112B; respective speakers120A and120B; a respective microphones122A and122B; at least one respective camera124A and124B; and at least one respective optical emitter126A and126B.

Each of respective transceivers104A and104B are coupled to respective processors134A and134B via a respective converter, i.e. an analog to digital converters ADCs, and a digital to analog converter DACs, (not shown). Each of respective IMUs130A and130B, and respective memory stores132A and132B are coupled to respective processors134A and134B. Each of the respective optional components listed above, i.e. respective user interfaces102A and102B; respective GPS receivers110A and1108; respective indoor GPS receivers112A and112B; respective speakers120A and120B; respective microphones122A and122B; respective cameras124A and124B; and respective optical emitters126A and126B, when included within respective devices100A and100B, are coupled to respective processors134A and134B.

Cameras124A and124B may include any of: a visible light camera, an infra-red camera, an ultra-violet camera, a monoculer camera, a stereoscopic camera, a scanning camera, and combinations thereof. Optical emitters126A and126B can include one or more light illuminating modules, such as a light emitting diode (LED) visible light illuminator, an infra-red illuminator, an ultraviolet illuminator, a laser emitter, and the like.

Respective user interfaces102A and102B may be a touch-sensitive interface operative to display digital content, including one or more soft keys. Mobile devices100A and100B are operative to receive input via respective user interfaces102A and102B by sensing a touch on one or more of the displayed soft keys.

Processors134A and134B determine the absolute and relative position, orientation, and motion detection of respective mobile devices100A and100B according to at least one of: measurements acquired via respective IMUs130A and130B, images acquired via respective cameras124A and124B, measurements communicated via respective transceivers104A and104B, and measurements communicated via any of respective GPSs110A and1108, and respective GPSs112A and112B. For example, measurements received via any of respective transceivers104A and104B, respective GPSs110A and1108, and respective indoor GPSs112A and1128may be applied by respective processors134A and134B to determine position, orientation and motion of respective mobile devise100A and100B. As such, processors134A and134B may perform any of: triangulation calculations conducted with respect to multiple base stations (not shown) in communication with respective mobile devices100A and100B, calculations relating to signal flight technology; applications of calculations relating to electro-magnetic (EM) technology, and the like. Processors134A and134B may additionally apply one or more measurements received via respective IMUs130A and130B, and respective cameras124A and124B to further determine the position, orientation and motion estimations of respective mobile devise100A and100B determined above. Processors134A and134B are further operative to perform one or more calculations for determining the absolute and relative position, orientation, and motion detection of respective mobile devices100A and100B in conjunction with an operating system, such as Android, IOS, and the like, configured with respective mobile devices100A and100B.

In general, mobile devices100A and100B each receive translational motion information from respective 3D accelerometers116A and116B, rotational motion information from respective gyroscopes118A and118B, and changes in absolute orientation from respective compasses114A and114B. Additionally, mobile devices100A and100B periodically receive absolute positioning information from any of respective GPSs110A and1108, and respective indoor GPSs112A and112B. Respective processors134A and134B each apply the information received from respective compasses114A and1148, respective 3D accelerometers116A and116B, respective gyroscopes118A and188A, respective GPSs110A and1108, and respective indoor GPSs112A and112A to determine and update the position properties of respective mobile devices100A and100B using conventional techniques. Additionally, respective cameras124A and124B of respective mobile devices100A and100B may capture one or more images of a reference object, and the captured images may be analyzed to determine the respective position properties of mobile devices100A and100B at least relative to that reference object. Additionally or alternatively, an external processor (not shown) may be provided to compute updated position properties of mobile devices100A and100B. Additionally or alternatively, an external camera (not shown) may be provided to capture one or more images of mobile devices100A and100B, and the capture images may be used to derive one or more position properties of mobile devices100A and100B. Additionally or alternatively, respective cameras124A and124B of mobile devices100A and100B may capture an image of the respective other one of mobile devices100A and100B. These images may be analyzed by any of processors134A and134B to determine one or more position properties of mobile devices100A and100B.

The relative orientation of mobile device100A with respect to mobile device100B is defined by three angles of rotation (e.g., yaw, pitch and roll) between the respective mechanical frames of reference of mobile device100A and mobile device100B. In general, the mechanical frame of either of mobile devices100A or100B may be designated arbitrarily as a frame of reference. The relative position of mobile device100A with respect to mobile device100B is defined as a vector from the reference frame origin of mobile device100A to the reference frame origin of mobile device100B. The reference frame origin of mobile device100A may be chosen arbitrarily as well.

The reference frames for the position and orientation of each respective mobile device100A and100B are calculated by each of respective processors134A and134B. The respective reference frames of mobile device100A and100B are calculated based on measurements acquired by respective micro-accelerometers116A and116B, respective gyroscopes118A and118B, and respective magnetometers (compasses)128A and128B. Gyroscopes118A and1188measure rotational velocity of each respective reference frame, serving to smooth the dynamic effect on outputs from respective accelerometers116A and116B. Accelerometers116A and116B measures the respective gravity force and linear acceleration of respective mobile devices100A and100B: when in a static position, respective accelerometers116A and116B define the elevation and the roll of the mechanical frame of respective mobile devices100A and100B with respect to the vertical direction. Respective compasses128A and128B measure the local permanent magnetic field vector affecting each respective mobile device100A and100B. In an open area, the measured field coincides with the Earth's magnetic field. Inside buildings and urban environments, the magnetic field is distorted by nearby metallic objects. However, the magnetic field is smoothed due to the fact that the magnetic deformations are identical for closed points, and mobile devices100A and100B sense the same magnetic field, and hence deformations. Additionally, one or more signals received via respective transceivers104A and104B may be used to calculate the respective reference frames, such as to correct for drift, noise and the like.

In a steady, or quasi-steady state, respective compasses128A and128B and accelerometers116A and116B unambiguously define the orientation of the mechanical frame of respective mobile devices100A and100B with respect to the local magnetic and gravity fields. Thus, the relative orientations of mobile devices100A and100B located in proximity to each other may be determined from the respective orientations of each mobile device100A and100B with respect to the local magnetic and gravity fields.

In a dynamic state, respective accelerometers116A and116B measure the gravitational force, as well as any mechanical jitters imposed by the motion of respective mobile devices100A and100B. Respective gyroscopes118A and118B may filter out errors caused by respective accelerometers116A and116B, and provide the correct orientation of respective mobile device100A and100B.

If a metal or ferromagnetic object is positioned in proximity to any of mobile devices100A and100B, the magnetic field may be distorted, influencing the calculated orientation of the reference frame. In such a case, respective gyroscopes118A and118B temporary compensate for the distortion and provides the correct orientation of respective mobile devices100A and100B.

Reference is now made toFIGS. 2C-2Fwhich are schematic illustrations of techniques for determining relative or absolute position, orientation, or motion of mobile devices100A and100B, in accordance with a further embodiment of the disclosed technique and still referring toFIGS. 2A and 2B. The techniques described in conjunction withFIGS. 2C-2Fmay be employed separately, in combination, or in combination with additional techniques as are known in the art, to determine any of the relative or absolute positions, orientations, and trajectories of mobile devices100A and100B. The relative or absolute positions, orientations, and trajectories of mobile devices100A and100B may be determined for a range of distances separating mobile devices100A and100B, including local distance ranges, i.e. distances within direct detection via BlueTooth, optical detection, WiFi, and the like, and remote distance ranges, such as may require routing via cable, cellular, radio, or other long-range communication networks for detection. It is to be noted that the techniques described inFIGS. 2C-2Fare meant to be illustrative only, and not limiting. It may further be noted that although the description ofFIGS. 2C-2Frelate to two mobile devices100A and100B, the techniques may be extended to acquire any of a relative or absolute position, orientation, or motion of three or more mobile devices.

Reference is now made toFIG. 2C, which when taken additionally with reference toFIGS. 2A-2B, together illustrate a block diagram for determining a three-dimensional (3D) line of bearing (LOB) between mobiles devices100A and100B, constructed and operative in accordance with another embodiment of the disclosed techniques. By measuring the RES phase in respective receivers104A and104B using the techniques described above, and synchronizing the phase measurements with the accelerometer data obtained by respective IMUs130A and130B of each of respective mobiles devices100A and100B, the 3D line of bearing (LOB) can be estimated relative to the axes (X,Y,Z) of respective receivers104A and104B, accelerometers116A and116B, and compasses114A and116B of respective IMUs130A and130B of each of mobiles devices100A and100B. The respective axes (X,Y,Z) for each of mobile devices100A and100B may be calibrated in advance to correspond to each other.

Reference is now made toFIG. 2D, which when taken additionally with reference toFIGS. 2A-2B, illustrates a block diagram for short-term acquisition of parameters for mobiles devices100A and100B, constructed and operative in accordance with another embodiment of the disclosed techniques. IMUs130A and130B continuously acquire short-term position, orientation, and motion parameters for mobile devices100A and100B, respectively. Processors134A and134B of respective mobiles devices100A and100B use these parameters to calculate the direction of the relative motion between mobile devices100A and100B. By continuous gathering of data from respective IMUs130A and130B and by determining the residual phase (RES) for each of mobiles devices100A and100B detected via respective transceivers104A and104B, respective processors134A and134B can resolve any ambiguity of the wrapped phase, calculated above, and calculate the 3D LOB between mobiles devices100A and100B.

Reference is now made toFIG. 2E, which when taken additionally with reference toFIGS. 2A-2B, illustrates a block diagram for long-term gathering of parameters for mobiles devices100A and100B, constructed and operative in accordance with another embodiment of the disclosed techniques. IMUs130A and130B continuously acquire long-term position, orientation, and motion parameters for mobile devices100A and100B, respectively. Processors134A and134B of mobiles devices100A and100B use these long-term parameters to determine the respective location in 3D space for each of mobiles devices100A and100B. By gathering long term continuous data from respective IMUs130A and130B and from the respective RES phase sensors of each of each of mobiles devices100A and100B, respective processors134A and134B of each of mobiles devices100A and100B can apply known triangulation techniques to calculate the respective position of each of mobiles devices100A and100B.

Reference is now made toFIG. 2F, which when taken additionally with reference toFIGS. 2A-2B, illustrates a system for optical detection one of mobiles devices100A and100B by the respective other one of mobile devices100A and100B, constructed and operative in accordance with another embodiment of the disclosed techniques. Mobile device100A is operable to discover mobile device100B via camera124A. In one implementation, camera124A captures an image of mobile device100B, and processor134A analyzes the captured image, such as by applying object recognition techniques to discover mobile device100B. In another implementation, optical emitter126B of mobile device100B emits an identifying optical signal, such as an identifying pattern (i.e. structured light), color, or a series of pulses (i.e. Morse code) that identify mobile device100B. Camera124A of mobile device100A captures one or more images of the identifying optical signal, and processor134A analyzes the captured images to discover mobile device100B. The discovery may be limited detecting the presence of mobile device100B. Alternatively the discovery may additionally include determining any of a position, orientation, and trajectory of mobile device100B by mobile device100A. In a similar mobile device100B discovers mobile device100A via camera124B.

Additionally or alternatively, mobile device100A is operative to acquire a position property of mobile device100B via camera124B of mobile device100B. Mobile device100B is operative to capture one or more images indicating one or more position properties of mobile device100B via camera124B. For example, such images may include an image of a reference object, an image of a projection of a structured light pattern emitted by emitter126B of mobile device100B, and the like. In one implementation, mobile device100B transmits the image capture by camera124B to mobile device100A, via respective transceivers104A and104B, and mobile device100A analyzes the image to determine the position property of mobile device100B. In another implementation, mobile device100B analyzes the image to determine the position property of mobile device100B, and transmits the determined position property to mobile device100A.

Mobile devices100A and100B are additionally operative to acquire the position and orientation of any of the components included mobile devices100A and100B, such as the position and orientation of respective cameras124A and124B, optical emitters126A and126B, respective speakers120A and120B, and respective microphones122A and122B.

Additionally, or alternatively, each of mobile devices100A and100B may similarly use an identifying RF signal to discover and identify each respective other one of mobile devices100A and100B via respective transceivers104A and104B. Additionally, or alternatively, each of mobile devices100A and100B may similarly use an identifying acoustic signal, emitted via respective speakers120A and120B and detected by respective microphones122A and122B to discover and identify each respective other one of mobile devices100A and100B.

Additionally, or alternatively, each of mobile devices100A and100B may use an access point mode of discovery to discover and identify each respective other one of mobile devices100A and100B. In the access point mode of discovery, mobile device100A activates a mutual interaction application that scans for nearby devices and networks. Mobile device100B activates the mutual interaction application as well, allowing mobile device100B to be discovered by mobile device100A. The mutual interaction application may change the network name of mobile device100B to a known contact identifier, such as account ID, or a phone number, associated with mobile device100B. Mobile device100B transmits an identifying signal via transceiver104B of mobile device100B using an available WiFi or Bluetooth network. On receiving the identifying signal at transceiver104A of mobile device100A, mobile device100A identifies mobile device100B according to the known contact identifier, thereby discovering mobile device100B. The mutual interaction application displays the discovered contact at user interface102A of mobile device100A. The user of mobile device100A may select the displayed contact using known techniques, such as by clicking or touching, confirming the connection.

Once the connection is confirmed, inertial data acquired by respective IMUs130A and130B of each of mobile devices100A and100B are subsequently mutually exchanged via respective transceivers104A and104B of mobile devices100A and100B. Additionally, location information acquired by respective GPSs110A and110B, and indoor GPSs112A and112B of each of mobile devices100A and100B are subsequently mutually exchanged via respective transceivers104A and104B of mobile devices100A and100B.

Additionally, or alternative to employing an available WiFi or Bluetooth network, mobile devices100A and100B may discover each other via a web service installed on each respective mobile device100A and100B. Such web services include interactive applications such as WhatsApp, Messenger, chat services and the like.

Subsequent to the mutual discover of each of mobile devices100A and100B, mobile devices100A and100B may proceed to track each respective other one of mobile devices100A and100B. For example, the tracking may implemented using any of optical, RF, acoustic means, using conventional techniques.

Reference is now made toFIGS. 3A-3T, which are schematic illustrations of a system, generally referenced350, for implementing a mutual interaction scheme, constructed and operative in accordance with a further embodiment of the disclosed technique. In the description that follows, mobile devices300A,300B, and300C are understood to be operable to perform any of the procedures and calculations described above with respect toFIGS. 2B-2F. Referring toFIG. 3A, system350includes mobile device300A and at least mobile device300B, represented by respective mobile devices100A and100B described above with reference toFIGS. 2A-2F. System350may additional include a third mobile device300C or more mobile devices (not shown) represented by any of mobile devices100A and100B described above. In particular, mobile devices300A,100B, and300C communicate with each other via any of: respective transceivers304A,304B,304C corresponding to any of transceivers104A and104B; respective speakers320A,320B,320C corresponding to any of speakers120A and1204B; respective microphones322A,322B,322C corresponding to any of microphones122A and122B; respective cameras324A,324B,324C corresponding to any of cameras124A and124B; and respective optical emitters326A,326B,326C corresponding to any of optical emitters126A and126B, included therein. Mobile devices300A,300B, and300C may additionally communicate via a network352interfacing with respective transceivers304A,304B, and304C. System350may optionally include a server354. Mobile devices300A,300B, and300C may communicate with server354via respective transceivers304A,304B, and304C and network352.

Mobile devices300A and300B include respective IMUs330A and330B, corresponding to IMUs130A and130B described above with reference to mobile devices100A and100B ofFIGS. 2B-2F. Each of IMUs330A and330B includes respective compasses338A and338B, respective accelerometers326A and326B, and respective gyroscopes328A and328B, corresponding to respective compasses138A and138B, respective accelerometers126A and126B, and respective gyroscopes128A and128B described above with reference to mobile devices100A and100B ofFIGS. 2B-2F. Mobile devices300A and300B include respective GPSs310A,310B, and respective indoor GPSs312A and312B corresponding to respective GPSs110A,110B, and respective indoor GPSs112A and112B described above with reference to mobile devices100A and100B ofFIGS. 2B-2F. Mobile devices300A and300B include respective memory devices332A and332B, respective user interfaces302A and302B, and respective processors334A and334B corresponding to respective memory devices132A and132B, respective user interfaces102A and102B, and respective processors134A and134B described above with reference to mobile devices100A and100B ofFIGS. 2B-2F.

For the purpose of clarity, the description that follows with reference toFIGS. 3A-3Trelates to the operation of mobile device300A with respect to mobile device300B. However, any of the procedures performed by mobile device300A with respect to mobile device300B may be additionally and simultaneously performed by mobile device300B with respect to mobile device300A. Similarly, any procedures performed by mobile device300B with respect to mobile device300A may be additionally and simultaneously be performed by mobile device300A with respect to mobile device300B. Furthermore, any of the procedures performed by mobile device300A with respect to mobile device300B, may be additionally and simultaneously performed by mobile device300A with respect to mobile device300C. Similarly, any procedures performed by mobile device300B with respect to mobile device300A may be additionally and simultaneously be performed by mobile device300C with respect to any of mobile devices300A or300B, allowing the implementation of a fully mutually interactive scheme therebetween. It may further be noted that any of the procedures described hereinbelow may be implemented by server354in communication with any of mobile devices300A and300B.

Mobile device300A discovers mobile device300B via a mutual interaction application configured with each of mobile devices300A and300B using any of the discovery techniques described above with respect toFIGS. 2A-2F. The mutual interaction application may be received in advance at each of mobile devices300A and300B from server354.

Once each of mobile devices300A and300B have mutually discovered and identified each other respective mobile device300A and300B, mobile devices300A and300B register to a mutually interactive application, which associates a mutual interaction scheme therebetween, the description of which now follows.

Reference is now made toFIG. 3B, which illustrates an exemplary mutual interaction scheme360, in accordance with a further embodiment of the disclosed techniques. The mutual interaction application described above with reference toFIG. 3Aprovides each of mobile devices300A and300B with access to mutual interaction scheme360. For example, mobile devices300A and300B may receive mutual interaction scheme360from a cloud-sharing platform configured with server354via respective transceivers304A and304B and network352. Alternatively, mobile devices300A and300B may access mutual interaction scheme360via network352and respective transceivers304A and304B as a cloud-based resource stored at server354. Alternatively, one of mobile devices300A or300B may provide the other respective device with mutual interaction scheme360via respective transceivers304A and304B, either directly (i.e. BlueTooth), or via network352.

Referring back toFIG. 3B, mutual interaction scheme360associates at least one position scheme with at least one respective action, such as by defining one or more association rules. The associations between any of the position schemes and the respective actions may be one-to-one, one-to-many, many-to-one, or many-to-many. The positions schemes of mutual interaction scheme360relate to at least one of: an absolute orientation, a relative orientation, an absolute position, a relative position, an absolute trajectory, and a relative trajectory of any of mobile devices300A and300B.

One or more of the position schemes of mutual interaction scheme360may be static in that compliance with a static position scheme does not require detecting motion of mobile device300A. For example, compliance with a static position scheme may require stabilizing mobile device300A at a predefined orientation for a minimal time duration, or maintaining a predefined distance between two mobile devices. A static position scheme may specify an absolute orientation and position for mobile device300A, or a relative orientation and position for mobile device300A with respect to mobile device300B or another object. The first position scheme of mutual interaction scheme360illustrates an exemplary static position scheme: i.e. mobile device300A complies with the first position scheme when mobile device300A is stabilized at an absolute vertical orientation for two seconds.

One or more of the position schemes of mutual interaction scheme360may be dynamic in that compliance with the dynamic position scheme requires detecting a motion by mobile device300A. For example, compliance with a dynamic position scheme may require tracing a spatial trajectory with mobile device300A. The spatial trajectories include one or more translations and/or rotations by the mobile device, such as but not limited to: a linear translation of the mobile device in any of 360° of freedom, twisting, turning, and tilting of the mobile device about an internal axis of rotation, drawing a shape in space or forming a gesture with the mobile device, combinations thereof.

One or more of the dynamic position schemes may be path-independent where compliance require moving mobile device300A and terminating at a predefined orientation or position, regardless of the specific trajectory traced. The second position scheme of mutual interaction scheme360illustrates an exemplary path-independent dynamic position scheme, i.e. mobile device300A complies with the second position scheme when mobile device300A is moved in a trajectory that terminates in a relative orientation of 45° with respect to mobile device300B, irregardless of the specific trajectory.

One or more of the dynamic position schemes may be path-dependent, in that compliance requires tracing a specific spatial trajectory with mobile device300A. The third position scheme of mutual interaction scheme360illustrates an exemplary path-dependent dynamic position scheme, i.e. mobile device300A complies with the third position scheme when mobile device300A is moved in a “C”-shaped trajectory. Additionally, one of the position schemes of mutual interaction scheme360may be a combination of path-dependent and path-independent position schemes.

In one embodiment, mutual interaction scheme360is fully reciprocal. In a fully reciprocal scheme, every association rule applicable by mobile device300A is also applicable by mobile device300B, and thus each of mobile devices300A and300B are provided with the same set of association rules.

In another embodiment, mutual interaction scheme360is partially reciprocal in that one or more association rules applicable by mobile device300A is also applicable by mobile device300B, and one or more association rules are exclusively applicable only one of mobile device300A or mobile device300B. For example, mobile device300A may be associated with a paid subscription having access to a full set of features, corresponding to a complete set of association rules, and mobile device300B may be associated with a free subscription having access to only a partial set of features, corresponding to a subset of the association rules. Alternatively, each of mobile devices300A and300B subscribe to a different set of features, corresponding to one or more commonly shared, or reciprocated association rules, and one or more association rules exclusive to each of mobile devices300A and300B.

In a further embodiment, mutual interaction scheme360is exclusive in that mobile device300A is provided with a first set of association rules, and mobile device300B is provided with a second set of association rules, where the first and second sets of association rules are disjoint but complementary, resulting in a mutual interaction scheme formed by their combination. For example, mobile devices300A and300B may be configured in a master-slave framework.

The mutual interaction scheme360illustrated inFIG. 3B, defines multiple association rules numbered1through10. Each association rule is applicable to each of mobile devices300A and300B, each association rule at least associating a position scheme of one of mobile devices300A and300B with a respective action to be executed by the other of mobile devices300A and300B.

In the exemplary mutual interaction scheme360shown inFIG. 3B, compliance of mobile device300A with the first position scheme is associated with the action of sending a business card by mobile device300B, in accordance with the first association rule. Compliance by mobile device300A with the second position scheme is associated with the action of displaying an icon on user interface302B of mobile device300B, in accordance with the second association rule. Compliance of mobile device300A with third position scheme is associated with the action of invoking on mobile device300B an interactive chat session with mobile device300A, in accordance with the third association rule, and so on.

Reference is now made toFIGS. 3C-3D, which illustrate an exemplary implementation of the first rule of mutual interactive scheme360. Referring toFIG. 3C, a user (not shown) supports mobile device300A vertically for at least 2 seconds. Mobile device300A acquires one or more respective position properties relating to any of: an absolute position, a relative position, an absolute orientation, a relative orientation, an absolute motion or trajectory, and a relative motion or trajectory of mobile device300A by applying at least one of the techniques described above with reference toFIGS. 2A-2F. In this case, mobile device300A acquires the absolute vertical orientation of mobile device300A, stabilized for at least 2 seconds.

With reference now additionally made toFIG. 3B, on acquiring at least one position property of mobile device300A, mobile device300A compares the acquired position property with the position schemes defined by mutual interaction scheme360. Mobile device300A determines that the acquired position property complies with the first rule of mutual interaction scheme360, corresponding to a stabilized absolute vertical orientation of at least 2 seconds. Mobile device300A identifies the first association rule corresponding to the first position scheme with which the acquired position properties complies.

Accordingly, mobile device300A triggers on mobile device300B the execution of the respective action according to the identified association rule of mutual interaction scheme360. In this example, mobile device300A triggers mobile device300B to send mobile device300A a business card in accordance with the first association rule. Mobile device300A may perform the triggering by sending mobile device300B a notification to execute the respective action via respective transceivers304A and304B of mobile devices300A and300B. Additionally or alternatively, mobile device300A emits the notification as an optical signal via optical emitter326A to trigger mobile device300B, and mobile device300B receives the optical notification via camera324B. Additionally or alternatively, mobile device300A emits the notification as an acoustic signal via speaker320A to trigger mobile device300B, and mobile device300B receives the acoustic notification via microphone322B.

Mobile device300B executes the respective action triggered by mobile device300A. Thus, in the example ofFIG. 3C, mobile device300B responds to the notification, and sends the business card to mobile device300A, such as via respective transceivers304A and304B of mobile devices300A and300B.

In one implementation, mobile device300A remotely controls mobile device300B to execute the respective action via the notification. Alternatively, mobile device300B maintains control of the execution of the respective action indicated by the notification, and locally imposes one or more criterion, such as the compliance of mobile device300B with a position property of mutual interaction scheme360. In such a case, mobile device300B conditions the execution of the respective action on the fulfillment of the criterion.

According to another embodiment of the disclosed techniques, mobile device300A additionally determines the compliance of an acquired position property of mobile device300B with one of the position schemes of mutual interaction scheme360. The position property of mobile device300B relates to at least one of: an absolute orientation, a relative orientation, an absolute position, a relative position, an absolute trajectory, and a relative trajectory of mobile device300B.

Reference is now made toFIGS. 3E-3F, which illustrate an exemplary implementation of the second rule of mutual interactive scheme360ofFIG. 3Bin accordance with this embodiment. The user (not shown) moves mobile device300A in a trajectory, indicated by arcs220. Mobile device300A detects the motion, and acquires the position property of mobile device300A, as described hereinabove with reference toFIGS. 2A-2F. In order to determine compliance with the second position scheme of mutual interactions scheme360, mobile device300A is required to determine the relative orientation of mobile device300A with respect to mobile device300B, and thus additionally requires one or more position properties of mobile device200B. Mobile device300A acquires the one or more position properties of mobile device300B, using any of the techniques described hereinabove with reference to mobile devices100A and100B ofFIGS. 2A-2F. Mobile device300A applies the combination of the acquired position properties of mobile devices300A and300B to determine that the relative orientation therebetween is 45°.

With reference now made additionally toFIG. 3B, mobile device300A compares the determined relative orientation between mobile devices300A and300B to the position schemes of mutual interaction scheme360. Accordingly, mobile device300A determines that the relative orientation of 45° between mobile devices300A and300B complies with the position scheme corresponding to the second rule of mutual interaction scheme360. Mobile device300A triggers the execution of the respective action accordingly, by sending of a notification via respective transceivers304A and304B. Additionally or alternatively, the notification may be sent via any of optical emitter326A optically coupled with camera324B, and speaker320A acoustically coupled with microphone322B. In this example, the respective action is to display an icon362B identifying mobile device300A on user interface302B of mobile device300B.

Referring toFIG. 3F, mobile device300A sends the notification to mobile device300B, thereby triggering the execution of the respective action. Mobile device300B executes the triggered action and displays icon362B (a star) identified with mobile device300A, on display302B of mobile device300B, accordingly. In another implementation, mobile device300B independently determines that the relative orientation between mobile device300B and mobile device300A complies with the second position scheme of mutual interaction scheme360before displaying icon362B on display302B of mobile device300B.

Reference is now made toFIGS. 3G-3H, which together illustrate an exemplary implementation of the third rule of mutual interactive scheme360, in accordance with a further embodiment of the disclosed techniques. The user (not shown) moves mobile device300A in a “C” shaped trajectory364, shown inFIG. 3G. IMU330A of mobile device300A acquires the traced namely the “C” shaped trajectory. With reference made additionally toFIG. 3B, mobile device300A compares the trajectory to the position schemes of mutual interaction scheme360. Mobile device300A determines that the acquired “C” shaped trajectory complies with the third, dynamic, path-dependent position scheme of mutual interaction scheme360. Accordingly, mobile device300A triggers mobile device300B to invoke an interactive chat session with mobile device300A, as shown inFIG. 3H, by sending a notification using any of the techniques described hereinabove, such as the methods described with respect toFIGS. 3E-3F.

According to another embodiment of the disclosed techniques, mobile device300A executes one or more of the respective actions in response to determining that the acquired position property of mobile device300A complies with one of the position schemes of mutual interaction scheme360.

Reference is now made toFIGS. 3I-3Jtogether withFIG. 3Bwhich show an implementation of a fourth rule of mutual interaction scheme360, in accordance with this embodiment of the disclosed techniques. According to the fourth rule of mutual interaction scheme360, compliance with the fourth position scheme by mobile device300A causes each of mobile devices300A and300B to execute a respective action: one of the respective actions is triggered by device300A objectively onto device300B for execution by device300B, and the other respective action is executed reflexively by device300A.

The user (not shown) moves mobile device300A in a circular shaped trajectory366, shown inFIG. 3I, resulting in tracing a closed circular shape. IMU330A of mobile device300A acquires the position property of mobile device300A, namely the closed-circle trajectory traced by mobile device300A. Mobile device300A compares the trajectory to the position schemes of mutual interaction scheme360. Mobile device300A determines that the acquired closed-circle trajectory complies with the fourth position scheme of mutual interaction scheme360. Accordingly, referring toFIG. 3J, mobile device300A applies the fourth rule of mutual interaction scheme360and executes the respective action to indicate the acquired trajectory of mobile device300A reflexively, by displaying star icon362A, identified with mobile device300A, tracing a closed circular trajectory366A at user interface302A of mobile device300A. Additionally, mobile device300A triggers mobile device300B to display the acquired trajectory of mobile device300A at user interface302B mobile device300B by sending a notification using any of the techniques described above with respective toFIGS. 3E-3F. For example, the notification may be sent via respective transceivers304A and304B. Mobile device300B responds to the notification and executes the respective action by displaying star icon362B, identified with mobile device300A, tracing a closed circular trajectory366B at user interface302B mobile device300B. Thus, in response to tracing a circular shaped trajectory by mobile device300A, respective star icons362A and362B, associated with mobile device300A are displayed moving in a corresponding circular trajectory at each of the respective user interfaces302A and302B of mobile devices300A and300B.

The respective action for the fourth association rule of mutual interaction scheme360indicates the compliance of mobile device300A with the fourth position scheme. In the example given above, the indication of the compliance is implemented by displaying the indication at respective user interfaces302A and302B of respective mobile devices300A and300B. However, the indication may be implemented by additional or alternative means. For example, the compliance of mobile device300A may be indicated by sounding an audio indication at any of speakers320A and320B of mobile devices300A and300B, or by emitting an optical signal by any of optical emitters326A and326B of respective mobile devices300A and300B, shown inFIG. 3A.

As described above, mutual interaction scheme360may be any of: fully reciprocal, partially reciprocal, or exclusive. Thus, the association rules of mutual interaction scheme360that are used to determine compliance of the acquired position properties of mobile device300B may be the same, partially the same, or different than the association rules of mutual interaction scheme360that are used to determine compliance of the acquired position properties of mobile device300A.

According to another a further embodiment of the disclosed techniques, the execution by mobile device300B of the respective action triggered by mobile device300A, is conditioned on the determined compliance of the acquired position property of mobile device300B with the position scheme of mutual interaction scheme360. For example, mobile device300B may execute the action triggered by mobile device300A only if both mobile devices300A and300B trace the same gesture, or are both orientated at an absolute, predefined orientation, or if their respective acquired trajectories, relative orientation or position complies with a predefined position scheme. Turning to the example above with reference toFIGS. 3I-3J, mobile device300B conditions displaying icon362B on display302B of mobile device300B on determining that a distance between mobile device300A and mobile device300B is less than 2 meters.

According to a another embodiment of the disclosed technique, any of the techniques described above for mobile device300A respective of mobile device300B, may be applied by mobile device300B respective of mobile device300A. Accordingly, IMU330B of mobile device300B acquires a position property of mobile device300B, and mobile device300B determines that the acquired position property of mobile device300B complies with at least one of the position schemes of mutual interaction scheme360. Mobile device300B identifies the respective action associated with the position scheme with which the acquired position property complies, and triggers the execution of the identified action on mobile device300A using any of the techniques described hereinabove with respect to mobile device300A. Mobile device300A executes the respective action triggered by mobile device300B, accordingly.

In one implementation of this embodiment, mobile device300B remotely controls mobile device300A to execute the respective action via the notification. Alternatively, mobile device300A maintains control of the execution of the respective action indicated by the notification, by conditioning the execution of the respective action on the fulfillment of one or more criterion.

Reference is now made toFIGS. 3K-3Ltogether withFIG. 3B, which illustrate another application of the fourth rule of mutual interaction scheme360, in accordance with this embodiment of the disclosed techniques. The user (not shown) moves mobile device300B in a triangular trajectory368, shown inFIG. 3K, resulting in tracing a closed triangular shape. IMU330B of mobile device300B acquires the position property of mobile device300B, namely the closed-triangular trajectory traced by mobile device300B. Mobile device300B compares the trajectory to the position schemes of mutual interaction scheme360. Mobile device300B determines that the acquired closed-triangular trajectory complies with the fourth position scheme of mutual interaction scheme360. Accordingly, mobile device300B applies the fourth rule of mutual interaction scheme360.

Referring toFIG. 3L, mobile device300B executes the respective action associated with the fourth position scheme reflexively, by indicating the acquired trajectory368of mobile device300B at user interface302B. Mobile device300B indicates the trajectory by displaying a “+” icon364B, identified with mobile device300B, tracing a closed triangular trajectory368B, corresponding to the acquired trajectory368. Additionally, mobile device300B triggers the execution of the respective action objectively onto mobile device300A. Mobile device300B triggers mobile device300A using any of the techniques described hereinabove, with respect toFIGS. 3E-3F, such as by sending a notification via respective transceivers304A and304B. Mobile device300A responds to the notification and executes the respective action by displaying a “+” icon364A, identified with mobile device300B, tracing a closed triangular trajectory368A at user interface302A mobile device300A. Thus, in response to tracing a triangular shaped trajectory368by mobile device300B, respective icons368A and368B, identified with mobile device300B, are displayed moving in respective corresponding triangular trajectories368A and368B at each of the respective user interfaces302A and302B of respective mobile devices300A and300B.

As discussed above with respect to mobile device300A, additionally or alternatively, the compliance of mobile device300B may be indicated by sounding an audio indication at any of speakers320A and320B of mobile devices300A and300B, or by emitting a visual signal by any of optical emitters326A and326B of respective mobile devices300A and300B, shown inFIG. 3A.

Each of mobile devices300A and300B may display their respective position properties using a different, identifying icon, such as icons362A and362B (stars) associated with mobile device300A and shown inFIGS. 3I-3J, and icons364A and364B (“+” signs) associated with mobile device300B and shown inFIGS. 3K-3L. This may be useful for tracking the positions, orientations and trajectories for respective mobile devices300A and300B in real-time, such as when executing an interactive application of multiple interacting entities, each represented by a different icon. Alternatively, each of mobile devices300A and300B may display their respective position properties using the same icon, such as may be useful for playing an interactive soccer or ping-pong game.

Reference is now made toFIGS. 3M-3Ntogether withFIG. 3Bwhich illustrate an additional application of the fourth rule of mutual interactive scheme360, in accordance with a further embodiment of the disclosed techniques. In this embodiment, each of mobile devices300A and300B triggers the execution of the respective action on the other one of devices300A and300B, in response to determining compliance with the fourth position scheme of mutual interaction scheme360, as follows:

Mobile device300A traces circular-shaped trajectory366. IMU330A of mobile device300A acquires the traced circular trajectory366. Processor334A of mobile device300A determines that the acquired trajectory of mobile device300A complies with the fourth position scheme of mutual interaction scheme360. Mobile device300A triggers mobile device300B to execute the respective action associated with the fourth position scheme, namely to display star icon362B on display302B of mobile device300B moving in circular shaped trajectory366B corresponding to acquired trajectory366. Accordingly, mobile device300B responds to the trigger, and displays star icon362B on user interface302B of mobile device300B, moving in circular trajectory366B.

Simultaneously, mobile device300B traces triangular shaped trajectory368. IMU330B of mobile device300B acquires traced triangular trajectory230. Processor334B of mobile device300B determines that the acquired trajectory of mobile device300B complies with the fourth position scheme of mutual interaction scheme360. Mobile device300B triggers, onto mobile device300A, the execution of the respective action associated with the position scheme in accordance with the fourth rule, namely to display “+” icon364A on display302A of mobile device300A moving in a triangular shaped trajectory368A, corresponding to acquired trajectory368. Accordingly, mobile device300A responds to the trigger, and displays “+” icon364A on user interface302A of mobile device300A, moving in triangular trajectory368A.

In one implementation of the embodiment described above in conjunction withFIGS. 3A-3T, in response to determining compliance of each of mobile devices300A and300B with mutual interaction scheme360, mobile devices300A and300B only trigger the respective indications of the compliance on the respective other one of mobile devices300A and300B. This implementation is illustrated inFIGS. 1A-1B.

Referring back toFIGS. 3M-3Ntogether withFIG. 3B, in another implementation of the of the embodiment described above, in response to determining compliance of each of mobile devices300A and300B with mutual interaction scheme360, mobile devices300A and300B, in addition to triggering the respective indications of the compliance on the respective other one of mobile devices300A and300B as described above, each of mobile devices300A and300B indicates the compliance reflexively. Thus mobile device300A indicates compliance of the acquired position property of mobile device300B with mutual interaction scheme360, and additionally indicates compliance of the acquired position property of mobile device300A with mutual interaction scheme360. Accordingly, mobile device300A displays at display302A of mobile device300A, “+” icon364A identified with mobile device300B, tracing triangular trajectory368A, as well as star icon362A, identified with mobile device300A, tracing circular trajectory366A.

Similarly, mobile device300B indicates compliance of the acquired position property of mobile device300A with mutual interaction scheme360, and in addition, indicates compliance of the acquired position property of mobile device300B with mutual interaction scheme360. Accordingly, mobile device300B displays at display302B of mobile device300B, star icon362B, identified with mobile device300A, tracing a circular trajectory366B, as well as “+” icon364B, identified with mobile device300B, tracing a triangular trajectory368B.

In accordance with another embodiment of the above techniques, mobile device300A conditions the triggering of the execution of the respective action on any of mobile devices300A and300B on determining the compliance of both the acquired position property of mobile device300A and the acquired position property of mobile device300B with at least one position scheme of mutual interaction scheme360. This embodiment is exemplified by the fifth rule of mutual interactive scheme360.

Reference is now made toFIG. 30, which together withFIG. 3Billustrate an implementation of an exemplary fifth rule of mutual interactive scheme360, in accordance with this embodiment of the disclosed technique. According to the fifth rule of mutual interactive scheme360, the fifth position scheme of holding both mobile devices300A and300B in proximity and tracing a common trajectory, such as a predefined shape as sensed by each of respective IMU330A and IMU330B, and corresponding to a predefined signature, is associated with the action of authentication. The authentication may enable money transfer, file transfer, data transfer, or joining a permission group associated with a social media application respective of a first account associated with mobile device300A and a second account associated with mobile device300B. Additionally or alternatively, the authentication may initiate the establishment of a communication link, or initiate audio or video recording respective of mobile devices300A and300B. For example, the money transfer may be enabled by transferring a credential from one of mobile devices300A and300B to the other respective one of mobile devices300A and300B. In this case, the predefined signature is “AB”, where “A” corresponds to the initial of Alice, the user of mobile device300A and owner of the first account, and ‘8’ corresponds the initial of Bob, the user of mobile device300B and owner of the second account. Additionally or alternatively, in some embodiments, authentication may be implemented using an audio fingerprint, as is known in the art.

Each of mobile devices300A and300B detect their respective proximities to each using any of the techniques described hereinabove with reference to mobile devices100A and100B ofFIGS. 2A-2F. Additionally or alternatively, mobile devices300A and300B detect their respective proximities to each other via a touch sensitive sensor (not shown). While held in proximity, mobile devices300A and300B together trace a trajectory forming the letters “AB”. Mobile device300A determines that the trajectory traced by each of mobile devices300A and300B complies with the fifth position scheme of mutual interaction scheme360, namely, the signature “AB” corresponding to the respective initials of the users of mobile devices300A and300B, Alice and Bob. Similarly, mobile device300B determines that the trajectory traced by each of mobile devices300A and300B complies with the fifth position scheme of mutual interaction scheme360, corresponding to the signature “AB”. Mobile devices300A and300B mutually notify each respective other one of mobile devices300A and300B of the compliance of each respective trajectory of mobile devices300A and300A with the fifth position scheme of mutual interaction scheme360. The mutual notification may be implemented using any of the techniques described hereinabove with respect toFIGS. 3E-3F, such as via respective transceivers304A and304B.

Accordingly, on determining that both of mobile devices300A and300B comply with the position scheme of tracing the trajectory “AB”, at least one of mobile devices300A and300B triggers the respective action of enabling the money transfer between the respective accounts associated with devices300A and300B.

Mutual interaction scheme360may be incorporated into any suitable application, such as a communication platform (email or chat), an interactive game, a navigation platform, a platform for performing a medical procedure, a platform for performing a mechanical construction or repair, a file sharing application, a data merging application, and the like. Each such application may have integrated therein a different respective mutual interaction scheme, having different sets of rules associating different position properties with different respective actions.

Thus, to prevent the unintentional triggering of respective actions, invoking and exiting an application may require registering and un-registering any of mobile devices300A and300B with the desired application. In one implementation, the registration and un-registration of mobile devices300A and300B with the application is triggered responsive to acquiring a position property of mobile devices300A and300B, such as illustrated by a sixth rule of mutual interaction scheme360. Triggering of any of the respective actions described herein may be conditional on first registering mobile device300A and300B with the application in accordance with the sixth rule, as follows.

Reference is now made toFIG. 3P, which together withFIG. 3B, illustrate an application of an exemplary sixth rule of mutual interactive scheme360, in accordance with another embodiment of the disclosed technique. The sixth rule of mutual interactive scheme360associates a rotation about a longitudinal axis of a mobile device with registering to the mutual interaction application. Accordingly, mobile device300A is rotated about the longitudinal axis370A of mobile device300A. Mobile device300A acquires the rotation using any of the techniques described above with respect to mobile devices100A and100B ofFIGS. 2A-2F. Processor334A of mobile device300A determines that the acquired rotation complies with the sixth position scheme of mutual interaction scheme360. Accordingly, mobile device300A registers to a mutual interaction application, and notifies mobile device300B of the registration using the notification techniques described above with reference toFIGS. 3E-3F, such as via respective transceivers304A and304B.

Similarly, mobile device300B is rotated about the longitudinal axis370B of mobile device300B. Mobile device300B acquires the rotation using any of the techniques described above with respect to mobile devices100A and100B ofFIGS. 2A-2F. Processor334B of mobile device300B determines that the acquired position property complies with the sixth position scheme of mutual interaction scheme. Accordingly, mobile device300B registers to the mutual interaction application, and notifies mobile device300A of the registration using the notification techniques described above with reference toFIGS. 3E-3F, such as via respective transceivers304A and304B.

When each of mobile devices300A and300B receive from each respective other one of mobile devices300A and300B, the respective notification, mobile devices300A and300B proceed to mutually interact in response to the acquisition of one or more respective position properties, as described herein with respect to the other rules of mutual interaction scheme360. Similarly, in accordance with the sixth rule, mobile devices300A and300B un-register from the application by rotating about the respective longitudinal axes of mobile devices300A and300B. Although the description discloses the same position scheme for both registering and un-registering from an application, it may be noted that this is for exemplary purposes only, and the actions of registering and un-registering from an application may each be associated with a different position scheme.

Reference is now made toFIGS. 3Q-3R, which together withFIG. 3Billustrate an implementation of an exemplary seventh rule of mutual interactive scheme360, in accordance with a further embodiment of the disclosed techniques. According to the seventh rule, maintaining a distance between mobile devices300A and300B to within a predefined range is associated with indicating the distance at any of mobile devices300A and300B. Alternatively, the maintained distance may be between mobile devices300A,300B and one or more external sensors (not shown).

Mobile device300A acquires the distance between mobile device300A and mobile device300B using any of the techniques described hereinabove with reference to mobile devices100A and100B ofFIGS. 2A-2F. Processor334A of mobile device300A determines that the acquired distance complies with the seventh position scheme of mutual interaction scheme360, by determining that the acquired distance is within the predefined range. Mobile device300A triggers mobile device300B to indicate the distance, accordingly. In response to the trigger, mobile device300B indicates the acquired distance.

Mobile device300B indicates the acquired distance by modifying the size of displayed icon362B in reverse proportion to the acquired distances. Thus, with reference toFIG. 3Q, at time T1, mobile device300A acquired the distance D1between mobile devices300A and300B, and determines that D1complies with the range of the seventh position scheme of mutual interaction scheme360. Accordingly, mobile device300A triggers the display of star icon362B on user interface302B of mobile device300B sized with respect to D1. InFIG. 3R, at time T2, mobile device300A acquired distance D2between mobile devices300A and300B, and determines that D2complies with the range of the seventh position scheme of mutual interaction scheme360. Accordingly, mobile device300A triggers the display of star icon362B on user interface302B of mobile device300B sized with respect to D2. In response to each of the triggers, mobile300B adjusts the size of star icon362B displayed at user interface302B, accordingly. Since D2>D1, The size of star icon362B displayed to indicate D1(FIG. 3Q) is larger than the size of star icon362B displayed to indicate distance D2(FIG. 3R).

In a similar manner, mobile device300B triggers the display of “+” icon364A on user interface302A of mobile device300A sized with respect to D1and D2, respectively. In response to each of the triggers, mobile300A adjusts the size of “+” icon362A displayed at user interface302A, accordingly. Additionally or alternatively, any of the absolute or relative acquired orientations of mobile devices300A and300B may be indicated by displaying icons362B and364A in a manner that indicates the respective acquired orientations.

Additionally or alternatively, the respective action associated with the seventh scheme of mutual interaction scheme360may be to indicate an alert via any of respective speakers320A and320B and respective optical emitters326A and326B of mobile devices300A and300B, shown inFIG. 3AC. For example, the alert may indicate that the acquired distance does not comply with the predefined range.

Additionally or alternatively, on complying with the predefined range, an application may be invoked, such as a loudspeaker application, or a multi-perspective imaging application, illustrated inFIGS. 3S-3Tand described in greater detail hereinbelow.

Reference is now made toFIG. 3Swhich together withFIG. 3B, illustrates system350operative as a loudspeaker system350, in accordance with another embodiment of the disclosed techniques. In this embodiment, which is exemplified by the eighth rule of mutual interaction scheme360, mobile devices300A,300B and300C are used in unison to emulate a loudspeaker at an audio receiver372. Audio receiver372is shown as a user's ear, but may alternatively be a microphone (not shown).

With referenceFIG. 3B, the eighth rule of mutual interaction scheme360initiates loudspeaker application350, in response to an indication by the user of mobile device300A. Responsive to the indication, processor334A of mobile device300A determines that a distance D1between speaker320A and audio receiver372complies with the eighth rule of mutual interaction scheme360indicating that speaker320A is within audio range of audio receiver372. For example, processor334A may determine distance D1using one or more images of audio receiver372acquired by camera324A, together with the position and orientation of mobile device300A acquired by IMU330A, and the relative position of speaker320A respective of mobile device300A. As the position and orientation of mobile device300A, changes over time, D1is adjusted accordingly, in real-time, thereby synchronizing speaker320A to audio receiver372.

Mobile device300A detects one or more mobile devices300B and300C that are within audio range of mobile device300A and audio receiver372, and determines the respective distances D2and D3between mobile device300A and each of mobile devices300B and300C using any of the techniques describes herein. Mobile device300A applies the determined distances D2and D3to determine the respective distances D4and D5, indicated as dashed lines, between each of mobile devices300B and300C and audio receiver372. Mobile device300A determines that distances D2, D3, D4and D5comply with the eighth rule of mutual interaction scheme, indicating that speakers320B and320C are within audio range of audio receiver372, thereby synchronizing respective speakers320B and320C to audio receiver372. As the relative positions and orientations of mobile devices300A,300B, and300C change over time, changing respective distances D1, D2, and D3, mobile device300A recalculates respective distances D4and D5in real-time, and determines compliance with the eight rule of mutual interaction scheme360, thereby synchronizing speakers320B and320C to audio receiver360.

Mobile device300A synchronizes an internal clock of processor334A with an internal clock of respective processors334B and334C of mobile devices300B and300C, as follows. At time t0respective of the internal clock of processor334A, mobile device300A emits a predefined optical signal, such as a timed series of flashes, via optical emitter326A. Additionally, mobile device300A transmits the start time t0via transceiver304A. At time t1respective of the internal clock of processor334B, mobile device300B detects the emitted optical signal via camera324B, and start time t0via transceiver304B. Processor334B of mobile device300B determines the time shift t0−t1representing the relative time shift between the respective internal clocks of mobile device300A and mobile device300B. Similarly, at time t2respective of the internal clock of processor334C, processor334C of mobile device300C detects the emitted optical signal via camera324C, and start time t0via transceiver304C. Mobile device300C determines the time shift t0−t2describing the relative time shift between the respective internal clocks of mobile device300A and mobile device300C. Subsequently, a process initiated by mobile device300A at time t+t0respective of the internal clock of processor334A, is synchronized with a second process initiated by mobile device300B at time t+(t0−t1) respective of the internal clock of processor334B, and further synchronized with a third process initiated by mobile device300C at time t+(t0−t2) respective of the internal clock of processor334C.

To achieve constructive interference at audio receiver372of an audio file emitted by each of speakers320A,320B, and320C, and thereby implement the loudspeaker application, mobile devices300A,300B, and300C calibrate the respective times for emitting the audio file by each of speakers320A,320B, and320C such that the respective emitted audio files are in phase on arrival at audio receiver372and thus constructively interfere. The phase shift realized at audio receiver372resulting from simultaneously emitting the audio file by speaker320A and by speaker320B is expressed by dt1, and computed according to:
dt12=(|D1|−|D4|)/v

where v is the velocity of the acoustic wave. Similarly, the phase shift realized at audio receiver372resulting from simultaneously emitting the audio file by speaker320A by speaker320C is expressed by dt2, and determined according to:
dt22=(|D1|−|D5|)/v.

Mobile device300A computes the phase shift dt1and sends dt1together with a start time TSTART, representing a start time to initiate a rendering of an audio file by speaker320A, to mobile device300B. Similarly, mobile device300A computes the phase shift dt2and sends dt2together with start time TSTARTto mobile device300C. At time TSTART, speaker320A begins rendering the audio signal; at time TSTART+(t0−t1)+dt1, speaker320B begins rendering the audio file; at time TSTART+(t0−t2)+dt2, speaker320C begins rendering the audio file. The three audio files rendered by respective speakers320A,320B, and320C are thus synchronized in time and phase on arrival at audio receiver372, to realize a loudspeaker application at audio receiver372.

The above technique, i.e. synchronizing the internal clocks, and computing a respective phase shift, may be applied to as few as two mobile devices, or alternatively to more than three mobile devices. In such a case, mobile device300A may create a “left speaker” cluster of mobile devices to operate collectively as a left speaker for audio application350, an a “right speaker” cluster of mobile devices to operate collectively as a right speaker for the audio application350. Mobile device300A detects one or more mobile devices (not shown) in proximity to mobile device300B. Mobile device300A groups mobile device300B with the one or more mobile devices in proximity thereof as the “left speaker” cluster. Similarly, mobile device300A detects one or more mobile devices (not shown) in proximity to mobile device300C. Mobile device300A groups mobile device300C with the one or more mobile devices in proximity thereof as the “right speaker” cluster.

Mobile device300A determines the respective distances between each of the plurality of mobile devices included in each of the left speaker cluster and the right speaker cluster and audio receiver372using the techniques described hereinabove with respect to mobile devices300B and300C. Similarly, mobile device300A synchronizes the respective internal clock for each mobile device in each of the right speaker cluster and the left speaker cluster using the techniques described hereinabove with respect to mobile devices300B and300C. Mobile device300A computes the respective phase difference dti(left)for each mobile device of the left speaker cluster, as well as the respective phase difference dti(right)for each mobile device of the right speaker cluster and transmits the respective phase differences to each respective mobile device together with a start time for rendering an audio file. Subsequently, each mobile device of each of the left speaker cluster renders the audio file in accordance with the respective phase difference dti(left)at the start time synchronized with the internal clock of mobile device300A to emulate a left speaker at audio receiver372. Similarly, each mobile device of each of the right speaker cluster renders the audio file in accordance with the respective phase difference dti(right)at the start time synchronized with the internal clock of mobile device300A to emulate a right speaker at audio receiver372. As a result, the multiple audio files rendered by each mobile device of the left speaker cluster constructively interfere to emulate a left speaker at audio receiver372, and the multiple audio files rendered by each mobile device of the right speaker cluster constructively interfere to emulate a right speaker at audio receiver372, thereby emulating a loudspeaker in stereo.

In another embodiment of the disclosed technique, system350is operative as a multi-perspective imaging system, and which is exemplified by the ninth rule of mutual interaction scheme360. The multiple mobile devices are used in unison to create a multi-dimensional, or panoramic image of an object. Alternatively, the multiple mobile devices are used in unison to track the object.

The multiple mobile devices are positioned around the object to allow capturing different perspectives of the object. Once the multiple mobile devices are focused, or ‘locked’ onto the object, described in greater detail below with respect toFIG. 3T, each mobile device captures a different image of the object, accordingly. Simultaneous with capturing the image, each mobile device acquires metadata associated with the captured image, such as the respective position and orientation of the mobile device at the instant of capturing the image, as well as a time stamp corresponding to the instant of capturing the image. Each mobile device provides the captured image with the associated metadata to an image processor. In this manner, the image processor receives multiple images corresponding to multiple different perspectives of the object. The image processor processes the images by applying the associated metadata to merge the images into any of a panoramic or multi-perspective image of the object. Similarly, the image processor may apply the associated metadata with the images to track the object.

Reference is now made toFIG. 3Twhich together withFIG. 3B, illustrate an implementation of system350operative as a multi-perspective imaging system, in accordance with a further embodiment of the disclosed techniques. In this embodiment, which corresponds to the ninth position scheme of mutual interaction scheme360, respective cameras326A and326B of mobile devices300A and300B are positioned within viewing range of an object240. Mobile device300A acquires the position property {PA,OA} of mobile device300A respective of an object240, and mobile device300B acquires the position property {PB,OB} of mobile device300B respective of object240, using any of the techniques described above with respect to mobile devices100A and100B ofFIGS. 2A-2F. Each of mobile devices300A and300B determines that the respective acquired position property of each of mobile devices300A and300B complies with the ninth position scheme of mutual interaction scheme360. The ninth position scheme includes one or more optical range requirements respective of object240, such as a distance and orientation, or a range of distances and orientations that allow for capturing images of object240with a predefine resolution. The distances and orientations defined by the ninth position scheme may additionally account for different camera types, such as if cameras324A and324have a different field of view capabilities, focus capabilities, resolutions or other qualitative attributes.

Mobile device300A determines that the acquired position property {PA,OA} complies with the ninth position scheme of mutual interaction scheme360. Similarly, mobile device300B determines that the acquired position property {PB,OB} complies with the ninth position scheme of mutual interaction scheme360. Mobile devices300A and300B mutually notify each respective other one of mobile devices300A and300B of the compliance with the ninth position scheme of mutual interaction scheme360using any of the notification techniques described hereinabove with respect toFIGS. 3E-3F.

When both mobile devices300A and300B have determined mutual compliance with the ninth position scheme of mutual interaction scheme360, each of mobile devices300A and300B executes an instance of a multi-perspective imaging application. This has the effect of ‘locking’ each of cameras324A and324B of mobile devices300A and300B onto object240. In one implementation of the locking procedure, mobile device300B notifies mobile device300A of the compliance of mobile device300B with the ninth position scheme of mutual interaction scheme360. On determining compliance of both mobile devices300A and300B, mobile device triggers the execution of the multi-perspective imaging application on mobile device300B, and executes the multi-perspective imaging application at mobile device300A. In another implementation, each of mobile devices300A and300B conditions the execution of the multi-perspective imaging application at each respective mobile device300A and300B on both: a) determining compliance of each respective mobile device300A and300B with mutual interaction scheme360, as well as b) receiving a trigger to execute the multi-perspective imaging application from each other respective mobile device300A and300B. Thus, in this case mobile device300A triggers mobile device300B to execute the multi-perspective imaging application, and mobile device300B triggers mobile device300A to execute the multi-perspective imaging application, thereby mutually validating that each respective mobile device300A and300B complies with mutual interaction scheme360. It may be noted that this is but one exemplary implementation for locking cameras324A and325B onto object240, and other suitable techniques may be used as well.

Once cameras324A and325B are locked onto object240, each of mobile devices300A and300B executes the multi-perspective imaging application by performing the following steps:1) Capturing at least one image of object240via respective cameras324A and324B of respective mobile devices300A and300B.2) Simultaneous with capturing each image, acquiring metadata associated with the captured image. The metadata includes at least: the position and orientation of each of cameras324A and324B with respect to object240at the instant of capturing each image, and additionally a time stamp associated with the instant of capturing each image. The metadata may additionally include one or more optic properties of cameras324A and324B.3) Providing each captured image and the associated metadata to server354via respective transceivers304A and304B of respective mobile devices300A and300B, and network352.

Server354receives the multiple captured images with the associated metadata from each of mobile devices300A and300B via network352. Server354is configured with an image processor, and processes the multiple captured images using the associated metadata to create any of: a panoramic image of object240, a multi-perspective image of object240, or to track object240in real-time. Alternatively, any of mobile devices300A and300B may operate as the image processor, and process the images accordingly.

In another embodiment of the disclosed technique, the multiple mobile devices are used in unison to implement a virtual pen, which is exemplified by the tenth rule of mutual interaction scheme360.

Reference is now made toFIG. 3U, which together withFIG. 3B, illustrate an implementation of system380operative to interface a mobile device to control an application running on a host device. This embodiment corresponds to an additional position scheme (not shown) of mutual interaction scheme360. System380includes mobile device300A and additionally a computer382, and a display384. Computer382includes at least a processor390, a memory unit392, and a transceiver394. Display384is provided with a camera388configured to acquire images and provide the images to computer382.

Processor390is coupled to each of memory unit392and transceiver394. Computer382is coupled to display384. Transceiver304A of mobile device300A and transceiver394of computer382are coupled using any suitable wireless means, such as via WiFi, BlueTooth and the like. Mobile device300A is within optical range of display384.

Processor390is operative to execute instructions stored in memory unit392, and to control the rendering of graphical content on display384. Processor390is operative to receive images acquired by camera388and store the images in memory392, and process the images accordingly. Mobile device300A is operative to continually communicate respective relative and absolute position and orientation information acquired by at least one of IMU330A, indoor GPS310A, and GPS312A, and camera324A to computer382via respective receivers304A and394. On receiving the position and orientation information of mobile device300A, computer382is operative to apply the relative and absolute position and orientation information to control an application running thereon, such as to control the rendering of graphical content on display384. Additionally, or alternatively, one or more images acquired by any of camera324A of mobile device300A and camera388may be used by computer382to determine the position and orientation information of mobile device300A respective of display384.

In one embodiment, mobile device300A is operative to emulate a wireless 2D or 3D mouse, and interface with computer382to control the display of graphic content on display384, such as a cursor386. Computer382receives an initializing notification from mobile device300A to invoke a graphic display application respective of display384, such as wireless mouse application. The initializing notification additionally includes a starting position and orientation for mobile device300A, which includes the relative and absolute position and orientation of mobile device300A that were most recently acquired by IMU330A, indoor GPS310A, and GPS312A, and camera324A. Computer382applies the starting position and orientation of mobile device300A to calibrate subsequent motion by mobile device300A to a corresponding change in position of cursor386on display384. In one implementation, computer382associates the starting position and orientation of mobile device300A with a pre-determined starting position on display384, such as the center of display384. In an alternative implementation computer382determines the starting position on display384for associating with the starting position and orientation of mobile device300A in a manner to provide a wide range of motion for the user, such as when the current position and orientation of mobile device300A is not aligned with the center of display384.

For example, if mobile device300A is positioned at an angle θ that is offset to a central vertical or horizontal axis of display384, computer382associates the starting position and orientation of mobile device300A with a starting position on display384that is proportional to the offset. Alternatively, computer382may prompt the user of mobile device300A to select a starting position on display384that is different than the center of display384, to allow for a wider range of motion by the user. Alternatively, computer382may alert the user that the current position and orientation of mobile device300A is not aligned with the center of display384, thereby limiting the range of the cursor functionality. Computer382may prompt and guide the user to align the position and orientation of mobile device300A with the center of display384, and at a distance from display384that allows for a comfortable range of motion for the user.

Once the starting position on display384has been associated with the starting position and orientation of mobile device300A, computer382invokes a calibration phase to scale an allowable range of motion granted to mobile device300A to the dimensions of display384. For example, computer382may prompt the user to move mobile device300A horizontally and vertically to reach the edges of display384, and thereby scale the trajectory of mobile device300A to the respective horizontal and vertical dimensions of display384. Additionally or alternatively, computer382may use an image of display384acquired by camera324A to scale a trajectory of mobile device300A to a corresponding position on display384. Once calibrated, computer382applies a subsequently detected change in the position and orientation of mobile device300A to change the position of cursor386on display384, accordingly. Computer382implements one or more of the following functionalities respective of mobile device300A and display384:1) On determining that the elevation of mobile device300A has changed relative to display384, computer382adjusts the vertical position of cursor386on display384by a proportional distance scaled to the respective height of display384.2) On determining that the horizontal position of mobile device300A has changed relative to display384, computer382adjusts the horizontal position of cursor386on display384by a proportional distance scaled to the respective width of display384.3) On determining that a rotational parameter, such as the “roll” parameter of mobile device300A has changed in accordance with position and orientation data acquired by IMU330A, computer382may invoke the user to control the execution of the graphic display application by computer382. For example, computer382may prompt the user of mobile device100A to select between a plurality of applications running on computer382, in a manner similar to pressing a “tab” button.4) On determining that mobile device300A has been moved close to display384, computer382magnifies at least a portion of the content displayed on display384to implement a “zoom-in” functionality.5) On determining that mobile device300A has been moved further from display384, computer382miniaturizes at least a portion of the content displayed on display384to implement a “zoom-out” functionality.

In another implementation, mobile device300A is virtually coupled to a rendition of a three dimensional object (not shown) on display384, such as a computer aided design or manufacturing (CAD/CAM) software application. As the user manipulates the position and orientation of mobile device300A, the rendition of the three dimensional object is manipulated accordingly on display384, allowing the user the control the position and orientation of the three dimensional object by manipulating mobile device300A. Throughout, mobile device300A continually transmits relative and absolute position and orientation information acquired via any of IMU330A, indoor GPS310A, and GPS312A, and camera324A, such that changes in the relative and absolute position and orientation of mobile device300A translate to corresponding changes in the position and orientation of the three dimensional object rendered on display384. Computer382may additionally apply one or more images of mobile device300A acquired by camera388to determine respective changes in the relative and absolute position and orientation of mobile device300A. For example, rotating mobile device300A along any of the respective X, Y, or Z axes of mobile device300A results in a corresponding rotation along the respective X, Y, and Z axes of the 3D object on display384. Translating mobile device300A along any of the respective vertical and horizontal axes respective of display384causes a corresponding vertical and horizontal translation of the 3D object on display384. Similarly, moving mobile device300A closer to display384results in a zoomed-in view of the three dimensional object on display384, and moving mobile device300A further away from display384results in a zoomed-out view of the three dimensional object on display384. As with the calibration phase described above, computer382may prompt the user to adjust the position and orientation of mobile device300A when mobile device300A is initially coupled to the three dimensional object such as to allow for a wide range of motion by the user to manipulate the rendition of the three dimensional object. For example, if at initialization, the orientation of mobile device300A is not neutrally oriented with respect to any of the respective X, Y, and Z axes of mobile device300A, computer382may alert to user to align mobile device300A into a neutral position, to subsequently allow for a maximal range of motion by the user manipulating mobile device300A.

Reference is now made toFIGS. 4A-4Bwhich together withFIG. 3Billustrates a virtual pen system450, in accordance with another embodiment of the disclosed technique.

With reference toFIG. 4A, a schematic illustration of a mobile device400A is shown. Mobile device400A includes at least a processor434A, a memory unit432A, a transceiver404A, and an IMU430A. Transceiver404A includes at least medium range transceiver406A, short range transceiver408A, and cellular transceiver414A. IMU430A includes compass428A, 3D accelerometer416A, and gyroscope418A. Transceiver404A, memory unit432A, and IMU430A are coupled to processor434A. Each of transceiver404A, memory unit432A, IMU430A and processor434A are operative to correspond to transceiver104A, memory unit132A, IMU130A and processor134A ofFIG. 2A. Mobile device400A may include additional components not shown inFIG. 4A, such as any of: indoor GPS412A, GPS410A, speaker420A, microphone422A, user interface402A, optical emitter426A, and camera424A, each coupled to processor434A, and each operative to correspond to indoor GPS112A, GPS110A, speaker120A, microphone122A, user interface102A, optical emitter126A, and camera124A ofFIG. 2A.

With reference toFIG. 4B, system450includes two mobile devices: mobile device400A and a mobile device400B. Mobile device400B is represented by mobile device100B described above with reference toFIG. 4B. In particular, mobile device400B includes at least a processor434B, a transceiver404B and a user interface402B, corresponding respectively to processor134B, transceiver104B and user interface102B of mobile device100B, Mobile devices400A and400B are operative to perform one or more of the techniques described above with reference to mobile devices100A and100B ofFIGS. 2A-2F. In particular, mobile device400B is operative to receive data from mobile device400A via respective transceivers404A and404B. Each of mobile devices400A and400B are associated with mutual interaction scheme360.

Mobile device400A is coupled to a writing utensil452provided with a tip458configured to deposit a visible substance, such as ink, graphite, and the like, on a surface454. In one implementation, mobile device400A is integrated within writing utensil452. In another implementation (not shown), mobile device400A is implemented as a wearable mobile device, such as a ring, thimble, bracelet and the like, operative to be worn by a user while the user writes with writing utensil452. In both implementations, mobile device400A is operative to acquire a trajectory traced by writing utensil452, as writing utensil452is maneuvered to write on surface454.

As shown inFIG. 4B, writing utensil452is maneuvered over surface454to deposit a visible trace456via tip458, corresponding to the phrase ‘Hello World!” in script. Simultaneously, IMU430A of mobile device400A acquires a trajectory corresponding to ‘Hello World!” in script. Mobile device400A determines that the acquired trajectory corresponds to visible trace456. For example, camera424A of mobile device400A captures an image of visible trace456, and processor434A of mobile device400A matches the captured image with the trajectory acquired by IMU430A. Alternatively, additional sensors (not shown) include in mobile device400A detect the deposition of the visible substance on surface454, or detect that tip458is in contact with surface454, and the like.

Processor434A of mobile device400A determines that the acquired position property complies with the tenth position scheme of mutual interaction scheme360. Accordingly, mobile device400A triggers mobile device400B to execute the respective action associated with the tenth position scheme, namely to display on user interface402B of mobile device400B, a bit map corresponding to acquired trajectory, in real-time. In one implementation, mobile device400A transforms the trajectory to a bit map and transmit the bit map to mobile device400B via respective transceivers404A and404D. Alternatively, mobile device400A transmits the acquired trajectory to mobile device100B via respective transceivers404A and404D, and mobile device400B transforms the trajectory to the bit map. Mobile device400B displays the bit map on user interface402B of mobile device400B, accordingly. Additionally, mobile device400B may store the bit map at memory store432B of mobile device400B.

Reference is now made toFIG. 5A, which is a schematic illustration of a method for initiating at least one mutual interaction scheme between a first mobile device and at least a second mobile device, in accordance with another embodiment of the disclosed technique.

In procedure500, at least one mutual interaction scheme is associated between a first mobile device and at least a second mobile device. The mutual interaction scheme may be provided, or otherwise made accessible, to each of the first mobile device and at least a second mobile device via a network from a cloud service. The mutual interaction scheme associates at least one position scheme with at least one respective action, where the at least one position scheme relates to at least one of: an absolute orientation, a relative orientation, an absolute position, a relative position, an absolute trajectory, and a relative trajectory of any of the first and at least second mobile devices. With reference to the system ofFIGS. 2A-2F, and the system ofFIGS. 3A-3B, each of mobile devices300A and300B are provided with access to mutual interaction scheme360, via respective transceivers304A and304B of each of mobile devices300A and300B and network352.

In procedure502, a position property of the first mobile device is acquired. The acquired position property of the first mobile device relates to at least one of: an absolute orientation, a relative orientation, an absolute position, a relative position, an absolute trajectory, and a relative trajectory of the first mobile device. With reference to the system ofFIGS. 2A-2F, and the system ofFIGS. 3A-3B, IMU130A of mobile device100A acquires the position property of mobile device100A using the methods described above. Additionally or alternatively, camera124A of mobile device100A captures an image, such as of a reference object, and the position property of mobile device100A is acquired from the image captured by camera124A of mobile device100A. Additionally or alternatively, camera124B of mobile device100B captures an image of mobile device100A. Mobile device100A receives the captured image from mobile device100B, via respective transceivers104A and104B of mobile devices100A and100B. Mobile device100A determines the position property of mobile device100A from the image captured by mobile device100B.

In procedure504, compliance of the acquired position property of the first mobile device with at least one of the at least one position schemes of the mutual interaction scheme is determined. In one implementation, acquiring the position property of the first mobile device, and determining that the acquired position property of the first mobile device complies with one of the at least one position scheme of the mutual interaction scheme, is performed by the first mobile device. With reference to the system ofFIGS. 2A-2F, and the system ofFIGS. 3A-3B, mobile device300A determines that the acquired position property of mobile device300A complies with at least one of the position schemes of mutual interaction scheme360.

In procedure506, in response to the compliance, an execution of one of the at least one respective actions is triggered on the at least second mobile device. The respective triggered action is associated with the at least one position scheme with which the position property of the first mobile device complies, in accordance with the mutual interaction scheme. With reference to the system ofFIGS. 2A-2F, and the system ofFIGS. 3A-3B, mobile device300A triggers mobile device300B to execute the respective action associated with the position scheme with which the position property of mobile device300A complies, in accordance with mutual interaction scheme360. Thus, if the acquired position property of mobile device300A complies with the absolute vertical orientation corresponding to the first position scheme of mutual interaction scheme360, mobile device300A triggers mobile device300B to execute the respective action associated with the first position scheme, namely to send a business card to mobile device300A.

In procedure508, the respective triggered action is executed by the at least second mobile device in response to the compliance of the acquired position property of the first mobile device. With reference to the system ofFIGS. 2A-2F, and the system ofFIGS. 3A-3B, mobile device300B sends the business card to mobile device300A via respective transceivers304A and304B of mobile devices300A and300B.

In procedure510, one of the one or more respective actions is additionally executed by the first mobile device, in response to the determined compliance of the acquired position property of the first mobile device. With reference to the system ofFIGS. 2A-2F, and the system ofFIGS. 3A-3BandFIGS. 3I-3J, each of mobile devices300A and300B execute the respective action by each displaying a icon on respective user interface302A and302B of each of mobile devices300A and300B.

In procedure512, a position property of the at least second mobile device is acquired. The acquired position property of the at least second mobile device relates to at least one of: an absolute orientation, a relative orientation, an absolute position, a relative position, an absolute trajectory, and a relative trajectory of the at least second mobile device.

In procedure514, compliance of the acquired position property of the at least second mobile device with at least one of the at least one position schemes of the mutual interaction scheme is determined.

In one implementation, acquiring the position property of the at least second mobile device, and determining that the acquired position property of the at least second mobile device complies with one of the at least one position scheme of the mutual interaction scheme, is performed by the at least second mobile device. With reference to the system ofFIGS. 2A-2F, and the system ofFIGS. 3A-3BandFIGS. 3K-3L, IMU330B of mobile device300B acquires the position property of mobile device300B using the methods described above. Additionally or alternatively, camera324B of mobile device300B captures an image, such as of a reference object, and the position property of mobile device300B is acquired from the image captured by camera324B of mobile device300B. Additionally or alternatively, with additional reference toFIG. 2F, camera324A of mobile device300A captures an image of mobile device300B. Mobile device300B receives the captured image from mobile device300A via respective transceivers300A and300B, and determines the position property of mobile device300B from the received image.

In another implementation, acquiring the position property of the at least second mobile device, and determining the compliance of the position property of the at least second mobile device is performed by the first mobile device. With reference to the system ofFIGS. 2A-2F, and the system ofFIGS. 3A-3BandFIGS. 3E-3F, camera324A of mobile device300A captures an image of mobile device300B. Processor334A of mobile device300A analyzes the captured image and determines the position property of mobile device300B. Processor334A of mobile device300A determines that the position property of mobile device300B complies with mutual interaction scheme360. Mobile device300A may provide any of the position property of mobile device300B and an indication of the compliance to mobile device300B via respective transceivers304A and304B of respective mobile devices300A and300B.

In another implementation, the execution, by the at least second mobile device of the one or more respective actions triggered by the first mobile device, is conditioned on the determined compliance of the position property of the at least second mobile device.

In procedure516, the first mobile device is notified of the compliance of the position property of the at least second mobile device with at least one of the at least one position schemes of the mutual interaction scheme. With reference to the system ofFIGS. 2A-2F, and the system ofFIGS. 3A-3Band Figures and3K-3L, mobile device300B notifies mobile device300A of the compliance of the acquired position property of mobile device300B with one of the position schemes of mutual interaction scheme360, via respective transceivers304A and304B of mobile devices300A and300B.

In procedure518, an execution of one of the one or more respective actions is triggered on the first mobile device in response to the determined compliance of the acquired position property of the at least second mobile device. The respective action triggered on the first mobile device is associated with the position scheme with which the position property of the at least one second mobile device complies, in accordance with the mutual interaction scheme. With reference to the system ofFIGS. 2A-2F, and the system ofFIGS. 3A-3BandFIGS. 3K-3L, mobile device300B triggers mobile device300A to display an icon in response to the determined compliance of the trajectory of mobile device300B with the fourth position scheme of mutual interaction scheme360.

In procedure520, the respective action triggered by the at least second mobile device is executed on the first mobile device. With reference to the system ofFIGS. 2A-2F, and the system ofFIGS. 3A-3BandFIGS. 3K-3L, mobile device300A displays icon364A on user interface302A of mobile device300A, in accordance with the action triggered by mobile device300B.

In one implementation, the respective action comprises indicating the determined compliance of the acquired position property of the first mobile device. In another implementation, the respective action comprises indicating the determined compliance of the acquired position property of the at least second mobile device.

In another implementation, the respective action triggered and executed in response to the determined compliance of the acquired position property of the first mobile device comprises any of: registering and unregistering any of the first mobile device and the at least second mobile device to the mutual interaction scheme.

In another implementation, the respective action triggered and executed in response to the determined compliance of the acquired position property of the at least second mobile device comprises any of: registering and unregistering any of the first mobile device and the at least second mobile device to the mutual interaction scheme.

In another implementation, the acquired position property of the first mobile device corresponds to the distance between the first mobile device and the at least second mobile device. At least one of the position schemes of the mutual interaction scheme corresponds to the acquired distance. In this implementation, determining the compliance of the acquired position property with the position scheme comprises determining that the acquired distance complies with the position scheme corresponding to the acquired distance. The respective action associated with the position scheme corresponding to the acquired distance comprises indicating the acquired distance at any of the first mobile device and the at least second mobile device. With reference to the system ofFIGS. 2A-2F, and the system ofFIGS. 3A-3BandFIGS. 3Q-3R, each of mobile device300A and300B displays icons364A and362B, respectively, on respective user interfaces302A and302B of mobile devices300A and300B, where the size of icons364A and362B are scaled to correspond to the respective distances D1and D2between mobile devices300A and300B.

Reference is now made toFIG. 5B, which is a schematic illustration of a method for implementing a mutually interactive loudspeaker application with a first mobile device and at least a second mobile device, in accordance with another embodiment of the disclosed technique. The method ofFIG. 5Bis a continuation of the method described above with respect toFIG. 5A.

In procedure530, the position properties of the first mobile device and the at least second mobile device are all determined to comply with an audio range of an audio receiver. In this implementation, the position property acquired from the first mobile device corresponds to a distance and orientation of the first mobile device with respect to the audio receiver, and the position property acquired from the at least second mobile device corresponds to a distance and orientation of the at least second mobile device with respect to the audio receiver. The position scheme of the mutual interaction scheme corresponds to the audio range of the audio receiver. With reference to the system ofFIGS. 2A-2F, and the system ofFIGS. 3A-3BandFIG. 3S, mobile device300A determines that the distances D1, D4, and D5corresponding to the distances between respective mobile devices300A,300B, and300C, and audio receiver372comply with the audio range of audio receiver372, comprising eighth rule of mutual interaction scheme360.

In procedure532, an internal clock of the at least second mobile device is synchronized with an internal clock of the first mobile device. With reference to the system ofFIGS. 2A-2F, and the system ofFIGS. 3A-3BandFIG. 3S, at time t0mobile device300A emits an optical signal and time t0to mobile devices300B and300C. Each of mobile devices300B and300C detect the optical signal and determines the respective time shift between t0and the time that the optical signal was detected, respective of the internal clocks of mobile devices300B and300C.

In procedure534, a phase shift for each of the at least second mobile device is computed such that an audio file simultaneously played by each of the first mobile device and each of the at least second mobile device constructively interferes at the audio receiver. With reference to the system ofFIGS. 2A-2F, and the system ofFIGS. 3A-3BandFIG. 3S, mobile device300A computes a phase shift respective of each of mobile devices300B and300C as a function of the position and orientation of each of mobile devices300B and300C, and transmits each computed phase shift, accordingly.

In procedure536, the audio file is rendered by the first mobile device, and the audio file is rendered by the at least second mobile device according to the respective phase shift at a synchronized start time. With reference to the system ofFIGS. 2A-2F, and the system ofFIGS. 3A-3BandFIG. 3S, mobile device300A renders the audio file via speaker320A, and mobile devices300B and300C render the audio file according to the respective phase shifts.

In some embodiments, synchronizing step of procedure532, the computing step of procedure534, and the rendering step of procedure536are performed with respect to a plurality of mobile devices. A first portion of the plurality of mobile device is grouped as a left speaker cluster, and a second portion of the plurality of mobile devices as a right speaker cluster. The audio file is rendered by the mobiles devices grouped as the left speaker cluster to emulate a left speaker, and the audio file is rendered by the mobiles devices grouped as the right speaker cluster to emulate a right speaker, thereby emulating a stereo loudspeaker at the audio receiver

Reference is now made toFIG. 5C, which is a schematic illustration of a method for implementing a mutually interactive multi-perspective imaging application with a first mobile device and at least a second mobile device, in accordance with another embodiment of the disclosed technique. The method ofFIG. 5Cis a continuation of the method described above with respect toFIG. 5A.

In procedure560, the position property of the first mobile device, and the position property of the at least second mobile device are acquired by capturing an image of an object with at least one of: a camera of the first mobile device and a camera of the at least second mobile device, where the respective acquired position properties of the first and at least second mobile devices are with respect to the object. Additionally, the position scheme of the mutual interaction scheme with which the position property of the first mobile device complies, corresponds to an optical range of the camera configured with the first mobile device. Similarly, the position scheme of the mutual interaction scheme with which the position property of the second mobile device complies, corresponds to an optical range of the camera configured with the second mobile device. For example, the respective optical ranges may relate to a focal range, a field of view, and an optical resolution of any of the cameras with respect to the object.

With reference to the system ofFIGS. 2A-2F, and the system ofFIGS. 3A-3BandFIG. 3T, camera324A of mobile device300A captures an image of object240. Processor334A of mobile device300A uses the image captured by camera324A to acquire a position property corresponding to the position and orientation of mobile devices300A with respect to object240. Additionally, camera324B of mobile device300B captures another image of object240. Processor334B of mobile device300B uses the image captured by camera324B to acquire a position property corresponding to the position and orientation of mobile devices300B with respect to object240.

In procedure562, compliance of each of the acquired position properties with the position scheme of the mutual interaction scheme is determined, as described above with reference to Procedures504and514ofFIG. 5A. With reference to the system ofFIGS. 2A-2F, and the system ofFIGS. 3A-3BandFIG. 3T, processor334A of mobile device300A determines that the acquired position property of mobile device300A complies with the ninth position scheme of mutual interaction scheme360. Similarly, processor334B of mobile device300B determines that the acquired position property of mobile device300B complies with the ninth position scheme of mutual interaction scheme360. With reference toFIG. 3B, the ninth position scheme of mutual interaction scheme corresponds to one or more optical properties of respective cameras324A and324B with respect to object240, such as a field of view or a focal range, or an optical resolution requirement, and the like. Additionally or alternatively to the technique described above, mobile devices300A and300B may acquire the position and orientation of respective mobile devices300A and300B using at least one of the techniques described above with reference toFIGS. 2A-2F.

In procedure564, each of the first mobile device and the at least second mobile device is mutually notified of the respective compliance of each of the respective acquired position properties of the first mobile device and at least second mobile device. With reference to the system ofFIGS. 2A-2F, and the system ofFIGS. 3A-3BandFIG. 3T, mobile device300A notifies mobile device300B of the compliance of the acquired position of mobile device300A with the ninth position scheme of mutual interaction scheme360. Similarly, mobile device300B notifies mobile device300A of the compliance of the acquired position property of mobile device300B with the ninth position scheme of mutual interaction scheme360. The mutual notification may be implemented via any of respective transceivers304A and304B, respective cameras324A and324B, respective emitters326A and326B, respective speakers320A and320B, and respective microphones322A and322B of respective mobile devices300A and300B.

In procedure566, responsive to the mutual notification, a multi-perspective imaging application is invoked at each of the first and at least second mobile devices in accordance with the mutual interaction scheme. With reference to the system ofFIGS. 2A-2F, and the system ofFIG. 3A-3BandFIG. 3T, responsive to determining that the respective acquired position properties each of mobile devices300A and300B comply with the ninth position scheme of mutual interaction scheme360, mobile device300A invokes a multi-perspective imaging application. Similarly, responsive to determining that the respective acquired position properties each of mobile devices300A and300B comply with the ninth position scheme of mutual interaction scheme360, mobile device300B invokes the multi-perspective imaging application.

In procedure568, the invoking of the multi-perspective imaging application at each of the first and at least second mobile devices causes at least one image of the object to be captured by each of the first and at least second mobile devices. With reference to the system ofFIGS. 2A-2F, and the system ofFIGS. 3A-3BandFIG. 3T, mobile device300A captures an image of object240from a perspective P1at time T1, and mobile device300B captures an image of object240from a perspective P2at time T2. In one embodiment, the images captured by mobile devices300A and300B are captured within a predefined time-limit ε, i.e. |T1−T2|<ε. In another embodiment the images are captured simultaneously, or almost simultaneously.

In procedure570, simultaneous with capturing each image, metadata associated with each image is acquired. The associated metadata includes position and orientation properties associated with the captured image, and a time stamp associated with the captured image. With reference to the system ofFIGS. 2A-2F, and the system ofFIGS. 3A-3BandFIG. 3T, while capturing the image of object240, mobile device300A acquires metadata comprising: position and orientation properties corresponding to P1and a time-stamp corresponding to T1. Mobile device300A stores the captured image with the metadata at memory store332A of mobile device300A. Similarly, while capturing the image of object240, mobile device300B acquires metadata comprising: position and orientation properties corresponding to P2and a time-stamp corresponding to T2. Mobile device300B stores the captured image with the metadata at memory store332B of mobile device300B.

In procedure572, the images captured by each of the first and at least second mobile devices and the associated metadata are processed. With reference to the system ofFIGS. 2A-2F, and the system ofFIGS. 3A-3BandFIG. 3T, mobile devices300A and300B each provide the respective captured images with the associated metadata to image processor354via respective transceivers304A and304B of mobile devices300A and300B, and network352.

In procedure574, the multiple captured images with the associated metadata are received from the first and at least second mobile devices by a processor. With reference to the system ofFIGS. 2A-2F, and the system ofFIGS. 3A-3BandFIG. 3T, image processor354receives the images and associated metadata from mobile devices300A and300B via network352and respective transceivers304A and304B.

In procedure576, the multiple captured images are processed using the associated metadata to perform any of: creating a panoramic image of the object, creating a multi-perspective image of the object, and tracking the object. The results of the imaging processing step of procedure576may be provided. With reference to the system ofFIGS. 2A-2F, and the system ofFIGS. 3A-3BandFIG. 3T, image processor354processes the received images and metadata to create any of: a panoramic image of object240, and a multi-perspective image of object240. Additionally, or alternatively, image processor354processes the received images and metadata to track object240in real-time. Additionally, image processor354may provide the resulting image created from the received images and metadata to mobile devices300A and300B.

Reference is now made toFIG. 5D, which is a schematic illustration of a method for implementing a mutually interactive virtual pen application, in accordance with another embodiment of the disclosed technique. The method ofFIG. 5Dis a continuation of the method described above with respect toFIG. 5A.

In procedure580, the first mobile device is maneuvered to trace a trajectory while a visible trace, corresponding to the trajectory, is deposited. With reference to the systems ofFIGS. 2A-2G,FIGS. 3A-3B, andFIGS. 4A-4B, mobile device400A is provided with a writing utensil452having a tip458. Writing utensil452, together with mobile device400A, are maneuvered in a trajectory that causes tip458to deposit a visible trace456on a sheet of paper454corresponding to the trajectory.

In procedure582, the position property of the first mobile device is acquired, corresponding to the acquisition of the first position property of procedure502ofFIG. 5A. With reference to the systems ofFIGS. 2A-2G,FIGS. 3A-3B, andFIGS. 4A-4B, IMU430A of mobile device400A acquires the trajectory resulting from maneuvering writing utensil452to write visible trace456. Additionally of alternatively, camera424A of mobile device400A acquires the trajectory resulting from maneuvering writing utensil452to write visible trace456.

In procedure584, the acquired position property is determined to comply with one of the position schemes of the mutual interaction scheme by determining that the acquired position property corresponds to the deposited visible trace. With reference to the systems ofFIGS. 2A-2G,FIGS. 3A-3B, andFIGS. 4A-4B, mobile device400A determines that the acquired trajectory complies with the tenth position scheme of mutual interaction scheme360by determining that the acquired trajectory corresponds to visible trace456. For example, camera424A of mobile device400A captures an image of visible trace456, and the captured image is matched to the trajectory acquired by IMU430A of mobile device400A.

In procedure586, the at least second mobile device is triggered to display a bit map corresponding to the visible trace. With reference to the systems ofFIGS. 2A-2G,FIGS. 3A-3B, andFIGS. 4A-4B, mobile device400A triggers mobile device400B to display a bit map corresponding to visible trace456, via respective transceivers404A and404B. Any of mobile devices400A and400B may derive the bit map, such as from the image of visible trace456captured by camera424A of mobile device400A, or alternatively, from the trajectory of mobile device452acquired by IMU430A of mobile device400A.

In procedure588, the bit map corresponding to the visible trace is displayed. Additionally, the bit map is stored at a memory of the at least second mobile device With reference to the systems ofFIGS. 2A-2G,FIGS. 3A-3B, andFIGS. 4A-4B, mobile device400B displays the bit map at user interface402B of mobile device400B. Additionally, mobile device400B stores the bit map a memory store432B of mobile device400B.

Reference is now made toFIG. 6, which is a schematic illustration of a method for initiating at least one mutual interaction scheme between a mobile device and a stationary computer, in accordance with another embodiment of the disclosed technique.

In procedure600, at least one mutual interaction scheme is associated between a mobile device and a stationary computer. The mutual interaction scheme may be provided, or otherwise made accessible, to each of the mobile device and computer via a network from a cloud service. The mutual interaction scheme associates at least one position scheme with at least one respective action, where the at least one position scheme relates to at least one of: an absolute orientation, a relative orientation, an absolute position, a relative position, an absolute trajectory, and a relative trajectory of the mobile device. With reference to the system ofFIGS. 2A-2F, and the system ofFIG. 3U, each of mobile device300A and computer382are provided with access to mutual interaction scheme360, via transceiver304A of mobile device300A and transceiver392of computer382.

In procedure602, a position property of the mobile device is acquired. The acquired position property of the mobile device relates to at least one of: an absolute orientation, a relative orientation, an absolute position, a relative position, an absolute trajectory, and a relative trajectory of the mobile device. With reference to the system ofFIGS. 2A-2F, and the system ofFIG. 3U, IMU330A of mobile device300A acquires the position property of mobile device300A using the methods described above. Additionally or alternatively, camera324A of mobile device300A captures an image, such as of a reference object, and the position property of mobile device300A is acquired from the image captured by camera324A. Additionally or alternatively, a camera388of display384captures an image of mobile device300A, and computer382determines the position property of mobile device300A using the acquired image.

In procedure604, compliance of the acquired position property of the mobile device with at least one of the at least one position schemes of the mutual interaction scheme is determined. With reference to the system ofFIGS. 2A-2F, and the system ofFIG. 3U, mobile device300A determines that the respective acquired position property complies with at least one of the position schemes of the mutual interaction scheme.

In procedure606, in response to the compliance, an execution of one of the at least one respective actions is triggered on the computer. The respective triggered action is associated with the at least one position scheme with which the position property of the mobile device complies, in accordance with the mutual interaction scheme. With reference to the system ofFIGS. 2A-2F, and the system ofFIG. 3U, computer382is triggered to execute the respective action associated with the position scheme with which the position property of mobile device300A complies.

In procedure608, the respective triggered action is executed by the computer in response to the compliance of the acquired position property of the mobile device.

In procedure610, the triggered action causes the computer to virtually couple the mobile device to a rendition of a three-dimensional object, and manipulating the rendition of the three dimensional object to correspond to the acquired position property of the mobile device. With reference to the system ofFIGS. 2A-2F, and the system ofFIG. 3U, mobile device300A is virtually coupled with a three dimensional object (not shown) displayed on display384. Subsequently, computer382manipulates a rendition of the three dimensional object on display384to correspond to the acquired position property of mobile device300A.

The present invention relates to a system, a method, or a computer program product. The computer program product may comprise memory store132having computer readable program instructions thereon for causing processor134to carry out aspects of the present invention.

Computer readable program instructions related to herein can be downloaded to mobile device100from a computer readable storage medium via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network and transceiver104. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. A network adapter card or network interface in each mobile device100receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in memory store132within mobile device100.

These computer readable program instructions may be provided to processor134of mobile device100to produce a machine, such that the instructions, which execute via processor134of mobile device100, create means for implementing the procedures specified in the flowcharts. These computer readable program instructions may also be stored in memory store132that can direct mobile device100to function in a particular manner, such that memory store132having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the procedures specified in the flowcharts. The computer readable program instructions may also be loaded onto mobile device100to cause a series of operational steps to be performed on mobile device100to produce a computer implemented process, such that the instructions which execute on mobile device100implement the procedures specified in the flowcharts.

The flowcharts in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each procedure in the flowchart may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the procedures may occur out of the order noted in the Figures. For example, two procedures shown in succession may, in fact, be executed substantially concurrently, or the procedures may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each procedure, and combinations thereof, of the flowchart illustrations can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.