Patent ID: 12197885

DETAILED DESCRIPTION

Although the above operations are described sequentially, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be rearranged. A process may have additional steps not included inFIG.2. Furthermore, examples of the methods may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code message segments to perform the necessary tasks may be stored in a non-transitory computer-readable medium such as a storage medium. Processors may perform the described tasks.

The embodiments described herein utilize a communication system comprising a plurality of portable communication devices (PCDs) that are managed by a cloud based communication platform server (CPS). Each individual portable communication device establishes a connection with the CPS. The CPS may then manage communication among the PCDs. It should be noted that a PCD may comprise a purpose built device like those made by Relay, Inc., a smartphone executing an application that is configured to communicate with the CPS, or a laptop or other computer executing an application that is configured to communicate with the CPS.

In operation, the various PCDs may coordinate with the CPS to create logical communication channels. A logical communication channel may be characterized as a specific grouping of PCDs. Each PCD may be a member of multiple logical channels. The CPS maintains a list of logical channels and the PCDs that are members of the logical channel. A PCD may then switch itself to a logical channel for purposes of communicating with other PCDs on that logical channel. The CPS, in some situations, can manipulate the logical channels on a given PCD causing it to switch to another logical channel if necessary.

Each PCD is enabled with a user profile. The user profile is associated with a person and is unique to that person. Any given PCD may be configured with a user profile and user profiles may be deleted or added to PCDs via an administration function. A PCD may only be configured with a single user profile at any given time.

There are a variety of settings that may be associated with a user profile. These settings may then influence or control what the PCD can access on the network. For instance, a user profile may contain the role or job function (e.g., security, housekeeping, front desk, etc.) of the user which in turn may affect which logical channels that PCD may be a party to. Similarly, the job function may define a geo-fenced boundary of where that PCD may be permitted.

Of more relevance to the present disclosure, the user profile may include language settings. For example, the user may specify one or more languages in which the user may be fluent or conversant. If there is more than one language associated with the profile, one may be deemed the default language. Moreover, the user may specify one language for outbound communications and another language for inbound communications.

Consider a logical channel comprised of four (4) users. The logical channel may have been created and given a label such as, for instance, HQ Security. Each user is associated with a PCD and each PCD is provisioned with that user's user profile. In this example, we will refer to the users as Bob, Maria, Pierre, and Alice. Bob speaks only English, Maria speaks Spanish, Pierre speaks both French and English, and Alice speaks Chinese. While Pierre speaks both French and English, he prefers to speak in French but listen in English.

Any user may initiate a conversation on the HQ Security channel by speaking a message into the PCD while it is set to the HQ Security channel and is in transmit mode. It is assumed that all the PCDs are also set to the HQ Security channel for purposes of this illustration. In general, a user will speak a message in their default language which is relayed up to the CPS. The CPS will determine the language settings for the PCD that sent the audio message. This may be done either by looking up the language setting in the user profile associated with the PCD. Alternatively, the CPS can determine the spoken language using a language detection software and processes. The CPS will then convert the received audio message from speech to text in the spoken language.

The CPS then determines the other PCDs that comprise the HQ Security channel and the language settings (specifically the inbound communication language setting) in the user profile(s) for the other PCDs in the HQ Security channel. The CPS then translates (if necessary) the text of the converted original audio message into text in the language specified by the other PCDs in the HQ Security channel. This text is then converted to speech in the target language for each PCD.

In one embodiment, the CPS will re-broadcast the original audio message in the original language to all PCDs in the HQ Security channel. Simultaneously, the CPS will perform the translation process and relay the translated audio to each PCD in their specified language. If the receiving PCD has the same language settings as the sending PCD, no translation services are required for that receiving PCD.

In another embodiment, the CPS will not re-broadcast the original audio message in the original language to any PCDs having different language settings than the sending PCD. Rather, the CPS will perform the translation process and relay only the translated audio to each PCD in their specified language. If the receiving PCD has the same language settings as the sending PCD, no translation services are required for that receiving PCD.

When Bob is in transmit mode on the HQ Security channel and says “Is anyone on the 5th floor?”, the audio message is relayed to the CPS. The CPS determines that the audio message is in English and also determines the language settings for the other three PCDs that comprise the HQ Security channel. In this example, the CPS translates the English audio message into English text. It then translates the English text to Spanish text and then converts the Spanish text to Spanish audio for Maria's PCD and translates the English text to Chinese text and then converts the Chinese text to Chinese audio for Alice's PCD. No translation is needed for Pierre's PCD as it has specified English as the preferred language for inbound communication.

Meanwhile the CPS may have processed the original audio message from Bob by broadcasting it in its original form to the other PCDs in the HQ Security channel. Thereafter, the CPS sends the translated Spanish audio to Maria's PCD and the translated Chinese audio to Alice's PCD.

FIG.1illustrates a network environment for a wireless communication system according to the embodiments of the invention. A communications platform server (CPS)110is a cloud based remote server whose function is to manage communications among multiple portable communication devices (PCDs)150,152,154,156. While four (4) PCDs have been illustrated, the CPS may manage any number of PCDs.

The CPS110sits in a packet based computer network120such as, for instance, the Internet. The CPS may be configured to communicate directly with each PCD150,152,154,156in the network. In this environment, the PCDs may be purpose built communication devices capable of multiple modes of wireless communication. A PCD may also be a smartphone device160that is also capable of multiple modes of wireless communication. The PCDs150,152,154,156typically have two modes of connectivity that provide a connection with the CPS110. A first may be WiFi (e.g., 802.11 type protocols and interfaces) that utilize a WiFi access point130. A second may be cellular (e.g., cellular based RF wireless protocols and interfaces) that utilize a cellular basestation140. The WiFi access point130and the cellular basestation140each have backhaul connectivity to the packet based computer network120for direct communication with CPS110.

Anytime a PCD150,152,154,156is within range of a WiFi access point130or a cellular basestation140, it establishes a connection with the CPS and the CPS considers the PCD on-line.

FIG.2illustrates some of the data residing on a personal communication device (PCD) according to the embodiments of the invention. As mentioned above, a PCD200may store or have access to certain data necessary to enable the invention. For instance, PCD200may be loaded with a user profile202that may be comprised of, among other things, a language setting. The language setting is indicative of the language preference of a particular user when using PCD200.

PCD200may also store or have access to various device settings204or other status information. Some of the data stored may include a current logical channel setting and the current preferred received message format. The received message format may be changed by the user to reflect current preferences. For instance, when the user is in a location that is not suitable for spontaneous noisy message bursts, the current received message preference may be set to text. For participating in verbal conversations, the current received message preference may be set to voice.

FIG.3is a logic flow diagram300describing a language translation process carried out by the communication platform server (CPS)110.FIG.3may be described in the context of a four person user group (Bob, Maria, Pierre, and Alice) in which each user has their own user profile configured and executing on a PCD. The user profile may include language settings for outbound and inbound messages. In addition, it may be assumed that each PCD is on-line with the CPS and set to the same logical channel on which they are all members. Thus, when one member transmits a message on the channel, the other members will receive it. If the PCD for a member of the channel was not on-line or was not tuned to the proper channel, any inbound messages may not be received by that member's PCD or may be queued in a missed messages queue for later retrieval. Messages are generally in audio form similar to the way one would communicate using a walkie-talkie. For instance, a user engages transmit mode by depressing a button on a PCD and speaks into a microphone. Upon completion of the speech, the user releases the button to return to receive mode.

In step305, the CPS110may receive an audio message from a PCD150. For instance, Bob may engage transmit mode on his PCD150and speak the words, “Does anyone know where Jim is?” This audio message is transmitted wirelessly from Bob's PCD150via a Wifi130or cellular140connection through network120where it is received by CPS110.

CPS110first determines the language of the received audio message in step310. For instance, CPS110is aware of which PCD150sent the audio message. CPS110also knows which user profile202is currently associated with the PCD150that sent the audio message. In this example, Bob sent the audio message. The CPS110determines from Bob's user profile202that his language setting is set to English for both inbound and outbound communication. Alternatively, the CPS110may independently determine the language of the received message from Bob's PCD150by analyzing the audio itself.

Once the sending language has been determined, CPS110converts the audio into text in the same language in step315. Here, the English audio is converted into English text. CPS110also knows which logical channel Bob's PCD150is currently set to in step320and uses that knowledge to look up other PCDs152,154,156that are included on that particular logical channel in step325. Each logical channel may be given a label or name when it is created. This name is known to and stored by CPS110, along with all other user profiles202that are members of that channel. For illustrative purposes, the logical channel in use may be named HQ Security channel. In this case, CPS110determines that Maria's PCD152, Pierre's PCD154, and Alice's PCD156are all included on the HQ Security channel.

CPS110may then fork into two paths. In the first path, CPS110will broadcast, in step330, the original English audio message from Bob's PCD150to the other three PCDs152,154,156associated with Maria, Pierre, and Alice where it will be played back in English.

In the second path, CPS110will determine the language settings for each of the receiving PCDs152,154,156based on the user profile202for the PCDs152,154,156in step335. CPS110determines that Maria's user profile202is set to Spanish for inbound communications, Pierre's user profile202is set to English for inbound communications, and Alice's user profile202is set to Chinese for inbound communications. CPS110may then translate the English text into Spanish text for Maria's PCD152and Chinese text for Alice's PCD156in step340. No translation is needed for Pierre's PCD154because it is set to English which is the same language as the originally sent message. CPS110may then convert the Spanish translated text into Spanish speech for Maria's PCD152and the Chinese translated text into Chinese speech for Alice's PCD156in step345.

At this point, CPS110may determine if it has broadcast, in step330, the original English audio message from Bob's PCD150to the other three PCDs152,154,156in decision block350. If not, CPS110will wait until step330occurs prior to relaying translated audio to any PCD152,154,156needing a translation in step355. In this case, once the original audio has been broadcast in its original language, CPS110will send Maria's PCD152the translated Spanish audio and send Alice's PCD156the translated Chinese audio.

In this scenario, Maria will receive the original English audio followed immediately thereafter with a Spanish translation. Similarly, Alice will receive the original English audio followed immediately thereafter with a Chinese translation. Pierre may only receive the original English audio as his language setting specifies English and no translation is deemed necessary.

As stated earlier, the process may be implemented without first playing the original audio in the original language. For instance, Maria and Alice may only receive the translated audio in their respective languages without hearing the original message.

It should also be noted that since this is a speech-to-text followed by a text translation and finally a text-to-speech system, combinations of text and audio may be supported. For example, if the sending PCD150is text enabled, the original message may be sent as text which would negate the need for the first speech to text step. Similarly, any receiving PCDs152,154,156that are text enabled and have a preference of receiving text rather than audio would not need to have the text-to-speech step performed.

One can envision a scenario in which a member of the group may be in either a very noisy environment making audio difficult to hear or in a meeting where sudden audio emissions would be unwanted. In such cases, the PCDs152,154,156may be set to receive incoming messages via text.

FIG.4is another logic flow diagram describing a language translation process carried out by a communication platform server.FIG.4may also be described in the context of a four person user group (Bob, Maria, Pierre, and Alice) in which each user has their own user profile configured and executing on a PCD. The user profile may include language settings for outbound and inbound messages. In addition, it may be assumed that each PCD is on-line with the CPS and set to the same logical channel on which they are all members. Thus, when one member transmits a message on the channel, the other members will receive it. If the PCD for a member of the channel was not on-line or was not tuned to the proper channel, any inbound messages may not be received by that member's PCD or may be queued in a missed messages queue for later retrieval. Messages may be in audio form similar to the way one would communicate using a walkie-talkie or text form similar to SMS or other text/data based direct messaging (DM) systems.

In step405, the CPS110may receive a message from a PCD150. For instance, Bob may engage transmit mode on his PCD150and speak the words, “Does anyone know where Jim is?” This audio message is transmitted wirelessly from Bob's PCD150via a Wifi130or cellular140connection through network120where it is received by CPS110. Alternatively, Bob may send the message as a text message using a text based interface.

CPS110may first determine whether the received message is a voice message or a text message at decision block410. The incoming data packets associated with the received message may be characterized as either voice or text. If the received message is determined to be voice, the voice message is converted to a first text message in step415using speech-to-text processing. If the received message is determined to be text, no further processing on the received message need be performed.

CPS110then determines the language setting of the PCD150that sent the message in step420. For instance, CPS110is aware of which PCD150sent the message. CPS110also knows which user profile202is currently associated with the PCD150that sent the message. In this example, Bob sent the message. The CPS110determines from Bob's user profile202that his language setting is set to English for both inbound and outbound communication. Alternatively, the CPS110may independently determine the language of the received message from Bob's PCD150by analyzing the audio itself if the message was in voice format or the words if the message was in text format.

In our example, CPS110also knows which logical channel Bob's PCD150is currently set to in step425and uses that knowledge to look up other PCDs that are included on that particular logical channel in step430. Each logical channel may be given a label or name when it is created. This name is known to and stored by CPS110along with all other user profiles that are members of that channel. For illustrative purposes, the logical channel in use may be named HQ Security channel. In this case, CPS110determines that Maria's PCD152, Pierre's PCD154, and Alice's PCD156are all included on the HQ Security channel.

CPS110may then determine the language settings for each of the receiving PCDs152,154,156based on the user profile202for the PCDs152,154,156in step435. CPS110determines that Maria's user profile202is set to Spanish for inbound communications, Pierre's user profile202is set to English for inbound communications, and Alice's user profile202is set to Chinese for inbound communications. CPS110may then translate the English text into Spanish text for Maria's PCD152and Chinese text for Alice's PCD156in step440. No translation is needed for Pierre's PCD154because it is set to English which is the same language as the originally sent message.

CPS110may then determine the received message format setting for each PCD152,154,156in decision block445. For instance, any given PCD may be set to receive messages in voice format or text format. This is a setting controllable by the user. For PCDs set to receive messages as text, CPS110will send the second text message to that PCD in the language specified by the user's user profile202in step450. In this example, Maria's PCD152may be set to receive text messages. The CPS110will forward the second text message which is now in Spanish to Maria's PCD152.

For PCDs set to receive messages as voice, CPS110will convert the second text message intended for that PCD in the language specified by the user profile202in step455. In this example, Pierre's PCD154and Alices's PCD156may be set to receive voice messages. The CPS110will then forward the voice message in English to Pierre's PCD154and in Chinese to Alice's PCD156in step460.

In this scenario, Bob sent the original message from his PCD150as a text message on the HQ Security channel. Maria set her PCD152to receive messages in text format so she receives a Spanish translation text message of Bob's text message. Pierre set his PCD154to receive messages in voice format so he receives a text-to-speech English version of Bob's text message. Alice set her PCD156to receive messages in voice format so she receives a text-to-speech Chinese version of Bob's text message.

As stated earlier, the process may be implemented without first playing the original audio in the original language. For instance, Maria and Alice may only receive the translated audio in their respective languages without hearing the original message.

The methods, devices, and systems described above are examples. Various configurations may omit, substitute, or add various procedures or components. For example, in alternative configurations, the methods may be performed in a different order. In another example, the methods may be performed with fewer steps, more steps, or in combination. In addition, certain configurations may be combined in various configurations. As technology evolves, many of the elements are examples and do not limit the scope of the disclosure or claims.

While some examples of methods, devices, and systems herein are described in terms of software executing on various machines, the methods and systems may also be implemented as specifically-configured hardware, such as a FPGA specifically to execute the various methods according to this disclosure. For example, examples can be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in a combination thereof. In one example, a device may include a processor or processors. The processor comprises a computer-readable medium, such as a RAM coupled to the processor. The processor executes computer-executable program instructions stored in memory, such as executing one or more computer programs. Such processors may comprise a microprocessor, a digital signal processor (“DSP”), an application-specific integrated circuit (“ASIC”), FPGAs, and state machines. Such processors may further comprise programmable electronic devices such as PLCs, programmable interrupt controllers (“PICs”), programmable logic devices (“PLDs”), programmable read-only memories (“PROMs”), electronically programmable read-only memories (“EPROMs” or EEPROMs), or other similar devices.

Such processors may comprise, or may be in communication with, media, for example one or more non-transitory computer-readable media, that may store processor-executable instructions that, when executed by the processor, can cause the processor to perform methods according to this disclosure as carried out, or assisted, by a processor. Examples of non-transitory computer-readable medium may include, but are not limited to, an electronic, optical, magnetic, or other storage device capable of providing a processor, such as the processor in a web server, with processor-executable instructions. Other examples of non-transitory computer-readable media include, but are not limited to, a floppy disk, CD-ROM, magnetic disk, memory chip, ROM, RAM, ASIC, configured processor, all optical media, all magnetic tape or other magnetic media, or any other medium from which a computer processor can read. The processor, and the processing, described may be in one or more structures, and may be dispersed through one or more structures. The processor may comprise code to carry out methods (or parts of methods) according to this disclosure.

Reference herein to an example or implementation means that a particular feature, structure, operation, or other characteristic described in connection with the example may be included in at least one implementation of the disclosure. The disclosure is not restricted to the particular examples or implementations described as such. The appearance of the phrases “in one example,” “in an example,” “in one implementation,” or “in an implementation,” or variations of the same in various places in the specification does not necessarily refer to the same example or implementation. Any particular feature, structure, operation, or other characteristic described in this specification in relation to one example or implementation may be combined with other features, structures, operations, or other characteristics described in respect of any other example or implementation.

Use herein of the word “or” is intended to cover inclusive and exclusive OR conditions. In other words, A or B or C includes any or all of the following alternative combinations as appropriate for a particular usage: A alone; B alone; C alone; A and B only; A and C only; B and C only; and A and B and C.

Specific details are given in the description to provide a thorough understanding of example configurations (including implementations). However, configurations may be practiced without these specific details. For example, well-known circuits, processes, algorithms, structures, and techniques have been shown without unnecessary detail in order to avoid obscuring the configurations. This description provides example configurations only, and does not limit the scope, applicability, or configurations of the claims. Rather, the preceding description of the configurations will provide those skilled in the art with an enabling description for implementing described techniques. Various changes may be made in the function and arrangement of elements without departing from the spirit or scope of the disclosure.

The foregoing description of some examples has been presented only for the purpose of illustration and description and is not intended to be exhaustive or to limit the disclosure to the precise forms disclosed. Numerous modifications and adaptations thereof will be apparent to those skilled in the art without departing from the spirit and scope of the disclosure. It is intended that the scope of the present disclosure be limited not by this detailed description, but rather by the claims appended hereto. Future filed applications claiming priority to this application may claim the disclosed subject matter in a different manner, and may generally include any set of one or more limitations as variously disclosed or otherwise demonstrated herein.