Patent Description:
The present subject matter relates to electronic devices and, more particularly, to using emojis in communications between electronic devices.

Textual communication is a common means of communication between users of electronic devices (e.g., texting). Textual communication is conventionally performed using standardized computer fonts. Emojis can be used in text communications to enhance communications between the users.

<CIT> discloses a system and method for entering icons and other pieces of media through a text entry interface.

The drawing figures depict one or more implementations, by way of example only, not by way of limitations.

One aspect of the present disclosure describes use of emoji-first messaging. Text messaging is automatically converted to emojis by an emoji-first application so that only emojis are communicated from one client device to another client device. Each client device has a library of emojis that are mapped to words, which libraries are customizable and unique to the users of the client devices, such that the users can communicate secretly in code. Upon receipt of a string of emojis, a user can select the emoji string, such as by tapping it, to convert to text if desired, for a predetermined period of time. This disclosure provides a more engaging user experience.

The description that follows includes systems, methods, techniques, instruction sequences, and computing machine program products illustrative of examples of the disclosure. In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide an understanding of various examples of the disclosed subject matter. It will be evident, however, to those skilled in the art, that examples of the disclosed subject matter may be practiced without these specific details.

<FIG> is a block diagram illustrating a system <NUM>, according to some examples, configured to enable users of client devices to communicate with one another using only emojis, referred to in this disclosure as emoji-first messaging. Text created by users is automatically converted to emojis based on customizable libraries. The system <NUM> includes two or more client devices <NUM>. The client device <NUM> includes, but is not limited to, a mobile phone, eyewear, desktop computer, laptop, portable digital assistants (PDA), smart phone, tablet, ultrabook, netbook, laptop, multi-processor system, microprocessor-based or programmable consumer electronic, game console, set-top box, computer in a vehicle, or any other communication device that a user may utilize to access the system <NUM>. The client devices <NUM> include a display displaying information, e.g., in the form of user interfaces. In further examples, the client device <NUM> includes one or more of touch screens, accelerometers, gyroscopes, cameras, microphones, global positioning system (GPS) devices, and so forth. The client device <NUM> may be a device of a user that is used to access and utilize an online social platform.

For example, client device <NUM> is a device of a given user who uses a client application <NUM> on an online social platform, a gaming platform, and communication applications. Client device <NUM> accesses a website, such as an online social platform hosted by a server system <NUM>. The user inputs login credentials associated with the user. Server system <NUM> receives the request and provides access to the online social platform.

A user of the client device <NUM> launches and engages a client application <NUM> hosted by the server system <NUM>, which in one example is a messaging application. The client device <NUM> includes an emoji-first module <NUM> including a processor running client code for performing the emoji-first messaging on the client device <NUM>. The emoji-first module <NUM> automatically converts text words entered by a user on a client device <NUM> to generate a string of one or more emojis based on a customizable library <NUM>. The library <NUM> contains a list of emojis matched to one or more words of text. The messaging client application <NUM> communicates the emoji string between client devices <NUM>. When a user of another client device <NUM> having the same customizable library <NUM> receives the generated emoji string, it displays the string of emojis on a device display, and the user can optionally select converting the received string of emojis to text, such as by tapping on the emoji string.

One or more users may be a person, a machine, or other means of interacting with the client device <NUM>. In examples, the user may not be part of the system <NUM> but may interact with the system <NUM> via the client device <NUM> or other means. For instance, the user may provide input (e.g., touch screen input, alphanumeric input, verbal input, or visual input) to the client device <NUM> and the input may be communicated to other entities in the system <NUM> (e.g., third-party servers <NUM>, server system <NUM>, etc.) via a network <NUM> (e.g., the Internet). In this instance, the other entities in the system <NUM>, in response to receiving the input from the user, may communicate information to the client device <NUM> via the network <NUM> to be presented to the user. In this way, the user interacts with the various entities in the system <NUM> using the client device <NUM>.

One or more portions of the network <NUM> may be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), a portion of the Internet, a portion of the public switched telephone network (PSTN), a cellular telephone network, a wireless network, a WiFi network, a <NUM> LTE network, another type of network, or a combination of two or more such networks.

The client device <NUM> may access the various data and applications provided by other entities in the system <NUM> via a web client <NUM> (e.g., a browser) or one or more client applications <NUM>. The client device <NUM> may include one or more client application(s) <NUM> (also referred to as "apps") such as, but not limited to, a web browser, messaging application, multi-player gaming application, electronic mail (email) application, an e-commerce site application, a mapping or location application, and the like.

In some examples, one or more client application(s) <NUM> are included in a given one of the client device <NUM>, and configured to locally provide the user interface and at least some of the functionalities, with the client application(s) <NUM> configured to communicate with other entities in the system <NUM> (e.g., third-party server(s) <NUM>, server system <NUM>, etc.), on an as-needed basis, for data processing capabilities not locally available (e.g., to access location information, to authenticate a user, etc.). Conversely, one or more client application(s) <NUM> may not be included in the client device <NUM>, and then the client device <NUM> may use its web browser to access the one or more applications hosted on other entities in the system <NUM> (e.g., third-party server(s) <NUM>, server system <NUM>, etc.).

The server system <NUM> provides server-side functionality via the network <NUM> (e.g., the Internet or wide area network (WAN)) to: one or more third party server(s) <NUM>, and one or more client devices <NUM>. The server system <NUM> includes an application server <NUM> including an application program interface (API) server <NUM>, a web server <NUM>, and one or more personalized font modules <NUM>, that may be communicatively coupled with one or more database(s) <NUM>. The one or more database(s) <NUM> may be storage devices that store data related to users of the server system <NUM>, applications associated with the server system <NUM>, cloud services, and so forth. The one or more database(s) <NUM> may further store information related to third-party server(s) <NUM>, third-party application(s) <NUM>, client device <NUM>, client application(s) <NUM>, users, and so forth. In one example, the one or more database(s) <NUM> may be cloud-based storage.

The server system <NUM> may be a cloud computing environment, according to some examples. The server system <NUM>, and any servers associated with the server system <NUM>, may be associated with a cloud-based application, in one example.

The emoji-first module <NUM> is stored on the client device <NUM> and/or server <NUM> to optimize processing efficiency. In some examples, all modules for performing a specific task are stored on the device/server performing that action. In other examples, some modules for performing a task are stored on the client device <NUM> and other modules for performing that task are stored on the server <NUM> and/or other devices. In some examples, modules may be duplicated on the client device <NUM> and the server <NUM>.

The one or more third-party application(s) <NUM>, executing on third-party server(s) <NUM> may interact with the server system <NUM> via API server <NUM> via a programmatic interface provided by the API server <NUM>. For example, one or more of the third-party applications <NUM> may request and utilize information from the server system <NUM> via the API server <NUM> to support one or more features or functions on a website hosted by the third party or an application hosted by the third party. The third-party application(s) <NUM>, for example, may provide software version analysis functionality that is supported by relevant functionality and data in the server system <NUM>.

<FIG> provides an overview of one example of communicating using the emoji-first module <NUM> among a plurality of client devices 110a-n using messaging application <NUM>. The client devices 110a-n in <FIG> each include the emoji-first module <NUM> and messaging application <NUM>, and a respective display screen 200a-n configured to display the messaging. The display screen 200a-n is also referred to as a "chat" interface.

The emoji-first module <NUM> is an application, such as an iOS app, that enables emoji-first communication between two people in a close relationship, leveraging their closeness and history with each other to foster a shared emoji vocabulary between them. Each user creates an account and specifies a single partner with whom they will use the emoji-first module <NUM>. The chat interface <NUM> allows the user pair to send and receive emoji-first messages between them, such that the messages comprise of only emojis, such as shown in <FIG>. The emoji-first module <NUM> automatically and dynamically converts/translates all text into emojis on the fly as the user types, as shown in <FIG>, by using automatic text-to-emoji mapping. The users have the option to personalize the emoji-first messages they send by defining their own text-to-emoji mappings, as shown in <FIG>, which mappings are stored in library <NUM>. Emojis can be selectively associated with words that are different than the mapping provided by Unicode CLDR data.

The chat interface <NUM> allows users to exchange emoji-first messages with their partners. That is, the users receive sequences of emoji, referred to as strings, representing a text message without being accompanied by text at first, though they may choose to view the message in text later by tapping the messages. As shown in <FIG> and <FIG>, the emoji-first messages appear in bubbles, where yellow messages indicate sent, and grey messages indicate received. The user can type, personalize, and preview their message using the three boxes <NUM>, <NUM> and <NUM> in the chat interface <NUM>. Once the user is satisfied with a message they have created in the chat interface <NUM>, they can send the message by tapping the send button <NUM> which is represented by a yellow circle containing an upward arrow. When a message is received, the receiving user only sees the emoji string at first in both the iOS notification <NUM> in the chat interface <NUM> as shown in <FIG>, and in the chat interface <NUM> as shown at <NUM> in <FIG>, where the user can tap the emoji string to reveal the fully translated message's corresponding text. Upon tapping, the revealed message will display for a predetermined time, such as <NUM> seconds in on example, which is helpful to maintain privacy.

Referring to <FIG>, there is shown a method <NUM> of operating the emoji-first application <NUM> on client device <NUM> in view of <FIG>. The method <NUM> is performed by a processor of the client device <NUM>, shown as processor <NUM> in <FIG> as will be discussed more shortly.

At block <NUM>, the recipient, referred to as "Friend <NUM>", always sees the received emoji string first, shown as the notification message <NUM> in <FIG>. The notification message <NUM> from "Friend <NUM>" includes only a string of emojis that are found in the libraries <NUM> of each client device <NUM>. The emoji-first application <NUM> sends users standard iOS push notifications whenever the user receives a message from their partner.

At block <NUM>, Friend <NUM> can long press message <NUM> from Friend <NUM> to toggle between the emoji string and text words as shown at <NUM> in <FIG>. The library <NUM> is used to map the received emojis into words. Responsively, Friend <NUM> can type a reply to Friend <NUM> in box <NUM>, and the emoji-first application <NUM> automatically and fully translates the reply to a string of emojis on the fly as shown in box <NUM>. Box <NUM> is tappable and allows Friend <NUM> to modify the automatic mappings. Box <NUM> is the "main" box that users type into, and they use their client device's standard text-based keyboard to do so. Users can enter emoji here as well through their smartphone's keyboard. Any emoji that are entered directly are transmitted directly and will not be forwarded to the translation process.

At block <NUM>, as shown in <FIG>, the word "craving" in box <NUM> is mapped to a single emoji, indicated by the matching the width of the bubble above this word in box <NUM>. The width (in pixels) of the bubble around each emoji mapping in box <NUM> matches the width (in pixels) of the corresponding input token from box <NUM>. To personalize the emoji mapping for the word "craving", Friend <NUM> selects that emoji in box <NUM> to choose a new mapping for the word "craving. " This topmost box <NUM> acts as a final preview of the emoji message that will be sent, without the artificial spacing between emoji that box <NUM> shows. Box <NUM> is also interactive. The user can select substrings of emoji within box <NUM>, as shown in <FIG> to open the "personalization menu" in box <NUM> and replace the emoji substring with their own emoji string mapping. Shown at <NUM> is a set of possible emojis that are presented to Friend <NUM>, such that Friend <NUM> can choose from the set to establish the personalized emoji for the word "craving".

At block <NUM>, as shown in <FIG>, the chosen emoji mapping now appears in box <NUM> and becomes part of the pair's shared vocabulary which is stored in library <NUM>. Both friends can view and modify the shared vocabulary in library <NUM> anytime, thereby providing a personalized and modifiable shared vocabulary.

Referring to <FIG>, there is shown an example screen <NUM> of the chat interface <NUM> showing library <NUM> displayed on a client device display <NUM>, referred to in this disclosure as an Emotionary. This screen <NUM> shows the shared vocabulary of text-to-emoji mappings (from text strings to emoji strings) that a user and their partner has created over time. The library <NUM> serves as an emoji dictionary that both partners can contribute to and draw from in their communication. The on-the-fly text-to-emoji mapping algorithm <NUM> of <FIG> uses this library <NUM> as described. There are two portions, the user's text-to-emoji mappings shown on top, and their partner's text-to-emoji mappings shown on bottom. The mappings can be sorted alphabetically or by creation date by the user. Users can add new mappings to the library <NUM> in two ways. The first way is via the "Add" button <NUM> on the screen, and the second way is through the display <NUM> itself, by simply changing any automatically generated text-to-emoji mapping. The combined library <NUM> allows users to utilize both their and their partner's mappings when typing messages. The emoji-first application <NUM> prioritizes a user's own library <NUM> during message translation whenever their partner has a competing mapping.

<FIG> is a high-level functional block diagram of an example client device <NUM> including a client device that communicates via network <NUM> with server system <NUM> of <FIG>. Display <NUM> is a touch screen type display, although other non-touch type displays can be used. Examples of touch screen type client devices <NUM> that may be used include (but are not limited to) a smart phone, a personal digital assistant (PDA), a tablet computer, a laptop computer, eyewear, or other portable device. However, the structure and operation of the touch screen type client devices is provided by way of example, and the subject technology as described herein is not intended to be limited thereto. For purposes of this discussion, <FIG> therefore provides a block diagram illustration of the example client device <NUM> having a touch screen display for displaying content and receiving user input as (or as part of) the user interface. Client device <NUM> also includes a camera(s) <NUM>, such as visible light camera(s), and a microphone <NUM>.

The activities that are the focus of discussions here involve the emoji-first messaging, and also the personalized library of emojis that are shared between two users of client devices <NUM>. The emoji-first application <NUM> and the library <NUM> may be stored in memory <NUM> for execution by CPU <NUM>, such as flash memory 640A or RAM memory 640B.

As shown in <FIG>, the client device <NUM> includes at least one digital transceiver (XCVR) <NUM>, shown as WWAN XCVRs, for digital wireless communications via a wide area wireless mobile communication network <NUM>. The client device <NUM> also includes additional digital or analog transceivers, such as short range XCVRs <NUM> for short-range network communication, such as via NFC, VLC, DECT, ZigBee, Bluetooth™, or WiFi. For example, short range XCVRs <NUM> may take the form of any available two-way wireless local area network (WLAN) transceiver of a type that is compatible with one or more standard protocols of communication implemented in wireless local area networks, such as one of the Wi-Fi standards under IEEE <NUM>, <NUM> LTE and <NUM>.

To generate location coordinates for positioning of the client device <NUM>, the client device <NUM> can include a global positioning system (GPS) receiver (not shown). Alternatively, or additionally, the client device <NUM> can utilize either or both the short range XCVRs <NUM> and WWAN XCVRs <NUM> for generating location coordinates for positioning. For example, cellular network, WiFi, or Bluetooth™ based positioning systems can generate very accurate location coordinates, particularly when used in combination. Such location coordinates can be transmitted to the eyewear device over one or more network connections via XCVRs <NUM>.

The transceivers <NUM>, <NUM> (network communication interface) conforms to one or more of the various digital wireless communication standards utilized by modern mobile networks. Examples of WWAN transceivers <NUM> include (but are not limited to) transceivers configured to operate in accordance with Code Division Multiple Access (CDMA) and 3rd Generation Partnership Project (3GPP) network technologies including, for example and without limitation, 3GPP type <NUM> (or 3GPP2) and LTE, at times referred to as "<NUM>", and <NUM>. For example, the transceivers <NUM>, <NUM> provide two-way wireless communication of information including digitized audio signals, still image and video signals, web page information for display as well as web related inputs, and various types of mobile message communications to/from the client device <NUM> for user identification strategies.

Several of these types of communications through the transceivers <NUM>, <NUM> and a network, as discussed previously, relate to protocols and procedures in support of communications with the server system <NUM> for obtaining and storing friend device capabilities. Such communications, for example, may transport packet data via the short range XCVRs <NUM> over the wireless connections of network <NUM> to and from the server system <NUM> as shown in <FIG>. Such communications, for example, may also transport data utilizing IP packet data transport via the WWAN XCVRs <NUM> over the network (e.g., Internet) <NUM> shown in <FIG>. Both WWAN XCVRs <NUM> and short range XCVRs <NUM> connect through radio frequency (RF) send-and-receive amplifiers (not shown) to an associated antenna (not shown).

The client device <NUM> further includes microprocessor <NUM>, shown as a CPU, sometimes referred to herein as the host controller. A processor is a circuit having elements structured and arranged to perform one or more processing functions, typically various data processing functions. Although discrete logic components could be used, the examples utilize components forming a programmable CPU. A microprocessor for example includes one or more integrated circuit (IC) chips incorporating the electronic elements to perform the functions of the CPU. The processor <NUM>, for example, may be based on any known or available microprocessor architecture, such as a Reduced Instruction Set Computing (RISC) using an ARM architecture, as commonly used today in client devices and other portable electronic devices. Other processor circuitry may be used to form the CPU <NUM> or processor hardware in smartphone, laptop computer, and tablet.

The microprocessor <NUM> serves as a programmable host controller for the client device <NUM> by configuring the device to perform various operations, for example, in accordance with instructions or programming executable by processor <NUM>. For example, such operations may include various general operations of the client device <NUM>, as well as operations related to emoji-first messaging using emoji-first application <NUM>, and also personalized libraries <NUM> mapping emojis to text between a two or more users. Although a processor may be configured by use of hardwired logic, typical processors in client devices are general processing circuits configured by execution of programming.

The client device <NUM> includes a memory or storage device system, for storing data and programming. In the example, the memory system may include a flash memory 640A and a random access memory (RAM) 640B. The RAM 640B serves as short term storage for instructions and data being handled by the processor <NUM>, e.g., as a working data processing memory. The flash memory 640A typically provides longer term storage.

Hence, in the example of client device <NUM>, the flash memory 640A is used to store programming or instructions for execution by the processor <NUM>. Depending on the type of device, the client device <NUM> stores and runs a mobile operating system through which specific applications, including application <NUM>. Examples of mobile operating systems include Google Android®, Apple iOS® (I-Phone or iPad devices), Windows Mobile®, Amazon Fire OS®, RIM BlackBerry® operating system, or the like.

The terms and expressions used herein are understood to have the ordinary meaning as is accorded to such terms and expressions with respect to their corresponding respective areas of inquiry and study except where specific meanings have otherwise been set forth herein. Relational terms such as first and second and the like may be used solely to distinguish one entity or action from another without necessarily requiring or implying any actual such relationship or order between such entities or actions. The terms "comprises," "comprising," "includes," "including," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises or includes a list of elements or steps does not include only those elements or steps but may include other elements or steps not expressly listed or inherent to such process, method, article, or apparatus. An element preceded by "a" or "an" does not, without further constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

In addition, in the foregoing Detailed Description, it can be seen that various features are grouped together in various examples for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed examples require more features than are expressly recited in each claim.

The examples illustrated herein are described in sufficient detail to enable those skilled in the art to practice the teachings disclosed. Other examples may be used and derived therefrom, such that structural and logical substitutions and changes may be made without departing from the scope of this disclosure. The Detailed Description, therefore, is not to be taken in a limiting sense, and the scope of various examples is defined only by the appended claims.

Claim 1:
A client device (<NUM>, 110a), comprising:
a user interface (<NUM>, 200a) configured to allow a user to enter a string of text comprising a message, the user interface configured to display an incoming message from a second client device (110b, 110c);
a memory (<NUM>) including a customizable library (<NUM>) configured to map the string of text to customized emojis, wherein a user can selectively map the customized emojis to each of a plurality of words to generate the customized library;
a processor (<NUM>) coupled to the memory (<NUM>); and
instructions stored in the memory (<NUM>) that, when executed by the processor (<NUM>), configure the processor to:
automatically convert the string of text to a string of the customized emojis, such that all the text is automatically converted to the customized emojis to automatically generate the string of customized emojis representing the message without user input;
send the string of customized emojis representing the message to the second client device;
share the customizable library (<NUM>) with only a selected said second client device to create a private message exchange; and
toggle a displayed incoming emoji message from the second client device to text and automatically toggle the text back to only emojis after a period of time.