Adaptable schema based payloads

An approach is provided for providing adaptable schema based payloads for state advertisement messages. In some implementations, multiple schemas (e.g., payload data and structure definition) can be used to flexibly and dynamically define advertisement message payloads. For example, the computing device can use advertising message to share and broadcast information between devices by using adaptable schema based payloads. The computing device can store schemas, receive service advertising data, and determine a schema for encoding service advertising data into an advertising message. The computing device can generate the advertising message, encode a schema identifier into the advertising message header, and encode device state data into the advertising message payload according to the determined schema. In some implementations, the computing device can receive advertising message, determine a schema based on a schema identifier in the message, and extract device state data from the advertising message according to the determined schema.

TECHNICAL FIELD

The disclosure generally relates to encoding data for transmission between devices.

BACKGROUND

Many service providers and device manufactures use advertising messages to share and broadcast information between devices. For example, a device can advertise the availability of certain services or state information by broadcasting a Bluetooth Low Energy (BTLE) message that indicates the current state of an application or service. However, BTLE messages are limited in size. With all of the services and state information shared between devices using BTLE, a static payload definition will not provide enough flexibility to handle all of the data that needs to be transmitted between devices. Moreover, if a static BTLE payload definition is used, payload space (e.g., bits) may be wasted when not used by a corresponding service or application data.

SUMMARY

An approach is provided for providing adaptable schema based payloads for state advertisement messages. In some implementations, multiple schemas (e.g., payload data and structure definition) can be used to flexibly and dynamically define advertisement message payloads. For example, the computing device can use advertising message to share and broadcast information between devices by using adaptable schema based payloads. The computing device can store schemas, receive service advertising data, and determine a schema for encoding service advertising data into an advertising message. The computing device can generate the advertising message, encode a schema identifier into the advertising message header, and encode device state data into the advertising message payload according to the determined schema. In some implementations, the computing device can receive advertising message, determine a schema based on a schema identifier in the message, and extract device state data from the advertising message according to the determined schema.

Particular implementations provide at least the following advantages. The payload structure for state advertising messages can be dynamically redefined to accommodate new services, applications, and device features without having to load new software or operating system on the device. By flexibly or dynamically defining state advertising message payloads, fewer bits of the advertising message are wasted; the limited space available for advertisement message can be used more efficiently. By flexibly or dynamically defining state advertising message payloads, more different types of service, application, and/or device state information can be supported.

DETAILED DESCRIPTION

FIG. 1is a block diagram of an example system100for providing adaptable schema based payloads for advertising messages. In some implementations, system100can include user device102and/or user device122. For example, user device102and122can be a computing device, such as a smartphone, tablet computer, television, desktop computer, laptop computer, household device, electronic book device, game device, streaming media device, set top box, automobile audio/video system, smart watch, smart glasses and/or other mobile or wearable device, the accessories and peripherals of these devices, or any combination thereof.

In some implementations, advertising messages can be used to coordinate services between different user devices or enable features on different user devices. For example, user device102and/or user device122can include various services (e.g., software applications, processes, operating system services, etc.) that provide a variety of services and features for the user of user device102and/or user device122. These services can be configured to present media items (e.g., movies, music, videos, etc.) and/or present informative notifications to the user by generating audio, video, and/or haptic representations of data, as described further below. User device102and/or user device122can provide these services more intelligently when user device102and/or user device122have information about the current device state, current user activity, and/or services provided by other devices, such as user device102or user device122.

As an example use case, a user may start using service on user device102and wish to continue using the service on user device122. For example, a user may use a browser application on user device102(e.g., the user's desktop computer) to find a web page containing a recipe for bread. The user may wish to continue viewing the web page on user device122(e.g., a mobile device, smartphone, tablet computer, etc.) so that the user can take the recipe into the kitchen to bake the bread. To enable the transfer of the web browser state information (e.g., the current web page), user device102can broadcast a state advertisement message to user device122that indicates the user is using the browser application on user device102and is viewing the web page containing the recipe. For example, the state advertisement message can include an identifier for the browser application and/or the URL for the web page.

As another example, user device102and user device122can coordinate features or work to be performed by each device. For example, a user may have multiple devices (e.g., user device102and user device122) that receive notifications to present to the user. It may be annoying to the user to have the notifications show up on several devices. For example, each device may produce a sound when the notification is received and it might be bothersome to the user to hear all of the notifications from different devices. It would be more useful for the devices to coordinate presentation of notifications so that the notifications were only presented by the device the user is currently using. Thus, user device102and user device122can use the state advertising messages described herein to share information about their respective user activity levels so that only the device currently being used by the user will present received notifications.

Other services may wish to advertise state information as well. For example, the operating system of user device102may wish to advertise its charging state to other devices (e.g., user device122) so that the device the user is currently using (e.g., user device122) can present a reminder to charge user device102. User device102can use state advertising messages to advertise the need for a service. For example, if user device102is a streaming media device, user device102can advertise the need for a keyboard. User device122(e.g., a smartphone or tablet computer) can receive the advertisement and provide a keyboard that the user can use to provide textual input to user device102. These services may be active at different times and so may or may not need to broadcast current state information. If the advertising message has a static definition where specific service state data is always represented by specific bits of data in the advertising message and the specific service is not active, then those bits corresponding to the service will be left unused and wasted when the advertising message is broadcasted. However, if the advertising message has a flexible definition (e.g., using the schemas described herein), then the unused bits in the advertising message can be reassigned to other active services, as described further below.

In some implementations, system100can use multiple schemas (e.g., payload data and structure definition) to flexibly and dynamically define state advertising messages. In some implementations, user device102can advertise that certain services (e.g., a file sharing service, web browser, video application, music application, notification coordination services, remote keyboard services, etc.) are active by broadcasting an advertising message that indicates the service is active. For example, a file transfer service on user device102can broadcast its availability to other user devices (e.g. user device122) so that the user may transfer files from user device122to user device102. A video application presenting a video on user device122can broadcast its current playback state information (e.g., an identifier for video being played, current playback position, etc.) so that the user can continue watching the video started on user device122on user device102. A streaming media device can broadcast that a text input control has been selected and a keyboard is needed. Various user devices can broadcast their activity levels to coordinate presentation of notifications. When broadcasting the state advertising messages, user device102and/or user device122can select a schema that defines the structure of an advertising message payload based on what state information and/or what service is going to be advertised.

In some implementations, schemas can be selected based on security features and/or requirements. For example, a schema can be defined that specifies four bytes for encoding an authentication tag (e.g., a cryptographic hash value) or a nonce into an advertising message so that message sending devices can be authenticated by message receiving devices. Later, when stronger authentication is needed, another schema can be defined that specifies six bytes of data for encoding an authentication tag (e.g., a cryptographic hash value) or a nonce into an advertising message. This improvement in authentication strength can be implemented by simply storing a new schema that defines six bytes for the authentication data (e.g., tag, nonce, etc.) instead of four bytes.

In some implementations, the user device102can store multiple schemas (Schema ‘A’, ‘B’ . . . ‘NSCH’) corresponding to state information106a-106nfor Service1, Service2, . . . Service NSERin the schema repository104. For example, state information106can correspond to state information for various services (e.g., applications, utilities, daemons, and/or operating system services) running on user device102. For example, Service1can correspond to the virtual keyboard service described above. Service2can, for example, correspond to the notification coordination service described above.

In some implementations, advertising message encoder108can receive state advertising data from various processes on user device102and determine a schema for encoding state advertising data into a state advertising message. For example, encoder108can select a schema based on the received state advertising data and/or which services generated the state advertising data and/or sent the state advertising data to encoder108.

In some implementations, schema repository104can include multiple different schemas for various combinations of state advertising data. For example, schema “A” can be mapped to “services group A” associated with service1, service2and service3. Schema “B” can be mapped to “service group B” associated with service1and service3. To find the appropriate schema for the advertising processes or services, each schema in schema repository104can, for example, be mapped to a hash value that can be used as an index for finding the corresponding schema. In some implementations, the hash value can be generated based on service identifiers. For example, if schema “A” defines the state advertising message structure for advertising state information for service1(e.g., service identifier S1), service2(e.g., service identifier S2), and service3(e.g., service identifier S3), then the service identifiers S1, S2, and S3can be provided as input to a hashing algorithm to generate a hash value to be used as an index for looking up the corresponding schema A. Thus, when state advertising message encoder108receives state information (e.g., including service identifiers) from service1, service2and service3(or any other combination of services), encoder108can generate a hash value based on the service identifiers in the received state information and use the generated hash value as an index to lookup or determine the appropriate schema (e.g., schema A) for advertising the service state information in the schema mapping stored in schema repository104.

In some implementations, state advertising message encoder108can generate a state advertising message based on the determined schema. For example, encoder108can encode the schema identifier into the advertising message. For example, the advertising message can be a Bluetooth (e.g., Bluetooth Low Energy) advertising message that includes a header section and payload section. Encoder108can encode a schema identifier (e.g., “A”, or the corresponding hash value) corresponding to the schema associated with the advertising services into the advertising message (e.g., into the header section), as described above. Encoder108can determine the structure of the payload section of the advertising message based on the structure defined by the determined schema. For example, if the payload section includes 32 bits, the schema can define that the first ten bits are for service1data, the second ten bits are for service2data, and the last 12 bits are for service3data. Encoder108can then encode the service advertising data for the Service1, Service2, and Service3into the appropriate bits of the advertising message payload as defined by the determined schema.

In some implementations, user device102can broadcast the generated advertising message. For example, user device102can broadcast the generated advertising message using a wireless transmitter using to a wireless (e.g., peer-to-peer) technology, such as Bluetooth, Bluetooth Low Energy, peer-to-peer Wi-Fi, or some other wireless technology. The message can be broadcasted without selecting a specific recipient device. For example, any device (e.g., user device122) within range of the wireless signal transmitted by user device102can receive and/or process the advertising message generated by user device102. For example, user devices associated with a user account (e.g., a subscriber account, cloud service account, etc.) can receive and decode messages from other devices associated with the same user account. In some implementations, advertising messages can be received and decoded irrespective of user accounts or device types. For example, a device can receive and decode an advertising message even if the device is not associated with the same user as user device102. A device can receive and decode an advertising message even if the device does not have the same configuration (e.g., hardware configuration, software configuration, operating system, etc.) as user device102.

In some implementations, user device122can receive an advertising message. For example, user device122can receive the advertising message broadcasted by user device102, as described above. For example, user device122can be configured similarly to user device102. Schema repository124can be configured similarly to schema repository104. For example, schema repository124can store multiple schemas (Schema ‘A’, ‘B’ . . . ‘NSCH’) corresponding to state information126a-126nfor Service1, Service2, . . . Service NSER. as described above with reference to schema repository104. Schema repository124can include a mapping of hash values to advertising message schemas, where the hash values are generated based on service identifiers corresponding to services to be advertised using advertising messages, as described above.

When user device122receives the advertising message, advertising message decoder128can obtain the schema identifier (e.g., the hash value, or other identifier) from the advertising message header and look up the identified schema in schema repository124. After obtaining the correct schema based on the schema identifier, decoder128can determine the structure of the advertising message based on the structure defined by the obtained schema. For example, decoder128can determine that the first ten bits of the advertising message payload are for service1data, the second ten bits are for service2data, and the last 12 bits are for service3data. Decoder128can then decode or extract the service advertising data for Service1from the first ten bits to generate service1state data126a, decode or extract the service advertising data for Service2from the second 10 bits to generate service2state data126b, and decode or extract the service advertising data for Service3from the last 12 bits to generate service N state data126n. After extracting or decoding the data from the advertising message according to the schema, decoder128can send the advertised state information (126a,126b,126n) to the corresponding services (e.g., service1, service2, service3, etc.) on user device122or perform some other operation based on the service data.

While the above description describes user device102as the advertising message broadcasting device and user device122as the receiving device, each device can perform the operations of broadcaster or receiver. For example, the user device122can generate an advertising message based on the schemas, as described above with reference to user device102. User device102can receive an advertising message broadcasting and decoding the advertising message based on the schemas, as described above with reference to user device122. Thus, user device102and user device122can be configured to both broadcasting and receiving state advertising messages having adaptable schema based payloads, as described above.

FIG. 2is an example graph200illustrating for high definition202and low definition212versions of service advertising data. In some implementations, some service state information can be advertised using high definition202or low definition212versions of the state data. For example, Service1204on user device102may be configured to provide 10 bits of state data to encoder108. When encoder108receives state information from Service1(204), Service2(206), and Service3(208), as described above, the corresponding schema may allow encoder102to encode the full (e.g., high definition) 10 bits of data received from Service1. However, when encoder108receives state information from Service1(204), Service2(206), Service3(208) and Service4(216), encoder108may not have enough bits in the advertising message to encode all 10 bits of data received from Service1.

In some implementations, message payload encoder108can encode a high definition version and/or a low definition version of service state data. For example, all 10 bits of state information delivered to encoder108by Service1may not be needed or required when broadcasting service state advertisements for Service1. For example, out of the 10 bits of state data delivered to encoder108by Service1, the first 6 bits may be required data while the last 4 bits may be discretionary or informational data that may be used to enhance the service but are not necessary for providing the service. Thus, a high definition version (e.g. 10 bits) of Service1data204and a low definition version (e.g., 6 bits) of Service1state data214may be used in various schema definitions. Thus, when Service4data (e.g., 4 bits) is added to service data for Service1, Service2, and Service3, the low definition version (e.g., 6 bits instead of 10 bits) of Service1data214can be used in the corresponding schema to make room for the 4 bits of Service4data216, as illustrated by advertising message212. User device102can encode the service data received from Services1,2,3and4into advertising message212and user device122can decode the service data from the advertising message according to the schema defined for Services1,2,3, and4as described above. For example, advertising message202can include an identifier for a schema providing the high definition version of Service1data while advertising message212includes an identifier for a different schema providing the low definition version of Service1data.

Example Processes

FIG. 3is flow diagram of an example process300for generating an advertising message using adaptable schema based payloads. For example, process300can be implemented by user device102and/or user device122.

At step302, user device102can store advertising message payload schemas. As described above, the user device102can store multiple schemas (Schema ‘A’, ‘B’ . . . ‘NSCH’) for encoding state information106a-106nfor Service1, Service2, . . . Service NSER. in the schema repository104. The schemas can be obtained by user device102from a network resource (e.g., a server). User device102can update the schemas in schema repository104periodically (e.g., according to an update interval). User device102can update the schemas in schema repository as needed (e.g., when user device102received an advertising message that identifies a schema that is not in schema repository104). For example, state information106can correspond to state information for various services (e.g., applications, utilities, daemons, and/or operating system services) running on user device102. The schemas can define the organization and/or structure of advertising message payloads for encoding service data into advertising message payloads, as described above.

At step304, user device102can receive service advertising data. In some implementations, advertising message encoder108can receive state advertising data from various services (e.g., operating system services, user applications, processes, daemons, etc.) on user device102. These applications can be configured to present media items (e.g., movies, music, videos, etc.) and/or present informative notifications to the user by generating audio, video, and/or haptic representations of data. For example, user device102can receive state advertising data from other devices indicating the activity level of the other devices. When user device102receives a notification, user device102can determine whether to present the notification based on a comparison of the activity level of user device102with the activity levels of the other devices. If user device102is not the most active device (e.g., the user is using another device), then user device102can suppress the notification so that the notification is not presented on user device102. If the user is using the display of user device102, the operating system of user device102can send display status data (e.g., a bit indicating that the display is lit) to advertising message encoder108.

At step306, user device102can determine a schema for encoding service advertising data into advertising message. For example, encoder108can select a schema based on the received state advertising data and/or which process sent the state advertising data. As described above, schema repository104can include multiple different schemas for various combinations of state advertising data. For example, the Schema “A” can be mapped to “services group A” associated with service1, service2and service3. Schema “B” can be mapped to “service group B” associated with service1and service3. To find the appropriate schema for the advertising processes or services, each schema in schema repository104can be mapped to a hash value that can be used as an index for finding or identifying the corresponding schema. In some implementations, the hash value can be generated based on service identifiers. For example, if schema “A” defines the state advertising message structure for advertising state information for service1(e.g., service identifier S1), service2(e.g., service identifier S2), and service3(e.g., service identifier S3), then the service identifiers S1, S2, and S3can be provided as inputs to a hashing algorithm to generate a hash value to be used as an index for looking up the corresponding Schema A. Thus, when state advertising message encoder108receives state information (e.g., including service identifiers) from service1, service2and service3(or any other combination of services), encoder108can generate a hash value based on the service identifiers in the received state information and use the generated hash value as an index to lookup or determine the appropriate schema (e.g., Schema A) for advertising the service state information in the schema mapping stored in schema repository104.

At step308, user device102can generate an advertising message. In some implementations, state advertising message encoder108can generate a state advertising message based on the determined schema. For example, encoder108can encode the schema identifier into the advertising message. For example, the advertising message can be a Bluetooth (e.g., Bluetooth Low Energy) advertising message that includes a header section and payload section as described above. In addition, encoder108can encode a schema identifier (e.g., “A”, or the corresponding hash value) corresponding to the schema associated with the advertising services into the advertising message (e.g., into the header section), as described above. Encoder108can determine the structure of the payload section of the advertising message based on the structure defined by the determined schema. For example, if the payload section includes 32 bits, the schema can define that the first ten bits are for service1data, the second ten bits are for service2data, and the last 12 bits are for service3data. Encoder108can then encode the service advertising data for the Service1, Service2, and Service3into the appropriate bits of the advertising message payload as defined by the determined schema.

At step310, device102can encode the schema identifier into the advertising message header. For example, the advertising message can be a Bluetooth (e.g., Bluetooth Low Energy) advertising message that includes a header section and payload section. In addition, encoder108can encode a schema identifier (e.g., “A”, or the corresponding hash value) corresponding to the schema associated with the advertising services into the advertising message (e.g., into the header section), as described above.

At step312, user device102can encode the service advertising data into the advertising message payload according to the determined schema. For example, the advertising message payload encoder108can encode the service data associate with the services into the advertising message payload data according to the determined schema (e.g., Schema “A”). After encoding the service state information into the advertising message according to the determined schema, user device102can broadcast the generated advertising message as described above. For example, user device102can broadcast the generated advertising message using a wireless transmitter employing a wireless (e.g., peer-to-peer) technology, such as Bluetooth, Bluetooth Low Energy, peer-to-peer Wi-Fi, or some other wireless technology. The message can be broadcasted without selecting a specific recipient device. For example, user devices (e.g., user device122) within range of the wireless signal transmitted by user device102can receive and/or process the advertising message generated by user device102.

FIG. 4is flow diagram of an example process400for decoding advertising data from an advertising message payload that was encoded using adaptable schema based payloads. For example, process400can be implemented by user device102and/or user device122.

At step402, user device122can store multiple schemas. In some implementations, the user device122can store multiple schemas (Schema ‘A’, ‘B’ . . . ‘NSCH’) corresponding to state information126a-126nfor Service1, Service2, . . . Service NSER. in the schema repository124. For example, state information106can correspond to state information for various services (e.g., applications, utilities, daemons, and/or operating system services) running on user device102.

At step404, user device122can receive an advertising message. For example, user device122can receive the advertising message broadcasted by user device102, as described above. For example, user device122can be configured similarly to user device102. Schema repository124can be configured similarly to schema repository104. For example, schema repository124can store multiple schemas (Schema ‘A’, ‘B’ . . . ‘NSCH’) corresponding to state information126a-126nfor Service1, Service2, . . . Service NSER. as described above with reference to schema repository104. Schema repository124can include a mapping of hash values to advertising message schemas, where the hash values are generated based on service identifiers corresponding to services to be advertised using advertising messages, as described above.

At step406, device122can obtain a schema identifier from the advertising message header. For example, when user device122receives the advertising message, advertising message decoder128can obtain the schema identifier (e.g., the hash value, or other identifier) from the advertising message header.

At step408, user device122can obtain an advertising message schema from schema repository124based on the schema identifier. For example, decoder128can determine the structure of the advertising message based on the structure defined by the obtained schema. For example, decoder128can determine that the first ten bits of the advertising message payload are for service1data, the second ten bits are for service2data, and the last 12 bits are for service3data.

At step410, user device122can extract the advertising message payload data according to the obtained schema. For example, the advertising message payload decoder128can extract service advertising data from advertising message payload according to the payload structure definition provided by the obtained schema. Continuing the above example, assuming that the schema defines the structure of the advertising message payload to have the first ten bits of the advertising message payload assigned to service1data, the second ten bits assigned to service2data, and the last 12 bits assigned to service3data, decoder128can then decode or extract the service advertising data for service1from the first ten bits to generate service1state data126a, decode or extract the service advertising data for Service2from the second 10 bits to generate service2state data126b, and decode or extract the service advertising data for Service3from the last 12 bits to generate service N state data126n. After extracting or decoding the data from the advertising message according to the schema, decoder128can send the advertised state information (126a,126b,126n) to the corresponding services (e.g., service1, service2, service3, etc.) on user device122.

Graphical User Interfaces

This disclosure above describes various Graphical User Interfaces (GUIs) for implementing various features, processes or workflows. These GUIs can be presented on a variety of electronic devices including but not limited to laptop computers, desktop computers, computer terminals, television systems, tablet computers, e-book readers and smart phones. One or more of these electronic devices can include a touch-sensitive surface. The touch-sensitive surface can process multiple simultaneous points of input, including processing data related to the pressure, degree or position of each point of input. Such processing can facilitate gestures with multiple fingers, including pinching and swiping.

When the disclosure refers to “select” or “selecting” user interface elements in a GUI, these terms are understood to include clicking or “hovering” with a mouse or other input device over a user interface element, or touching, tapping or gesturing with one or more fingers or stylus on a user interface element. User interface elements can be virtual buttons, menus, selectors, switches, sliders, scrubbers, knobs, thumbnails, links, icons, radio buttons, checkboxes and any other mechanism for receiving input from, or providing feedback to a user.

Privacy

The present disclosure recognizes that the use of such personal information data, in the present technology, can be used to the benefit of users. For example, the personal information data can be used to deliver targeted content that is of greater interest to the user. Accordingly, use of such personal information data enables calculated control of the delivered content. Further, other uses for personal information data that benefit the user are also contemplated by the present disclosure.

Despite the foregoing, the present disclosure also contemplates embodiments in which users selectively block the use of, or access to, personal information data. That is, the present disclosure contemplates that hardware and/or software elements can be provided to prevent or block access to such personal information data. For example, in the case of advertisement delivery services, the present technology can be configured to allow users to select to “opt in” or “opt out” of participation in the collection of personal information data during registration for services. In another example, users can select not to provide location information for targeted content delivery services. In yet another example, users can select to not provide precise location information, but permit the transfer of location zone information.

Example System Architecture

FIG. 5is a block diagram of an example computing device500that can implement the features and processes ofFIGS. 1-4. The computing device500can include a memory interface502, one or more data processors, image processors and/or central processing units504, and a peripherals interface506. The memory interface502, the one or more processors504and/or the peripherals interface506can be separate components or can be integrated in one or more integrated circuits. The various components in the computing device500can be coupled by one or more communication buses or signal lines.

Sensors, devices, and subsystems can be coupled to the peripherals interface506to facilitate multiple functionalities. For example, a motion sensor510, a light sensor512, and a proximity sensor514can be coupled to the peripherals interface506to facilitate orientation, lighting, and proximity functions. Other sensors516can also be connected to the peripherals interface506, such as a global navigation satellite system (GNSS) (e.g., GPS receiver), a temperature sensor, a biometric sensor, magnetometer or other sensing device, to facilitate related functionalities.

A camera subsystem520and an optical sensor522, e.g., a charged coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) optical sensor, can be utilized to facilitate camera functions, such as recording photographs and video clips. The camera subsystem520and the optical sensor522can be used to collect images of a user to be used during authentication of a user, e.g., by performing facial recognition analysis.

Communication functions can be facilitated through one or more wireless communication subsystems524, which can include radio frequency receivers and transmitters and/or optical (e.g., infrared) receivers and transmitters. The specific design and implementation of the communication subsystem524can depend on the communication network(s) over which the computing device500is intended to operate. For example, the computing device500can include communication subsystems524designed to operate over a GSM network, a GPRS network, an EDGE network, a Wi-Fi or WiMax network, and a Bluetooth™ network. In particular, the wireless communication subsystems524can include hosting protocols such that the device100can be configured as a base station for other wireless devices.

An audio subsystem526can be coupled to a speaker528and a microphone530to facilitate voice-enabled functions, such as speaker recognition, voice replication, digital recording, and telephony functions. The audio subsystem526can be configured to facilitate processing voice commands, voiceprinting and voice authentication, for example.

The I/O subsystem540can include a touch-surface controller542and/or other input controller(s)544. The touch-surface controller542can be coupled to a touch surface546. The touch surface546and touch-surface controller542can, for example, detect contact and movement or break thereof using any of a plurality of touch sensitivity technologies, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch surface546.

The other input controller(s)544can be coupled to other input/control devices548, such as one or more buttons, rocker switches, thumb-wheel, infrared port, USB port, and/or a pointer device such as a stylus. The one or more buttons (not shown) can include an up/down button for volume control of the speaker528and/or the microphone530.

In one implementation, a pressing of the button for a first duration can disengage a lock of the touch surface546; and a pressing of the button for a second duration that is longer than the first duration can turn power to the computing device500on or off. Pressing the button for a third duration can activate a voice control, or voice command, module that enables the user to speak commands into the microphone530to cause the device to execute the spoken command. The user can customize a functionality of one or more of the buttons. The touch surface546can, for example, also be used to implement virtual or soft buttons and/or a keyboard.

In some implementations, the computing device500can present recorded audio and/or video files, such as MP3, AAC, and MPEG files. In some implementations, the computing device500can include the functionality of an MP3 player, such as an iPod™. The computing device500can, therefore, include a 36-pin connector that is compatible with the iPod. Other input/output and control devices can also be used.

The memory interface502can be coupled to memory550. The memory550can include high-speed random access memory and/or non-volatile memory, such as one or more magnetic disk storage devices, one or more optical storage devices, and/or flash memory (e.g., NAND, NOR). The memory550can store an operating system552, such as Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VxWorks.

The operating system552can include instructions for handling basic system services and for performing hardware dependent tasks. In some implementations, the operating system552can be a kernel (e.g., UNIX kernel). In some implementations, the operating system552can include instructions for performing voice authentication. For example, operating system552can implement the adaptable schema based payloads features as described with reference toFIGS. 1-4.

The memory550can also store communication instructions554to facilitate communicating with one or more additional devices, one or more computers and/or one or more servers. The memory550can include graphical user interface instructions556to facilitate graphic user interface processing; sensor processing instructions558to facilitate sensor-related processing and functions; phone instructions560to facilitate phone-related processes and functions; electronic messaging instructions562to facilitate electronic-messaging related processes and functions; web browsing instructions564to facilitate web browsing-related processes and functions; media processing instructions566to facilitate media processing-related processes and functions; GNSS/Navigation instructions568to facilitate GNSS and navigation-related processes and instructions; and/or camera instructions570to facilitate camera-related processes and functions.

The memory550can store other software instructions572to facilitate other processes and functions, such as the adaptable schema based payloads processes and functions as described with reference toFIGS. 1-4.

The memory550can also store other software instructions574, such as web video instructions to facilitate web video-related processes and functions; and/or web shopping instructions to facilitate web shopping-related processes and functions. In some implementations, the media processing instructions566are divided into audio processing instructions and video processing instructions to facilitate audio processing-related processes and functions and video processing-related processes and functions, respectively.