API Rules Verification Platform

A stateful rules verification platform is described. The verification platform implements a specification language to provide a formal definition for rules used to test target systems having a central module that provides APIs (“API provider”) and applications (“API clients”) that use the APIs. Rules may be defined in terms of transitions on state elements associated with interactions between API providers and API clients. The rules defined in accordance with the specification language enable run-time verification in which calls may be intercepted and run-time code to implement checks may automatically be generated and injected to test behaviors of the intercepted calls. The same set of rules may also be employed for static verification during compilation. Additionally, the specification language includes constructs to provide rule descriptions and comments with rules definitions that facilitate publication of the rules and documentation of misbehavior identified during verification.

DETAILED DESCRIPTION

Overview

Traditionally, verification rules are manually coded into a verification system and distinct set of rules may be authored by different people using different styles and/or ad hoc methods for different types of verifications (e.g., static vs. run-time), which results in inflexible and inconsistent rules that may be difficult to understand and maintain. Accordingly, traditional verification techniques may be inefficient and/or inadequate for some types of software systems and corresponding testing scenarios.

A verification platform that implements stateful rules for verification of code projects is described herein. The verification platform may be designed to facilitate verification of target systems composed of a central module that provides application programming interfaces (APIs), e.g., an “API provider”, for use by other application modules, e.g., “API clients,” to take advantage of functionality provided by the central module. For example, the techniques described herein may be applied to device drivers, system drivers, and services that interact with system APIs provided by an operating system. In one or more implementations, the verification platform implements a specification language to provide a formal definition for rules. Using the specification language, rules may be defined in terms of transitions on state elements associated with interactions between API providers and API clients. The rules defined in accordance with the specification language enable run-time verification in which calls may be intercepted and run-time code to implement checks may automatically be generated and injected to test behaviors of the intercepted calls. The same set of rules may also be employed for static verification during compilation. Additionally, the specification language includes constructs to provide rule descriptions and comments with rules definitions that facilitate publication of the rules and documentation of misbehavior identified during verification.

In the following discussion, an example operating environment is first described that may employ the techniques described herein. Next, example details and techniques are described which may be implemented in the example environment as well as other environments. Consequently, performance of the techniques is not limited to the example environment and the example environment is not limited to performance of the example techniques. Lastly, example systems and devices are described that may be employed to implement one or more embodiments.

Example Operating Environment

FIG. 1is an illustration of an environment100in an example implementation that is operable to employ techniques described herein. The illustrated environment100includes a client device102that is communicatively coupled via a network104to a service provider106. The service provider106may be configured to make various resources108(e.g. content and services) available over the network104to the client device102and other clients. Generally, resources108made accessible by a service provider106may include any suitable combination of services and/or content typically made available over a network by one or more providers. Some examples of services include, but are not limited to, a search service, an email service, an instant messaging service, an online productivity suite, and an authentication service to control access of clients to the resources. Services may also include a rules service109as illustrated and/or database used to publish information regarding rules for use by various clients and enable access to rules information over the network104that clients may utilize to document and/or understand behaviors/bugs detected in relation to verifications, as discussed in detail below. Content may include various combinations of text, multi-media streams, documents, application files, photos, audio/video files animations, images, web pages, web applications, device applications, content for display by a browser or other client application, and the like.

The client device102and service provider106may be implemented by one or more computing devices and also may be representative of one or more entities. A computing device may be configured in a variety of ways. For example, a computing device may be configured as a computer that is capable of communicating over the network, such as a desktop computer, a mobile station, an entertainment appliance, a set-top box communicatively coupled to a display device, a wireless phone, a game console, and so forth. Thus, the computing device may range from full resource devices with substantial memory and processor resources (e.g., personal computers, game consoles) to a low-resource device with limited memory and/or processing resources (e.g., traditional set-top boxes, hand-held game consoles). Additionally, although a single computing device is shown in some instances, the computing device may be representative of a plurality of different devices, such as multiple servers utilized by the service provider106.

The client device102is further illustrated as including an operating system110. The operating system110is configured to abstract underlying functionality of underlying hardware to applications112that are executable on the client device102. For example, the operating system110may abstract processing, memory, network, and/or display functionality such that the applications112may be written without knowing “how” this underlying functionality is implemented. The applications112, for instance, may provide data to the operating system110to be rendered and displayed by a display device without understanding how this rendering will be performed. Interaction of applications112with the operating system110may occur by way of one or more application programming interfaces (APIs) associated with the operating system110. In this regard, the operating system110is one example of an API provider as discussed herein.

In accordance with techniques described herein, the client device102is also illustrated as including API clients114and a verification platform116that represents functionality operable to test performance of the API clients114against a set of verification rules. The verification rules may include stateful rules defined in accordance with a suitable specification language as described above and below. In some implementations, the verification platform116may be provided as an operating system component, however, the verification platform116may also be implemented as a standalone component as illustrated.

The verification platform116facilitates verification of target systems composed of a central module that provides application programming interfaces (APIs), e.g., an “API provider”, for use by other application modules, e.g., “API clients,” to take advantage of functionality provided by the central module. In one approach, the techniques described herein may be applied to device drivers, system drivers, and services that interact with system APIs provided by an operating system. Other kinds of target systems are also contemplated. Generally, the verification platform116may be configured in various ways to at least: (1) implement and/or otherwise make use of a suitable specification language to define rules, (2) provide rules defined in accordance with the specification language, (3) implement run-time verification using the rules, (4) enable static verification using the rules, and (5) facilitate publication and documentation of rules via a rules service or otherwise. Details regarding these and other aspects of a verification platform116suitable to implement stateful rules are described in relation in relation to the following figures.

FIG. 2depicts an example representation of a target system200that may be the subject of verification by the verification platform116. Here, API clients114are illustrated as interacting with an API provider202that includes various APIs204. Generally, the APIs204provide abstractions of underlying system resources206and hardware208that are exposed to the API clients114. In the example of an operating system110, APIs204may be provided by the OS kernel to enable various API clients114to “plug-in” to core functionality of the OS (e.g., memory, CPU time, interrupts, device registers, communication protocols, system services, tools, functions, etc.) and implement extensions to the OS supported by the APIs. Other API providers202that provide corresponding APIs204for access to different types of functionality are also contemplated such as, by way of example and not limitation, a printer subsystem, a graphics rendering system, a wireless communication system, a sensor control system, and/or micro-processing subsystem that operates independently of the OS, to name a few examples. In this context, the API clients114are representative of various kinds of application modules, plugins, drivers (e.g., device drivers, printer drivers, graphics drivers, software drivers, class drivers, system drivers, etc.) and/or other third-party programs that are designed to interact with the OS or another central module through APIs204.

Having considered the foregoing discussion of an example operating environment, consider now details regarding a verification platform described in relation to the following example illustrations and procedures.

Verification Platform Details

This section discusses details of an example verification platform in accordance with one or more implementations. In portions of the following discussion reference may be made to the example operating environment described in relation toFIGS. 1 and 2.

In particular,FIG. 3depicts generally at300an illustration of an example verification platform116that may be implemented by a suitably configured client device102. As mentioned, the verification platform116may be configured in various ways to at least: (1) implement and/or otherwise make use of a suitable specification language to define rules, (2) provide rules defined in accordance with the specification language, (3) implement run-time verification using the rules, (4) enable static verification using the rules, and (5) facilitate publication and documentation of rules via a rules service or otherwise. In accordance with techniques described above and below, the verification platform116is depicted inFIG. 3as including or making use of a rules specification language302to define rules, stateful rules304that may defined in accordance with the rules specification language302, a run-time verifier module306that may include an interceptor module308and a checker module310to perform run-time verifications using the stateful rules304, a static verifier module312to implement static verification based on the stateful rules304, and a documenter module314operable to publish information regarding rules and/or obtain information from a rules service109to document behaviors observed during verifications. Details regarding configuration and operation of each of these components of the verification platform116are discussed in turn below.

Rules Specification Language

The rules specification language302or “SL” is implemented to define rules for verification in a consistent and formal manner. In one particular example, the rules specification language302comprises SLIC (Specification Language for Interface Checking), although various language having characteristics described herein are also contemplated. In general, a suitable SL is configured to express rules in relation to interactions of API clients with API providers in various ways. In one approach, the SL may be employed to define rules in terms of state transitions for elements of interest in the target system. In other words, the rules may be expressed as transitions that occur for state machines associated with API interactions. In this respect, the SL may encode expected behaviors for API clients with API providers in relation to various API interactions as rules that can be checked for compliance and/or violation during verification. Various state transitions for elements may occur in response to different events. The SL may be employed to define rules to confirm that a client uses an API correctly and/or rules to determine that an API behaves correctly in response to a call from a client.

In addition to tracking of state transitions, the SL provides constructs (e.g., designated fields, protocols, descriptors, tags, identifiers, data structures, strings, coding rules, etc.) to handle objects, parameters, and values that may inform particular rules checks. This may include function or procedure arguments used to verify particular conditions, such as filename to check that a particular file is the subject of an interaction, a token indicative of authorizations, an identifier of an application or subroutine, a parameter indicative of a device or file status, and other arguments. This may further include life-cycle objects that persist along multiple API calls from a client into a provider or multiple entries (calls) into a client from a provider. Some examples of life-cycle objects include but are not limited to device objects (data structures to represent devices), miniport objects (data structures to represent miniport drivers), NDIS objects and so forth. The SL enables rules to reference and make use of such arguments and/or life-cycle objects.

In operation, the verification module116may manage the arguments and/or life-cycle objects in a cache or designated memory location. Objects to reference and manage various items may be created as needed and may be destroyed or otherwise discarded after use. The platform also supports requests to track state data flow/transitions on various objects for the purpose of verifying rule compliance. The SL may also be employed to specify responsive actions to be taken upon detected misbehaviors, such as suspending the run, triggering a notification, accessing a rules service for documentation, writing data to a bug file and so forth. Additionally, the SL may specify reset points that indicate when objects are to be reset individually and/or circumstances in which to reset the verification process/run as a whole.

Further, the SL also provides constructs to enable rules documentation. This may include comment fields and descriptor codes used to provide descriptions and comments regarding rules as part of the rule definition. These comment fields and descriptor codes may be extracted and used to facilitate publishing of rules in a manner that enables developers to understand the rules in plain terms. Thus, the SL implements a formalized description and comment structure for rules. Moreover, rules that are published (such as via a rules service109or otherwise) may be accessible to document behaviors observed during verifications. For instance, a description and/or comments associated with a given rule may be extracted from the rule and added to a notification that is provided when a violation of the rule is detected. In another approach, links to information regarding rules maintained by the rules service109may be included with notifications, messages, or entries formed in response to misbehavior detection with respect to the rules. Details regarding stateful rules and some illustrative examples are discussed in the following section.

Stateful Rules

The stateful rules304are defined in accordance with a rules specification language302as just described. The rules may reflect formal partial specifications of distinct meaningful combinations (subsets) of APIs exposed by an API provider to API clients. The rules are configured to provide unambiguous definitions of the interface contract between a provider and its clients. The rules definitions may be partial, in a sense that they define some selected aspects of API usage that are considered appropriate for verification and may forego other aspects (taking into account availability of computational resources and other practical constraints of real life, such as time-to-market, cost, etc.).

As mentioned, the stateful rules304may be defined in terms of state transitions for elements of interest in the target system. In other words, the rules may be expressed as transitions that occur for state machines associated with API interactions. In this respect, rules may be considered “stateful” as the rules are based upon checking that transitions of the state machines to different states occur as expected (e.g., in accordance with the rule definition).

To further illustrate, consider now a few examples of stateful rules. In one example, a rule may be defined for behaviors related to acquiring and releasing spinlocks by a driver. In particular, a rule may define that a thread holding a spinlock is prevented from acquiring the same spinlock again. Another rule may indicate that a driver is restricted from holding any spinlock when calling into another driver. Still further, another rule may indicate that a driver is restricted from holding any spinlock when returning from a driver callback function. Additionally, rules may also indicate that a driver has to acquire the lock before it may be released. Similar rules may be defined to monitor states in relation to memory access, operation of a miniport driver for connection/disconnections, device interactions, and so forth. One illustrative example of code for a rule definition written in accordance with a suitable specification language (e.g., SLIC in this example) is as follows:

The foregoing example rule relates to spinlock and includes information such as the rule name, namespace, and a rule identifier. A description field portion is also included that provides comments/description to indicate that the rule enforces certain state conditions for calls to corresponding APIs for acquisition and release of the spinlock. The rule definition also includes a help link that as discussed herein may be used to facilitate publishing and documentation of the rule. The example rule specifies values and transitions for a state machine represented by the state variable “s”. A variety of other rules related to drivers as well as other kinds of API clients are also contemplated.

The run-time verifier module306represents functionality operable to perform run-time verifications based on stateful rules304. This may involve operations to intercept calls made to designated APIs and automatic generation of run-time code to implement verification checks for rules implicated by the intercepted calls. In an implementation, the run-time verifier module306includes an interceptor module308and checker module310. The interceptor module308is operable to intercept both calls to APIs issued by clients and calls to a client's procedures (callbacks, entry points, etc.) from a provider and initiate creation of corresponding verification checks based on the interceptions. For instance, the interceptor module308may invoke the checker module310with the intercepted calls to cause the interceptor module to set-up and apply various checks corresponding to the calls.

In particular, the checker module310may analyze the intercepted calls to identify any stateful rules304that apply to the calls. The checker module310may retrieve definitions for the applicable rules that are written using the rules specification language302. The checker module310may use the rules definition to automatically generate corresponding verification checks to monitor expected behavior and determine compliance with the rules.

In particular, implementation code to perform the verification checks may be generated automatically from definitions of the stateful rules304that are retrieved by the checker module310. The implementation code may be also be optimized through re-assembling state transitions in API rules, combining together transitions (from different rules) which are triggered by same events, and translating these combined transitions into an actual programming language (e.g., C, C++, etc.) for run-time implementation of the checks. Run-time code to implement verification checks may be produced dynamically in response to interception of calls or “offline” to produce pre-configured verification.

At run-time, the checker module310implements the checks derived from stateful rules304to verify that the system behaves as the rules prescribe. The run-time verification may be selectively turned on or off for a particular computing system or device. In some cases, a developer may enable the run-time verification for a code project on a test machine and/or in conjunction with code testing and/or debugging. Run-time verification may also be provided as an option to consumers that may be enabled on-demand for trouble shooting of APIs and clients related to an operating system110or other central module. When run-time verification is enabled for a regular run or a test run of a target system, the run-time verifier module306checks the entire system against a set of rules associated with the system. If misbehavior or non-compliance with one or more of the rules is detected the run-time verifier module306may take various responsive actions. Such responsive actions may be designated within the rules using the rules specification language302as noted above and may be encoded within the implementing code generated by the checker module310. By way of example and not limitation, responsive actions that may be specified include breaking the testing run, issuing an error message, initiating documentation of misbehavior, writing an entry describing the misbehavior to a results file or database, bypassing an API associated with the misbehavior, activating a recovery process, and so forth.

In one approach, tracked states are bound to OS, kernel, framework, or other target system objects (e.g., a thread object, a miniport object, etc.). This makes the implementation of state transitions efficient, by narrowing the scope of locks used to guard state manipulations. The tracked state objects may be manipulated on an individual basis rather than by employing global databases.

In the case of tracked states bound to thread objects, the run-time verifier module306may be configured to handle thread preemption and resolve situations in which multiple instances of executing components may rely upon the same thread object. With respect to verification using stateful rules, the run-time verifier module306may implement an algorithm to ensure that a state actually belongs to execution of a subject driver/API client and not to another component using the same thread object. To do so, the run-time verifier module306may detect and ignore call interceptions for some high priority components running on a given thread object in certain cases considered either impossible or infeasible to deal with. For example, interceptions that run at a high interrupt request level may be ignored. In addition, interceptions for a thread may be ignored when the thread is currently used to run a Deferred Procedure Call (DPC). Additionally, the run-time verifier module306may detect when multiple instances of executing components of normal priority are bound to the same thread object and apply a stack-based technique to extract the state that belongs to a component that most recently preempted the thread. In this way, the run-time verifier module306is able to handle situations in which multiple instances of executing components may rely upon the same thread object.

Static Verifier

The static verifier module312represents functionality operable to perform static verifications based on stateful rules304. In particular, the static verifier module312is a compile-time tool for inter-procedural static analysis of an API client's source code. Given source code of a client, the static verifier module312may operate to verify that each execution path from the client's entry point through the client's exit obeys the stateful rules304. Static verification may be employed in some scenarios to provide exhaustive verification of each execution path. Additionally, static analysis may occur without significant set-up costs or creation of test benches/test cases.

Notably, the static verifier module312may be configured to employ the same rules as the run-time verifier306in the same format and written using the same rules specification language302. This is in contrast to traditional models that employ entirely separate rules, systems, and techniques for run-time verification and static verification, and which therefore can be inefficient, resource intensive, and difficult for developers to utilize and maintain. Here, the static verifier module312may re-use stateful rules304created for run-time verification without modification. In an implementation, the static verifier module312may include a checker component comparable to the checker module310that can interpret an API client's source code to determine rules implicated by various calls. The checker component may then derive implementing checks for static analysis from the rules definitions and apply the checks during static verification runs. Accordingly, the verification platform116is designed to facilitate both run-time verification and static verification using a common rules specification language302and/or the same set of stateful rules304.

The documenter module314is representative of functionality to facilitate publishing and/or documentation of stateful rules304. Publishing and documentation of rules may occur in various ways. As mentioned, stateful rules304may be published via a rules service109of a service provider106or otherwise. Publication of the stateful rules304enables API client developers to understand how to use APIs correct (e.g., in accordance with the rules) and therefore to develop code that respects the interface contract with the API provider. In a rules creation setting, the documenter module314or comparable functionality may be operable to interact with a rule service109to publish rules descriptions and/or definitions for access by code developers and/or consumers. The formal specification of rules via the rules specification language302enables the developers and/or consumers to access information regarding rules in a consistent and understandable format. Developers may also be able to produce new rules to extend the verification platform116with new checks.

As mentioned, formalized description and comments may be included as part of the rules definition. This information may be extracted and used to publish the rules with a service or otherwise. In a static or run-time verification setting, the published rules information and/or links to the rules may also be employed to document observed behaviors/misbehaviors. For instance, rules definitions may contain data to facilitate linking to relevant comments and documentation published via a rules service109. The run-time verifier module306and the static verifier module312may utilize the links to comments and documentation to incorporate or include references to relevant comments and documentation for rules in responses (e.g., warnings, messages, notifications, entries, etc.) generated when misbehavior is detected during verification with respect to one or more rules.

Example Procedures

This section discusses details of techniques for a stateful rules verification platform with reference to example procedures ofFIGS. 4 and 5. In portions of the following discussion reference may be made to the example operating environment ofFIG. 1in which various aspects may be implemented. Aspects of each of the procedures described below may be implemented in hardware, firmware, or software, or a combination thereof. The procedures are shown as a set of blocks that specify operations performed by one or more devices and are not necessarily limited to the orders shown for performing the operations by the respective blocks. In at least some implementation the procedures may be performed by a suitably configured computing device, such as the example client device102ofFIG. 1that includes or makes use of a verification platform116or comparable functionality.

FIG. 4is a flow diagram depicting an example procedure400in which run-time verification is performed. Rules defined for API verification of a target system in accordance with a rules specification language are obtained (block402). For example, a verification platform116as described herein may be encoded with stateful rules304defined in accordance with a rules specification language302. The verification platform116may be pre-configured to include various rules. The verification platform116may also be configured to interact with a rules service109to obtain rules associated with different types of clients and providers and/or updates to rules that may occur from time to time. In connection with run-time verification, rules associated with a particular target system may be obtained and supplied for use by a run-time verifier module306configured to use the rules to check behaviors of API clients and API providers.

Run-time verification is enabled for verification of the target system via the verification platform (block404). For example, the verification platform116may include a run-time verifier module306that is configured and operates in the manner previously described. The run-time verifier module306may be selectively enabled on-demand to perform run-time verifications. In other words, a developer, consumer, or other user may toggle functionality represented by the run-time verifier module306to selectively perform verifications for code testing, troubleshooting, and so forth.

Calls made by components of the system are intercepted during a testing run (block406). For instance, an interceptor module308as described above or comparable functionality may operate to intercept calls made into APIs as well as callbacks made by an API provider to clients. Then, verification checks are automatically generated for the intercepted calls based on the defined rules (block408) and the verification checks are applied to determine adherence of the components of the target system to the defined rules (block410). Here, the an interceptor module308, in response to intercepted calls may invoke a checker module310as described above or comparable functionality to process the intercepted calls and create corresponding verification checks. This may occur in the manner discussed in relation toFIG. 3. To briefly reiterate, though, the checker module310may determine stateful rules304that are implicated by or otherwise associated with the intercepted calls and derive checks based on the applicable rules. The checks may be derived directly from the rules definitions. This may involve generating and injecting implementing run-time code for performance of the checks into the target system based on rules definitions.

In addition or alternatively, verification checks may be generated in advance by a standalone generation module or by comparable functionality incorporated with the checker module310and/or verification platform116. This may involve deriving the verification checks from the rules as discussed herein outside of verification runs. In other words, verification checks may be pre-configured “offline” prior to a verification run and encoded within the system for subsequent use by the run-time verifier module306and/or static verifier module312. For instance, the pre-configured verification checks may be encoded within a checker module, stored in a location accessible by a checker module, or otherwise be made available for use by the verification platform during testing. When a verification run occurs, the checker module may intercept various calls and perform a look-up to match the calls to corresponding verification checks that are pre-configured. Thus, verification checks that match the actual calls that are intercepted are identified and applied to verify adherence of the target system to corresponding rules (e.g., rules from which the pre-configured verification checks are derived).

The checker module310further operates to apply the checks to test for misbehaviors and bugs. In the case that one or more rules are broken, the checker module310may be configured to take responsive action designated by the rules, some examples of which were previously discussed. In the event that no misbehaviors or bugs are detected, the checker module310may continue operation until disabled, timed-out, or the verification is otherwise concluded. When verification is concluded, a report, message, or other notification may be output that may include an indication that no misbehaviors or bugs were detected, a summary of the testing, statistics, and/or other information regarding the verification.

FIG. 5is a flow diagram depicting an example procedure500in which a verification platform operates to employ stateful rules for code verifications. The procedure represents operations that may be implemented by a suitably configured verification platform, such as the example verification platform116and corresponding components depicted and described in relation toFIGS. 1 and 3. The verification platform116may be stored on some form of computer-readable media as described herein and/or may be implemented at least partially by hardware, such as via a processing system and/or hardware elements of a computing device.

A rules specification language is implemented to define rules for API verification in terms of transitions for state machines associated with events (block502). For example, a verification platform116as described herein may include or otherwise make use of a rules specification language302to provide a formal specification of rules for verifications. One or more rules are created for verification of target systems in accordance with the specification language (block504). In particular, the rules specification language302may be used to create stateful rules304and associate the rules with a target system. The target system may include an API provider202that exposes APIs204to API clients114. In one particular example, the target system comprises an operating system110that makes various system APIs available to drivers to extend operating system functionality. In one approach, a set of rules may be created for a particular target system. Thus, different sets of rules may be associated with different target system and/or with particular types of APIs and API clients. In some scenarios, though, particular rules may be applied across different systems, re-used, and/or re-purposed to define similar expected behaviors for different calls or APIs.

Run-time verifications are performed using the one or more rules (block506) and static verifications are performed using the one or more rules (block508). The run-time verifications and static verifications may be implemented by a run-time verifier module306and static verifier module312, respectively, in the various ways already discussed herein. Stateful rules304created for the verification platform116in relation to a particular target system may be used for both run-time verifications and static verifications of the target system. Accordingly, development and maintenance of separate rules and/or platforms for run-time verifications and static verifications may be avoided. Stateful rules304described herein may be employed for both types of verifications without having to make modifications to the rules or the specification language used to describe the rules.

The one or more rules and results of verifications based on the one or more rules are documented (block510). For example, a documenter module314or other comparable component of the verification platform may be provided to facilitate publishing or rules and/or descriptions of the rules in the manner described herein. In one approach, a service provider106may expose a rules service109to manage rules, provide a location to publish rules, enable access to the rules, and so forth. The documenter module314may also be operable to access and make use of information regarding the rules to document the results of verification runs. This may involve acquiring and adding descriptions and/or comments associated with rules to reports, warning, notifications, messages, and/or other instruments configured to provide information regarding the results of verification runs. For instance, when misbehavior is detected with respect to a particular rule a warning message may be generated and output to alert a user regarding the misbehavior. The documenter module314may operate to retrieve descriptions and/or comments associated with the particular rule and cause the warning message to incorporate the descriptions and/or comments for the particular rule to enable a developer/user to more fully appreciate the misbehavior.

Having considered the foregoing example details and procedures, consider now a discussion of an example system and device to implement various aspects in accordance with one or more embodiments.

Example System and Device

FIG. 6illustrates an example system600that includes an example computing device602that is representative of one or more computing systems and/or devices that may implement the various techniques described herein. The computing device602may be, for example, a server of a service provider, a device associated with a client (e.g., a client device), an on-chip system, and/or any other suitable computing device or computing system.

The processing system604is representative of functionality to perform one or more operations using hardware. Accordingly, the processing system604is illustrated as including hardware elements610that may be configured as processors, functional blocks, and so forth. This may include implementation in hardware as an application specific integrated circuit or other logic device formed using one or more semiconductors. The hardware elements610are not limited by the materials from which they are formed or the processing mechanisms employed therein. For example, processors may be comprised of semiconductor(s) and/or transistors (e.g., electronic integrated circuits (ICs)). In such a context, processor-executable instructions may be electronically-executable instructions.

Combinations of the foregoing may also be employed to implement various techniques and modules described herein. Accordingly, software, hardware, or program modules including operating system110, applications112, verification platform116, and other program modules may be implemented as one or more instructions and/or logic embodied on some form of computer-readable storage media and/or by one or more hardware elements610. The computing device602may be configured to implement particular instructions and/or functions corresponding to the software and/or hardware modules. Accordingly, implementation of modules as a module that is executable by the computing device602as software may be achieved at least partially in hardware, e.g., through use of computer-readable media and/or hardware elements610of the processing system. The instructions and/or functions may be executable/operable by one or more articles of manufacture (for example, one or more computing devices602and/or processing systems604) to implement techniques, modules, and examples described herein.

In various implementations, the computing device602may assume a variety of different configurations, such as for computer614, mobile616, and television618uses. Each of these configurations includes devices that may have generally different constructs and capabilities, and thus the computing device602may be configured according to one or more of the different device classes. For instance, the computing device602may be implemented as the computer614class of a device that includes a personal computer, desktop computer, a multi-screen computer, laptop computer, netbook, and so on.

The computing device602may also be implemented as the mobile616class of device that includes mobile devices, such as a mobile phone, portable music player, portable gaming device, a tablet computer, a multi-screen computer, and so on. The computing device602may also be implemented as the television618class of device that includes devices having or connected to generally larger screens in casual viewing environments. These devices include televisions, set-top boxes, gaming consoles, and so on.

The techniques described herein may be supported by these various configurations of the computing device602and are not limited to the specific examples of the techniques described herein. This is illustrated through inclusion of the verification platform116on the computing device602. The functionality of the verification platform116and other modules may also be implemented all or in part through use of a distributed system, such as over a “cloud”620via a platform622as described below.

The cloud620includes and/or is representative of a platform622for resources624. The platform622abstracts underlying functionality of hardware (e.g., servers) and software resources of the cloud620. The resources624may include applications and/or data that can be utilized while computer processing is executed on servers that are remote from the computing device602. Resources624can also include services provided over the Internet and/or through a subscriber network, such as a cellular or Wi-Fi network.

The platform622may abstract resources and functions to connect the computing device602with other computing devices. The platform622may also serve to abstract scaling of resources to provide a corresponding level of scale to encountered demand for the resources624that are implemented via the platform622. Accordingly, in an interconnected device embodiment, implementation of functionality described herein may be distributed throughout the system600. For example, the functionality may be implemented in part on the computing device602as well as via the platform622that abstracts the functionality of the cloud620.

CONCLUSION