Client-side policy enforcement of developer API use

Methods and systems are disclosed that allow automated pre-qualification and qualification of an application. An application description can be generated for an application submitted by a developer, the application description can be automatically examined to determine whether the application complies with rules or guidelines (e.g., policies) of a platform. If the application complies with the rules or guidelines, the application can be pre-qualified and submitted for approval and distribution. If the application does not comply with the rules or guidelines, the application developer can be notified of the errors in the application and the developer can be prevented from uploading the application for approval and distribution.

TECHNICAL FIELD

This subject matter is generally related to software development.

BACKGROUND

A software development process can include a structure for creating and maintaining a software product. A software development process can include multiple stages. Some example software development stages can include design, implementation, testing, and distribution. Some models of software development processes in existence today include waterfall model, spiral model, agile software development, extreme programming (XP), among others.

Modern software development processes for various data processing systems allow for participation of a vast number of diverse developers for a platform (e.g., a mobile device development platform). Tools for developing software for the platform can include a publicly available software development kit (SDK) and various rules and guidelines. The SDK can include various libraries and an integrated development environment (IDE). Using the SDK, a developer can develop an application program for the platform. The application program can be distributed to data processing systems that are compatible with the platform, for example, through an application store.

SUMMARY

Methods and systems are disclosed that allow for client-side policy enforcement of developer application programming interface (API) use. In some implementations, a developer can generate an application description for an application and submit the application description to a server for pre-qualification (e.g., API, configuration and/or resource validation). If the server determines that the application can be pre-qualified based on the application description, the client can upload the application for approval and distribution. If the server determines that the application cannot be pre-qualified based on application description, the server can send the client a message indicating errors in the pre-qualification process and prevent the client from uploading the application for approval and distribution.

In some implementations, a server can receive an application description for an application, or portion of an application, from a client device and pre-qualify the application based on the application description. Pre-qualification can include validating developer API usage based on the application description. The server can determine which APIs are described in the application description, compare the APIs to API usage policies (e.g., blacklists, whitelists, developer licenses, etc.), generate warnings and errors based on the comparison. Pre-qualification can include validating application metadata (e.g., configuration and/or resources). The server can compare the metadata to policies that describe allowed and disallowed configurations and resources and generate warnings and errors based on the comparison. The developer can be prevented from uploading the application for approval and distribution when errors are generated. The developer can be allowed to upload the application for approval and distribution when no errors are generated.

Particular embodiments of the subject matter described in this specification can be implemented to realize one or more of the following advantages. Using the automated pre-qualification process, a developer can quickly determine whether the developer's application complies with application approval policies without having to wait for a full application qualification and approval process to be completed. Pre-qualification can be based on the application description alone, without requiring an inspection of the source code of the application program. Thus, a developer need not divulge the source code. If the application description describes system libraries that are beyond the scope of a publicly available SDK, the system can notify the application developer by presenting error and warning messages. Likewise, if the application description includes disallowed configuration data or resources, the application developer can be notified. The system implementing the automated pre-qualification process can detect possible use of deprecated libraries, classes, or functions, and notify the developer that the application program, although working now, may break in a future release of the platform. The developer can thus take cautionary measures (e.g., by using another functionally similar library, class, or function) to make the application program more robust before submitting the full application for final approval.

DETAILED DESCRIPTION

Example Software Development Process

FIG. 1is a flowchart illustrating an example software qualification process100, including pre-qualification. In example process100, a developer acquires (e.g., by downloading) an SDK of a platform (e.g., a mobile device development platform), and develops an application program. The application program is to be distributed (e.g., provided for download) to other users of mobile devices compatible with the platform.

The developer can develop (102) an application. In this specification, any person who engages in any part of developing the application can be a developer. Developing the application can include, for example, gathering requirements, designing the application, writing source code of the application, compiling the source code into binary code, and linking the binary code into executable code.

The application being developed in process100can include any computer instructions that are configured to perform user tasks (e.g., tasks that produce results for a user) or system tasks (e.g., tasks that manage computing resources of a computer) or both. The application can be an application based on a specified platform (e.g., the mobile devices development platform). The platform can include published SDK and libraries. Not all libraries of the platform need to be published. For example, based on various licensing agreements, some system libraries, some classes in a particular library, or some particular functions in a class may not be available to the general public or to a particular developer, depending on a license (e.g., a public license, a preferred license, etc.) held by the developer. Thus, the developer can be prohibited from accessing certain libraries, classes, or functions, even if the libraries, classes, and functions are declared “public” (instead of “private” or “protected”) in an object-oriented programming environment. The developer can test (104) the application, for example, by executing and debugging the executable code.

Once the developer is satisfied with the testing, the developer can submit the application for pre-qualification. Pre-qualification can be performed prior to uploading the entire application to a server for full review and approval. Pre-qualification can provide the developer with feedback faster than if the developer had to wait for completion of the full application review and approval process. Pre-qualification can include API validation, configuration validation and/or application resource validation.

During prequalification, an application description can be generated for the application (106). For example, before uploading the entire application to the server, a software tool can be run on a directory containing the application, or a portion of the application (e.g., library, class, compiled code snippet, etc.), to generate an application description file (e.g., a zip file, archive file, tar file, etc.). The application description can be generated at a client device (e.g., the computer that the developer uses to develop the application). The application description can include executable binary files for the application (e.g., mach-o binary files, executable code, libraries, etc.). The application description can include metadata for the application. For example, application metadata can include application configuration information (e.g., info.plist file information, nib file information, mobile provisioning file information) and/or selected application resources (e.g., images, icons, etc.).

Once the application description is generated, the application description can be submitted to a server for pre-qualification (108). For example, the application description can be transmitted from the client device to a server over a network. The server can compare the application description to policy information to determine if the application description describes an application that complies with the policies specified in the policy information. Policy information (e.g., application development policies) can identify approved and/or prohibited APIs, configurations and/or resources for the platform. The policy information can include general policies applied to all developers and/or policies that are specific to a particular developer (e.g., based on a developer license).

During pre-qualification, the server can automatically determine whether the submitted application description describes libraries, classes, functions, configurations or resources that are prohibited for use by the submitting developer. Pre-qualification can also include automatically determining whether the application program uses deprecated libraries, classes, or functions, such that although the application program can execute properly at the time of the review, the libraries, classes, or functions are scheduled to be change or removed in the future, causing the application program to fail. The server can apply general policies and/or developer-specific policies that reflect licenses that the developer holds. The server can generate warnings and/or errors based on the comparison of the application description to the policy information.

In some implementations, the client can wait to receive results of the pre-qualification from the server. For example, the client application that uploaded the application description to the server may use a blocking or synchronous call when sending the application description to the server such that the client will wait to receive the results of the pre-qualification from the server before allowing the developer to upload the application to the server. Once the server has completed its analysis of the application description, the server can send the client a list of warnings and errors resulting from the pre-qualification process. If there are no warnings or errors, the server can send the client an indication that the application description has passed the pre-qualification process. The client can then upload the full application to the server for full review, approval and distribution.

The client can receive pre-qualification warnings and/or errors (110). If the client receives an error (112), the developer can be prevented from uploading the application for full review, approval and distribution. The application developer can continue developing (102) or revising the application to fix the errors and/or warnings.

If the client receives no errors (e.g., only warnings, no warnings and no errors), the developer can upload the application for full review, approval and distribution (114). Alternatively, the developer can skip uploading the application for full review and continue developing the application (102). For example, the developer can fix the problems that generated the warning messages and regenerate and resubmit the application description for pre-qualification. For example, the error and warning messages can be presented to the developer using the interfaces described with reference toFIG. 6. In some implementations, pre-qualification can ends at step110. For example, the application may still be in development and the developer may not wish to upload the application after the pre-qualification is complete.

Once the application has gone through pre-qualification without error, the developer can submit (114) the entire application for full review and approval by a system or by a system developer (e.g., a developer responsible for the integrity of the platform). Submitting the application for review can include uploading the source code, the linked binary executable code of the application program, or both, to a server for automatic or manual review. Submitting the application for review can include uploading application resources and configuration information. The review can include a qualification portion and optionally, and additional approval portion. During the qualification portion of the review, a system can automatically determine whether the submitted application, sometimes in compiled and linked binary format only (e.g., without the source code), uses libraries, classes, or functions that are prohibited for use by the submitting developer. The qualification can also include automatically determining whether the application program uses deprecated libraries, classes, or functions, such that although the application program can execute properly at the time of the review, the libraries, classes, or functions are scheduled to be change or removed in the future, causing the application program to fail. The additional approval portion of the review can include, for example, determining that user interface of the application program conforms to guidelines provided with the SDK, or content provided by the application program conforms to local community standards.

The developer can receive (116) results of the qualification and approval. If the application program is not qualified or approved, a message can be sent to the developer. The message can include a statement that the application program did not pass the review process, a list of one or more errors that occurred, and an explanation for each error. The developer can redesign, reimplement, and retest the application program for submission again.

Upon qualification and approval from the review, the application program can be distributed (118). Distributing the application program can include storing the application program in a data store and providing the application program for download by other users (e.g., the general public).

In some implementations, the pre-qualification process can be performed without uploading the full application at step114. For example, an application developer may wish to determine whether an application, or a portion of an application, under development will pass pre-qualification (e.g., is using appropriate APIs, configuration, resources, etc.). In this case, the process can include generating a description (106) of the application, or the portion of an application, submitting the description for prequalification (110), and receiving the prequalification warnings and/or errors (110) without performing the subsequent steps112-114.

Example System of Automated Qualification

FIG. 2illustrates components of an example system implementing automated qualification of an application, including pre-qualification. A developer develops application code on computing device202. The application code can be compiled into application204. Application204can be pre-qualified for approval and distribution. The pre-qualification process can provide the application developer early notification of errors in application204that might prevent application204from being later approved for distribution. During the pre-qualification process, application description205can be generated based on application204. For example, application description205can include binary files (e.g., compiled executables, libraries, etc.) and metadata (e.g., application configuration data, application resources, images, media, etc.) associated with application204.

Application description205can be sent from computing device202to a server through a communication network. The server can include one or more hardware components (e.g., processors and storage devices) and one or more software components. The server can include subsystems that include scanner206, classifier210, approver220, and notifier224, among others. Each subsystem can include a hardware component, a software component, or both.

Scanner206can include a subsystem that scans application description205and generates scanned data (e.g., scanned symbols)208. Scanner206can include various tools for analyzing application descriptions. An example scanner206can include an object file displaying tool (e.g., “otool” of Apple, Inc. or “nm” tool of Linux). Scanner206can include a subsystem that scans application description205for configuration data that can be used to pre-qualify, or disqualify, application204. For example, scanner206can parse an info.plist file included in application description205to determine a name associated with the application. Scanned data208can include the scanned configuration information (e.g., application name).

Scanned data208can include names and version numbers of libraries described by application description205, symbols (including indirect symbols that are defined in one source file and referenced in another source file), table of contents for a dynamically linked shared library, reference table of the dynamically linked shared library, module table of the dynamically linked shared library, and other information (e.g., configuration information). Scanned data208can include symbols in their original names (e.g., function “foo”) or mangled names (e.g., function “_i_xyz_foo_para1_”) or both. In some implementations, scanned data208can include automatically generated identifiers of functions (e.g., selectors). Scanned data208can be in American Standard Code for Information Interchange (ASCII) format, Unicode format, or another textual or binary format (e.g., a compressed archive format).

Classifier210can include a subsystem that generates warnings and errors for application description205based on comparison between scanned data208and application development policy information (e.g., symbol lists, approved/prohibited configuration data, approved/prohibited resources) in data store212. For example, the warnings and errors can be generated during pre-qualification to give a developer early feedback that the developer's application is not using approved APIs. Data store212can include one or more lists of symbols, configuration data and/or resources.

Each list of symbols can include symbols with specified characteristics that can be used to determine if application204is using approved or restricted (e.g., prohibited) APIs. For example, a first symbol list in data store212can include symbols that developers using a published SDK are not permitted to access. Some examples of these symbols include names of system libraries, names of classes or methods of a system programming interface (SPI), names of classes that inherit from a system class, etc. A second symbol list in data store212can include names of classes that conflict with system classes, names of dangling classes or functions (e.g., classes or functions that are defined but not referenced), and names of libraries, classes, or functions that are deprecated. Libraries, classes, or functions can be deprecated if the libraries, classes, or functions, or their respective features are superseded or removed in future versions. A third list of symbols can include names of classes or functions that are subject to abuse. Names of classes or functions subject to abuse can include symbols in a published application programming interface (API) that can be used to hide a call to a unpublished API (e.g., an SPI). A fourth list of symbols can include names of classes or functions that are available to a particular application developer. For example, classes or functions that may generally be unavailable to developers may be made available to a particular developer by license agreement. A developer-specific list (e.g., developer specific policy) that identifies symbols that the developer is allowed to use can be stored in data store212.

Data store212can also include configuration information and/or resources that can be compared with scanned data208. For example, the configuration information can identify allowed and disallowed configuration data, allowed and disallowed resources, or any other application information.

Classifier210can compare scanned data208with the lists of symbols and metadata (e.g., application development policy data) in data store212. For example, comparing the scanned data208with the lists of symbols in data store212can include calculating a match score indicating the likelihood that a symbol in scanned data208matches each of the lists of symbols. In some implementations, a perfect match score (e.g., 1.0) between a scanned symbol and a particular list can indicate that classifier210is certain that a symbol in that list is used in application204. A less than perfect match score (e.g., 0.7) can indicate that there is less than one hundred percent likelihood that a symbol in the list is used.

Based on the comparison, classifier210can generate errors and/or warnings message211. For example, classifier210can generate errors for symbols that are described in application description205and that application developers are not permitted to access (e.g., the first symbol list above). Classifier210can generate warnings for symbols that are described in application description205and that are related to deprecated libraries, classes or functions (e.g., the second symbol lists above). Warnings can also be generated when classifier210calculates less than a perfect match score (e.g., below a threshold score, below 0.7) for a symbol that would otherwise generate an error. For example, warnings can indicate to the application developer that the symbols that generated the warnings may be subject to later review and may cause rejection of the developer's application.

In some implementations, classifier210can upgrade or downgrade warnings and errors. For example, if a developer submits application description205having a symbol that is listed in the first symbol list, generally an error is generated. However, if developer has a license to use the symbol (e.g., the symbol is listed in the fourth symbol list associated with the developer) then the error can be downgraded to a warning or the error can be ignored. In some implementations, a warning can be upgraded to an error if a symbol described in application description205is subject to abuse (e.g., listed in the third symbol list above).

Once classifier210has compared scanned data208to the symbol lists, configuration data and/or resources in data store212and generated warnings and/or errors message211, message211can be sent to computing device202. Message211can include information identifying and/or describing warnings and/or errors. For example, message211can include only warnings, only errors, a combination of warnings and errors, or no warnings and no errors (e.g., indicating the application passed pre-qualification). When computing device202receives message211, computing device can present a user interface for displaying the warnings and/or errors. For example, computing device202can display the user interface ofFIG. 6. If computing device202receives a message211that includes errors, the application developer can be prevented from uploading application204. Notifying the application developer of warnings and errors early in the approval process allows the application developer to resolve issues (e.g., API misuse, disallowed configuration, disallowed resources) early in the application approval process without having to wait for completion of the application approval process described below.

If computing device202receives a message211that includes no errors, application204can be sent from computing device202to a server through a communications network. For example, the entire application204can be sent to the server (e.g., rather than just the application description205) once the application description205does not cause classifier210to generate any errors. The full application204can then be scanned by scanner206and classified by classifier210.

Scanner206can include a subsystem that scans application204and generates scanned data208(see above). Scanner206can include various tools for analyzing applications. An example scanner206can include an object file displaying tool (e.g., “otool” of Apple, Inc. or “nm” tool of Linux).

Classifier210can include a subsystem that classifies (e.g., categorizes) application204based on comparison between scanned data208and symbol lists in symbol data store212. Symbol data store212can include one or more lists of symbols. Each list of symbols can include symbols with specified characteristics that can be used to classify application204. For example, a first symbol list in data store212can include symbols that developers using a published SDK are not permitted to access. Some examples of these symbols include names of system libraries, names of classes or methods of a system programming interface (SPI), names of classes that inherit from a system class, etc. A second symbol list in data store212can include names of classes that conflict with system classes, names of dangling classes or functions (e.g., classes or functions that are defined but not referenced), and names of libraries, classes, or functions that are deprecated. Libraries, classes, or functions can be deprecated if the libraries, classes, or functions, or their respective features are superseded or removed in future versions. A third list of symbols can include names of classes or functions that are subject to abuse. Names of classes or functions subject to abuse can include symbols in a published application programming interface (API) that can be used to hide a call to a unpublished API (e.g., an SPI).

Classifier210can compare scanned data208with the lists of symbols in data store212. Comparing the scanned data208with the lists of symbols in data store212can include calculating a match score between the scanned data208with each of the lists of symbols. In some implementations, a perfect match score (e.g., 1.0) between a scanned symbol and a particular list can indicate that classifier210is certain that a symbol in that list is used in application204. A less than perfect match score (e.g., 0.7) can indicate that there is less than one hundred percent likelihood that a symbol in the list is used.

Based on the comparison, classifier210can classify application204into one of multiple categories (e.g., category one214, category two216, and category three218). Various actions can be respectively associated with the multiple categories. For example, category one214can be associated with an action of rejection. If application204is classified as category one214, application204can be denied further approval and eventual distribution. Category one214applications can be put in a first notification queue.

Notifier224can include a subsystem that can send one or more notifications226to developers based on the classification of the application204. For example, when application204is rejected, notifier224can send notification226indicating that application204is rejected and an explanation of reasons of the rejection (e.g., use of an SPI call). Rejected application204can be removed from the first notification queue and from the system.

If application204is classified as category two216, application204can be unconditionally qualified. Category two216applications can be sent for approval by approver220. Approver220can include a subsystem that approves qualified application204. Approver220can include one or more automated processes for determining whether application204complies with various development guidelines that can include, for example, user interfaces specifications, user age requirement, or compliance with local or national community requirements. Once approved by approver220, application204can be stored in application store220, from where users can access (e.g., purchase for download) application204. A reference (e.g., an identifier) of qualified application204can be put in a second notification queue. Notifier224can notify developers of applications in the second notification queue that the developers' applications are qualified. In some implementations, notification226sent by notifier224can include a warning that a symbol scanned from application204is deprecated.

If application204is classified as category three218, application204can be conditionally qualified. Category three218applications can be temporarily put on hold before they are sent for approval or distribution, pending developer response to further inquiry. Category three218applications can be put in a third notification queue. Notifier224can send notification226to developers of applications that are in the third notification queue. Notification226for applications in the third notification queue can include the inquiry. The inquiry can seek user explanation, for example, on why a symbol scanned from application204is sufficiently similar to an SPI call. Application204can be sent to approver220if the system receives a satisfactory response from the developer. A satisfactory response can include, for example, an explanation of how the symbol is different from a system symbol even though the symbol is literally similar to the system symbol, or a change to another symbol.

Three example categories214,216, and218are shown inFIG. 2. In various implementations, more or fewer categories can be used. Furthermore, the categories can be associated with actions other than rejection, conditional qualification, and warning. Other actions are possible.

FIG. 3illustrates an example match between symbols scanned from an application, or application description, and a reference list. The implementations described herein can also be used to match configuration data and/or resources from an application description and reference lists that identify approved and/or prohibited configuration data and resources.

Scanned symbols302can include symbols scanned from the application (e.g., application204ofFIG. 2) or symbols scanned from the application description (e.g., application description205ofFIG. 2). Scanned symbols302can include names of libraries linked into the application. The names can include names of statically linked libraries and dynamically linked libraries. Each library can include a collection of constants, macros, routines, classes, or a combination of two or more of the above. Some libraries can include routines for accessing system functions. For example, a system library for a mobile device platform can include routines for accessing a baseband processor that manages communications functions of the mobile device. Even if every class or function in the system library is declared “public” in an object-oriented programming environment, the library may not be provided to the general public for various reasons. Some other libraries not available to the general public can include libraries under testing. The libraries under testing can be made available only to a selected group of developers (e.g., beta testers). To illustrate, a library having the name “abc.lib” as shown inFIG. 3is a system library or a library under testing and is not to be linked into the application.

Scanned symbols302can include class names and function names. A function can include a global function, a local function, a class method, an instance method, or any other executable routine. In some implementations, names of functions can include mangled (or decorated) names. For example, a mangled function name can include an indicator “i” or “c” that indicates whether the function is an instance method or a class method. The mangled function name can also include a name of the class (e.g., “xyz”). In some implementations, each method can correspond to a unique identifier (e.g., a selector) of the method, in addition to or in place of a mangled name. The selector can include a unique symbol (e.g., an integer) assigned to the method. Scanned symbols302can include text (e.g., ASCII) or binary form of the selector.

Classifier304can compare the scanned symbols302to reference list308, which can include a list of known symbols. Reference list308can include library names310, class names312, and function names314. Function names314can include global or local functions, class methods, and instance methods. In some implementations, function names314can include method identifiers (e.g., selectors). In some implementations, reference list308can be a blacklist. If a match between a scanned symbol and a symbol in the list is found, the application from which the symbol is scanned can be automatically rejected.

Classifier304can be a classifier such as classifier210ofFIG. 2. Classifier304can include match analyzer306that compares scanned symbols302with symbols in reference list308. In some implementations, a match is found if there is an exact match. For example, classifier304can determine that scanned symbols302matches reference list308when a library name (e.g., “abc.lib”) in scanned symbols302literally matches a library name in reference list308, or when a selector in scanned symbol302matches a selector in function names314section of reference list308.

In some implementations, match analyzer306can identify a match even when an exact match does not exist. Match analyzer306can calculate a match score between a scanned symbol and a symbol in reference list308. The match score can indicate a similarity between portions of a first symbol (e.g., the scanned symbol) and a second symbol (e.g., the symbol in reference list308). For example, a first symbol can have mangled name _i_xyz_foo_para1_. The mangled name can indicate that the application includes an instance method (e.g., a method that is associated with an object of a class) “foo” associated with class “xyz” with parameter “para1.” This function is not identical, but is similar, to a method name in reference list308having a mangled name _c_xyz_foo_para1_, indicating that the second symbol is a class method (or static method, a method associated with a class “xyz”), as indicated by the “c” prefix. A class method can be treated differently from an instance method, even if they have the same name. In this example, other than the class/instance distinction, the first symbol and second symbol are identical. The identical portions are indicated in bold type in scanned symbols302. Match analyzer306can calculate a match score based on the difference and similarity. Likewise, a first symbol _c_xyz_foo_para2—partially match the second symbol _c_xyz_foo_para1_.

In some implementations, match analyzer306can identify a match based on class inheritance. Match analyzer306can identify classes of the scanned symbols302to determine whether a particular class (e.g., “subclass_of_xyz”) is a subclass of a class (e.g., class “xyz”) listed in list308.

Classifier304can determine whether the application is classified into a category based on match scores calculated by match analyzer306between scanned symbol302and each reference list308. In some implementations, the application is classified into a category (e.g., a “rejection” category) if match analyzer306determines that there is an exact match between a scanned symbol (e.g., “abc.lib”) and a symbol (e.g., “abc.lib”) in a list (e.g., a list of unpublished system symbols). In some implementations, the application is classified into a category (e.g., the “rejection” category) if, although there are no exact matches, the match score of a scanned symbol satisfies a match threshold, indicating that it is highly likely that the application is using an unpublished library, class, or function. In some implementations, the application is classified into a category (e.g., the “rejection” category) if an aggregation of a group of scanned symbols that are similar to the known symbols results in an aggregated match score that satisfied an aggregated match threshold. Aggregating of the group of scanned symbols can include calculating the aggregated match score using a number of scanned symbols that match the known symbols in the list, and the match score of each scanned symbol.

Classifier304can classify the application in a tiered manner. Multiple reference lists308can be used. Each list can correspond to a tier. For example, a tier1list can include unpublished symbols. A tier2list can include a list of symbols that are published, but, if appeared together, can indicate a likelihood of abuse based on historical data, or can be used to hide calls to unpublished SPI functions. A tier3list can include published but deprecated symbols. Each tier can correspond to a different action (e.g., reject, hold, or warn). The system can start by comparing scanned symbols301with tier1lists, tier2lists, and so on, in the order of the tiers.

In addition to or alternative to the tiered lists, classifier304can use characteristics of scanned symbols302to classify the application. For example, classifier304can specify that a threshold number of libraries are required of each application. If a number of linked libraries in scanned symbols302fails to satisfy the threshold number of libraries, classifier304can determine that the application is compiled using an unauthorized compiler or is written to purposefully circumvent unpublished libraries, classes, or functions. Accordingly, classifier304can classify the application such that the application will be rejected or put on hold.

Classifier304can generate warnings and errors during pre-qualification based on match scores calculated by match analyzer306between scanned symbol302and each reference list308. For example, errors can be generated when the application description contains information that would cause classifier304to place an application in the rejection category, described above. Warnings can be generated when the application description contains information that would cause classifier304to place an application in a category (e.g., hold) other than rejection or approve.

FIG. 4is a block diagram illustrating an example system that creates reference lists. The reference lists can be created from data store404that includes existing libraries. The existing libraries can include published libraries (e.g., the libraries that have been provided to developers in an SDK) and unpublished libraries (e.g., system libraries, experimental libraries, or proprietary libraries). Additionally or alternatively, the lists can be created from data store406that includes existing applications that have been qualified, applications that are created by system developers, or applications that are created by a selected group of developers (e.g., developers who are entitled to bypass the review process).

Library scanner402can scan libraries in data store404and applications in data store406to create reference lists414,416,418, and420using rules412. Rules412can specify access privileges of each library, class, and function. For example, rules412can specify which library is a published library, which library is an unpublished library, which classes in a particular library have limited access privilege, and which functions in a library or which methods in a class have limited access privilege. In some implementations, rules412can associate the access privileges with various licenses granted to developers. For example, a first license can specify that a developer under the first license can access a first set of libraries, classes, or functions. A second license can specify that a developer under the second license can access a second set of libraries, classes, or functions that are different from the first set. Library scanner can create a set of reference lists to be associated with each license or category of license. When a developer submits an application for qualification, a current license for the user is determined (e.g., based on the user's identification as the user signed up to acquire the SDK), and a corresponding set of references lists is used to qualify, or pre-qualify, the application submitted by the user.

Statistics engine408can monitor the libraries of data store404and existing applications of data store406for patterns of symbol usage. Monitoring can include analyzing the libraries and existing applications to determine a frequency of use of various libraries, classes, and functions. Statistics engine408can submit results of the monitoring to moderator410. Moderator410can determine whether a particular symbol is to be moved from a first reference list to a second reference list based on the results. For example, statistics engine408can determine a frequency with which a function is referenced. The statistic engine408can determine that the frequency satisfies a usage threshold. According to current rules412, the function is scheduled to be removed in a future release of the SDK. Based on the usage, moderator410can determine that the removal of the function should be delayed to avoid concurrent invalidations of a large number of applications. Moderator410can send a notice of the usage to a system developer and recommend that the function is to be preserved. Upon receiving a response from the system developer indicating that the function is to be preserved, moderator410can modify rule412to specify that the function is not to be deprecated. Additionally or alternatively, moderator410can modify one or more of reference lists414,416,418, and420, for example, to remove a symbol of the function from a deprecated function list and insert it into a permissible function list.

Likewise, moderator410can identify which library is used by developers the most, and recommend to the system developer that the most used library can be a first choice of improvement in a next release of the SDK. Moderator410can identify an unpublished library that has caused most rejections of applications, and recommend converting the unpublished library or a modified version of the unpublished library into a published library.

Example System of Pre-Qualification

FIG. 5illustrates components of example system500where a developer can pre-qualify an application before submitting the application for review. Pre-qualification can be utilized to reduce the time required for review by eliminating preventable rejections from the system. Pre-qualification can provide the developer with early feedback that allows the developer to fix problems with the application before uploading the application for approval and distribution thereby saving the developer time during the development, review, approval and distribution process.

A developer can create an application on computing device502using an SDK under a license for a platform (e.g., a mobile device development platform). The developer may not knowingly use libraries, classes, or functions to which the developer has no access according to the license. However, the developer links in one or more third party libraries into the executable of the application. The developer can run an application description utility on the application to generate an application description. The developer can send application description504to inspector506. Inspector506can include a set of instructions executed on a server that is connected to computing device502through a communications network. The server can be the same server that receives the full application for review, approval and distribution. Inspector506can be a stand-alone application program, a plug-in of an IDE, or a web-based service. Inspector506can include, for example, some or all of components scanner206, classifier210, and notifier224as described above with respect toFIG. 2.

Inspector506can be coupled to data store508, which can store one or more symbol, configuration and resource data reference lists against which the symbol, configuration and resource data scanned from application description504can be compared. For example, a reference list can identify prohibited symbols, configurations, and/or resources. A reference list can identify allowed symbols, configurations, and/or resources. A first reference list can be used to modify a second reference list. For example, a developer-specific reference list can identify APIs that the developer is allowed to use (e.g., according to a developer license) even though the APIs may be included on a prohibited API reference list. The reference lists in data store508can be stored remotely (e.g., on a server of the publisher of the SDK or on a server of a third party). The reference lists in data store508can be updated periodically or upon request.

For example, inspector506can scan application description504for symbols and compare the scanned symbols with symbols in the reference lists stored in data store508. Inspector506can identify one or more suspicious symbols in the scanned symbols. Each suspicious symbol can be a symbol that matches a symbol in a reference list of symbols that the developer is not permitted to access, a reference list of symbols that are prone to be abused, or a reference list of symbols of libraries, classes, or functions to be removed in the future. Likewise, inspector506can scan application description504for configuration data and/or resources, compare the scanned configuration data and/or resources with configuration data and resources in the reference lists stored in data store508, and identify suspicious configuration data and/or resources.

Inspector506can compile a list of warnings and errors associated with the suspicious symbols, configuration data and/or resources and send notification510to formatter512. Formatter512can identify the warnings and errors from notification510, associate each warning and error with a reason of suspicion, and format the errors, warnings and the reasons into formatted notification514. Formatted notification514can be sent to computing device502for display on a display screen. The developer can modify application504in response, for example, by linking a different library and repeat the pre-qualification process, until application description504no longer generates errors and warnings. The developer can submit the pre-qualified application504for approval once the pre-qualification process produces no errors.

In addition to pre-qualifying an application, system500can qualify other binaries. For example, inspector506can scan and pre-qualify object code (e.g., a compiled section of an application program) or a binary library (e.g., a downloaded third party library). In some implementations, inspector506can use reference lists in data store508to modify warnings and errors (e.g., upgrade to error, downgrade to warning or no error) based on a license associated with an SDK. For example, if a developer acquires a new license that grants the developer more access privileges (e.g., access privileges to a system library that is previously not accessible by the developer), reference lists in data store508can be modified to reflect the terms of the new license and inspector506can upgrade and/or downgrade warnings and errors according to the modified reference list associated with the developer.

FIG. 6illustrates example user interface600for notifying a developer of errors and warnings identified for an application description. Example user interface600can be a user interface for displaying formatted notification514ofFIG. 5, or a user interface for displaying notification226as described above with respect toFIG. 2. Example user interface600can include at least portion of a web page.

Error section602of user interface600can include a display area of user interface600that displays errors. For example, error section602can display a name of an unpublished system library (e.g., “abc.lib”) that is not accessible by the developer under a current license. Error explanation section604can include a display area of user interface600that displays explanation of the errors, as well as suggested change (e.g., “Consider using another library”). When multiple errors are present, error section602can be configured to accept a user input scrolling the errors and selecting a particular error. Explanation section604can display the explanation of the selected error.

Warning section606can include a display area of user interface600that displays warnings. In some implementations, warning section606can display a name of a class (e.g., “xyz”) that is historically subject to abuse. For example, class “xyz” from in the library linked in by the user can include features that are known to be subject to security attack by hackers. Warning section606can also display names of libraries, classes, and functions (e.g., class function “foo” and instance function “bar”) that will be changed or removed.

Warning explanation section608can include a display area of user interface600that displays explanation of the warnings, as well as suggested change. When multiple warnings are present, warnings section606can be configured to accept a user input scrolling the warnings and selecting a particular warning. Explanation section604can display the explanation of the selected warning.

Example Pre-Qualification and Qualification Processes

FIG. 7Ais a flowchart illustrating example process700of pre-qualifying an application. For convenience, process700will be described in reference to a system that implements process700. The system can receive an application description (702). The application description can include compiled and linked executable files (e.g., binary files, mach-o files, compiled libraries, etc.). The application description can include application metadata (e.g., info.plist file data, mobile provisioning file information, embedded provisioning file information, resources, and other configuration information).

The system can scan the application description for symbols and metadata (704). For example, the system can scan for symbols contained in the executable binary files that represent libraries, classes and functions that are referenced in the executable binary files. The system can scan the metadata for configuration information (e.g., product name, version number, etc.).

The system can compare the symbols and/or metadata to policy data (706). For example, the system can compare the symbols to API policy information that specifies which APIs an application developer can use. The system can compare the symbols and/or metadata to general policy data (e.g., policies that apply to all developers, reference lists). The system can compare the symbols and/or metadata to developer-specific policy data (e.g., policy data that reflects the terms of a developer's license).

If the symbols and metadata included in the application description do not comply with the policies defined in the policy data, the system can generate warnings and/or errors that can be transmitted (710) to the developer to notify the developer that the application does not comply with the policies. The application developer can be notified that that the application developer will not be able to upload the application for qualification and approval until the errors are resolved.

FIG. 7Bis a flowchart illustrating example process750of automatically qualifying an application. For convenience, process750will be described in reference to a system that implements process750.

The system can scan (752) an application for first symbols. The application can be an application program submitted by a developer for approval. The first symbols can include at least one of a library name, a class name, and a function name. The function name can include a mangled function name, a function identifier (e.g., a selector), or both. The function can include a global function, a local function name, or a class method or an instance method, or any executable routine.

The system can compare (754) the scanned first symbols with multiple lists of second symbols. Each of the multiple lists of second symbols can correspond to a specified access privilege of the second symbols in the lists that has been granted to the application. The second symbols can include at least one of a library name, a class name, and a function name. For example, the list of second symbols can include a first list of system symbols, including names of system libraries, classes, or functions. Under a current license of the developer submitting the application, an access from the submitted application to the system libraries, classes, or functions in the first list can be prohibited.

The list of second symbols used in stage754can include a second list of published symbols. Under the current license, access from the application to the published symbols is permitted for an indefinite amount of time. The list of second symbols used in stage754can include a third list of deprecated symbols. The access to the deprecated symbols can be permitted for a limited amount of time (e.g., until a deprecated library, class, or function is removed).

In stage754, comparing the scanned first symbols with the multiple lists of second symbols can include calculating a likelihood that the scanned first symbols match one of the lists of the second symbols. Calculating the likelihood can include calculating a match score between each scanned first symbol and each second symbol of a list of second symbols. The match score can measure a probability that a scanned first symbol is equivalent to a second symbol, even when the scanned first symbol literally differs from the second symbol. The system can associate the match score to the scanned first symbol, and calculate the likelihood based on an aggregation of the match scores. In some implementations, calculating the likelihood can include identifying one or more of the scanned first symbols whose associated matching scores exceed a threshold, and calculating the likelihood based on a ratio between the identified one or more of the scanned first symbols and a number of all of the scanned first symbols.

The system can classify (756) the application at least in part based on the comparing. In some implementations, the system can provide trustworthy users preferred status. The system can determine (758) a trustworthiness of a user (e.g., the developer submitting the application program currently being qualified) based on the classification of the currently submitted application and a history of classification of application programs submitted by the developer. For example, if the total number of applications submitted by the developer satisfies a threshold number, and a threshold percentage of the submitted applications (e.g., 100%) are qualified, the developer can be designated as a trustworthy developer.

The system can add (760) an identifier of the user to an expedited process list, such that future submissions of applications by the user are qualified in an expedited manner. Approving an application in the expedited manner can include moving the application to a head of a queue of application waiting to be examined. Approving an application in the expedited manner can include giving more weight to the symbols used in the application when the system determines a frequency of use of the symbols.

In some implementations, process750can optionally include creating the lists of second symbols automatically. Creating the lists of second symbols can include determining a popularity score of a first symbol scanned from historically submitted applications, and adding the first symbol to a list of published symbols the access to which is permitted for an indefinite amount of time based on the popularity score.

FIG. 8Ais a flowchart illustrating example process800for generating warnings and errors from on an application description. In some implementations, example process800can be a process that corresponds to stage708of example process700ofFIG. 7A. For convenience, process800will be described in reference to a system that implements process800.

The system can receive (802) results of comparison between first symbols scanned from an application description and lists of second symbols. The application description can be submitted by a user (e.g., a developer). Pre-qualifying the application can include performing an action on the user-submitted application description.

The system can make a first determination at stage804on whether to generate an error. The first determination can include determining whether the scanned first symbols include unpublished symbols. The first determination can include determining whether the scanned first symbols include symbols that indicate that the application uses classes inherited from system classes, or classes private to the system developers. The first determination can include determining whether the scanned first symbols include names of system libraries, or libraries private to the system developers. Upon determining that an error should be generated, the system can generate an error (806).

Upon determining that an error need not be generated, the system can make a second determination at stage808on whether to generate a warning. The second determination can include determining that the likelihood that at least one of the first symbols scanned from the application uses a system function or a function private to a system developer satisfies a threshold. The likelihood can be calculated based on the match scores associated with the first symbols. For example, the likelihood can be calculated based on an aggregation of the match scores. Upon determining that a warning should be generated, the system can generate the warning (810).

The system can make a third determine whether to generate a warning at stage812. The third determination can include determining whether a class name in the scanned symbols is identical to a name of a system class or a private class, but the scanned symbols excludes a library containing the system class or private class. This can indicate that the application uses a class definition that conflicts with the system class or private class. The third determination can include determining whether any libraries, classes, or functions are dangling. For example, the system can determine that a class or a function is defined but not referenced. The third determination can include determining whether the scanned first symbols include a deprecated library, class, or function. Upon determining that a warning should be generated at stage814, the system can generate the warning (814).

In some implementations, at least one of the second determination of stage808and the third determination of stage812can include determining whether the scanned symbols include symbols identified as part of an API that is subject to abuse. An API that is subject to abuse can include a published API that can be used to hide system or private API usage. Additionally or alternatively, at least one of the second determination of stage808and the third determination of stage812can include determining whether a number of libraries linked into the application falls below a threshold. If there are too few linked libraries, the system can determine that an unapproved library is used, which can circumvent system or private API usage.

Moreover, stages804,808and812can include determining whether the scanned symbols include symbols identified as part of an API that the developer has a license to use. For example, if the developer has a license to use an API that generated an error or a warning, the error or warning may be downgraded such that the error or warning is disregarded, no error or warning notification is sent to the developer, and the developer may be allowed to upload the application for review, approval and distribution.

Upon determining that no errors have been generated during pre-qualification (816), the system can notify (818) the developer of any warnings that have been generated and allow the developer to upload the application corresponding to the application description. For example, the system can send a message to the developer that warns the developer that the system has identified a likely call to an unpublished function.

Upon determining that at least one error was generated during pre-qualification, the system can notify (820) the developer of the reasons for the error and prevent the developer from uploading the full application for approval and distribution. The notification can include a description of the reasons for the errors, or details describing which symbols have caused the errors.

FIG. 8Bis a flowchart illustrating example process850of classifying an application. In some implementations, example process850can be a process that corresponds to stage756of example process750ofFIG. 7B. For convenience, process800will be described in reference to a system that implements process850.

The system can receive (852) results of comparison between first symbols scanned from an application and lists of second symbols. The application can be submitted by a user (e.g., a developer). Classifying the application can include performing an action on the user-submitted application.

The system can make a first determination at stage854on whether to take a rejection action. The first determination can include determining whether the scanned first symbols include unpublished symbols. The first determination can include determining whether the scanned first symbols include symbols that indicate that the application uses classes inherited from system classes, or classes private to the system developers. The first determination can include determining whether the scanned first symbols include names of system libraries, or libraries private to the system developers.

Upon determining that the rejection action is to be taken, the system can automatically reject (855) the application. Rejecting the application can include discarding the application. The system can notify (856) the developer reasons for the rejection. The notification can be send as an electronic mail. The electronic mail can include a summary of the reasons for the rejection, or details describing which symbols have caused the rejection.

Upon determining that the rejection action need not be taken, the system can make a second determination at stage858on whether to take a hold action. The second determination can include determining that the likelihood that at least one of the first symbols scanned from the application uses a system function or a function private to a system developer satisfies a threshold. The likelihood can be calculated based on the match scores associated with the first symbols. For example, the likelihood can be calculated based on an aggregation of the match scores.

Upon determining that the hold action is to be taken, the system can hold (860) the application in an on-hold queue. The system can automatically notify (862) the developer that the system has identified a likely call to an unpublished function. The notification can include an inquiry for explanation with regard to the symbol. Upon receiving a satisfactory response to the inquiry, the system can move the application from the on-hold queue and send the application for further approval or distribution.

Upon determining that the hold action need not be taken, the system can make a third determination at stage864on whether to take a warn action. The third determination can include determining whether a class name in the scanned symbols is identical to a name of a system class or a private class, but the scanned symbols excludes a library containing the system class or private class. This can indicate that the application uses a class definition that conflicts with the system class or private class. The third determination can include determining whether any libraries, classes, or functions are dangling. For example, the system can determine that a class or a function is defined by not referenced. The third determination can include determining whether the scanned first symbols include a deprecated library, class, or function.

Upon determining that the warn action is to be taken, the system can qualify (866) the application. The application program can be subject to further approval. The system can warn (868) the developer by sending a warning message to the developer informing the developer of the determination.

In some implementations, at least one of the second determination of stage808and the third determination of stage864can include determining whether the scanned symbols include symbols identified as part of an API that is subject to abuse. An API that is subject to abuse can include a published API that can be used to hide system or private API usage. Additionally or alternatively, at least one of the second determination of stage858and the third determination of stage864can include determining whether a number of libraries linked into the application falls below a threshold. If there are too few linked libraries, the system can determine that an unapproved library is used, which can circumvent system or private API usage.

Upon determining that the warn action need not be taken, the system can qualify (870) the submitted application and notify (872) the developer submitting the application that the application has been qualified.

Example System Architecture

FIG. 9is a block diagram of an example system architecture900for implementing the features and operations described in reference toFIGS. 1-8. Other architectures are possible, including architectures with more or fewer components. In some implementations, architecture900includes one or more processors902(e.g., dual-core Intel® Xeon® Processors), one or more output devices904(e.g., LCD), one or more network interfaces906, one or more input devices908(e.g., mouse, keyboard, touch-sensitive display) and one or more computer-readable mediums912(e.g., RAM, ROM, SDRAM, hard disk, optical disk, flash memory, etc.). These components can exchange communications and data over one or more communication channels910(e.g., buses), which can utilize various hardware and software for facilitating the transfer of data and control signals between components.

The term “computer-readable medium” refers to any medium that participates in providing instructions to processor902for execution, including without limitation, non-volatile media (e.g., optical or magnetic disks), volatile media (e.g., memory) and transmission media. Transmission media includes, without limitation, coaxial cables, copper wire and fiber optics.

Computer-readable medium912can further include operating system914(e.g., Mac OS® server, Windows® NT server), network communication module916, database interface920, scanner930, classifier940, approver950, and notifier960, as described in reference toFIGS. 1-8. Operating system914can be multi-user, multiprocessing, multitasking, multithreading, real time, etc. Operating system914performs basic tasks, including but not limited to: recognizing input from and providing output to devices906,908; keeping track and managing files and directories on computer-readable mediums912(e.g., memory or a storage device); controlling peripheral devices; and managing traffic on the one or more communication channels910. Network communications module916includes various components for establishing and maintaining network connections (e.g., software for implementing communication protocols, such as TCP/IP, HTTP, etc.). Database interface920can include interface to various data stores such as data stores212,404, and416, as described above in reference toFIG. 2andFIG. 4. Scanner930can include a scanner that corresponds to scanner206as described above with respect toFIG. 2. Classifier940can include a classifier that corresponds to classifier210as described above in reference toFIG. 2and classifier304as described above with respect toFIG. 3. Approver950can include an approver that corresponds to approver220as described above in reference toFIG. 2. Notifier960can include a notifier such as notifier224as described above in reference toFIG. 2.

Architecture900can be included in any device capable of hosting a database application program. Architecture900can be implemented in a parallel processing or peer-to-peer infrastructure or on a single device with one or more processors. Software can include multiple software components or can be a single body of code.

A number of implementations have been described. Nevertheless, it will be understood that various modifications can be made without departing from the spirit and scope of the disclosure. For example, an application is described. In various implementations, binaries other than application programs (e.g., libraries) or application programs other than binaries (e.g., applications written in a script language in plain text) can be automatically qualified. Accordingly, other implementations are within the scope of the following claims.