Patent ID: 12216567

all according to at least one embodiment described in the present disclosure.

SUMMARY

According to an aspect of the invention, an embodiment includes a method of evaluating a computer-implemented product that is deployed on one or more endpoints. The method may include identifying a first program and a second program of a product deployed on a first endpoint of one or more endpoints. The method may include implementing a diagnostic process at the first endpoint. The diagnostic process may include multiple subroutines including a first subroutine that is directed to the first program and a second subroutine that is directed to a second program. The first and second subroutines may each execute a set of installation and functional parameter tests of the first and second programs, respectively. The method may include itemizing results of the first subroutine and the second subroutine according to a corresponding program. In response to the results of the first subroutine indicating that the first program is installed and operational at the first endpoint, the method may include outputting data indicating the first program successfully passed the first subroutine. In response to a first test of the set of installation and functional parameter tests of the second subroutine returning an unexpected result, the method may include outputting data indicating the second program failed the second subroutine and details of the first test and the unexpected result and implementing a remediation specific to the unexpected result and the first test. The remediation may be configured to modify the second program or a condition at the first endpoint to mitigate the unexpected result and enable the second program to pass the first test.

An additional aspect of an embodiment includes a non-transitory computer-readable medium having encoded therein programming code executable by one or more processors to perform or control performance at least a portion of the method described above.

The object and advantages of the embodiments will be realized and achieved at least by the elements, features, and combinations particularly pointed out in the claims. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention, as claimed.

DESCRIPTION OF SOME EXAMPLE EMBODIMENTS

The embodiments described in this disclosure are related to test and automated remediation of deployed computing products, and in particular to template-based product diagnostic tests including program-specific test subroutines. These and other embodiments are described with reference to the appended Figures in which like item number indicates like function and structure unless described otherwise. The configurations of the present systems and methods, as generally described and illustrated in the Figures herein, may be arranged and designed in different configurations. Thus, the following detailed description of the Figures, is not intended to limit the scope of the systems and methods, as claimed, but is merely representative of example configurations of the systems and methods.

FIG.1depicts an example operating environment100in which some embodiments of the present disclosure may be implemented. The operating environment100may be configured to assess a product114of endpoints118A and/or118B (generally, endpoint118or endpoints118). Assessment of the product114may be implemented by product analysis modules116A and/or116B (generally, product analysis module116or product analysis modules116) on the first endpoint118A or the administrative device106. The assessment may be implemented by the product analysis module116after the product114is loaded to the first endpoint118A and prior to utilization of the product114by a user124. The product analysis module116may be separate from an agent208of the first endpoint118A and may at least partially test functionality of the agent208. Accordingly, the functionality of the product114may be assessed prior to the agent208being loaded or in circumstances in which the agent208is inoperable or partially inoperable.

In some conventional systems, a product such as the product114may be assessed by the agent208or using the agent208. For instance, the agent208may have a high level of access to systems of the first endpoint118A. The agent208may implement testing routines or access data related to parameters on the first endpoint118A. The agent may then communicate data indicative of the parameter and/or results of the testing to an administrator device106. These and other conventional systems cannot be implemented in devices such as the endpoints118, having an inoperable agent. Some embodiments of the present disclosure address this technical shortcoming.

Moreover, even in circumstances in which the agent208is functional, in some conventional systems, agent-based product assessments simply provide data regarding the assessment to the administrative device106. After the data is received, an administrator reviews and organizes the data and manually implements a remediation. Additionally, the product114may include multiple programs112A and112B. The programs112may be substantially separate. Accordingly, inoperability of a first program112A may not necessitate inoperability of a second program112B. Conventional, agent-based product assessment may mix results from testing of the product114between the programs112, which may increase complexity and difficulty in deciphering an actual failure and implementing a corresponding remediation.

Some embodiments of the present disclosure provide a technical solution to these and other technical limitations. For instance, the product analysis module116may implement one or more subroutines. Each of the subroutines may be directed to one of the programs112. The subroutines include tests that evaluate one or more parameters or features of the corresponding program112. Results of the tests may be itemized or organized according to the program112and/or the tests. The results may be caused to be displayed organized according to the programs112. Additionally, the subroutines of the product analysis module116may include remediations. The remediations may be specifically configured to address issues of one of the tests. For instance, the remediations, when executed, may each make a change to the first endpoint118A or to one of the programs112. The remediations may resolve a technical issue at the endpoint118or the program112such that the program112successfully passes the test. The remediations may be automatically implemented or may be implemented responsive to selection by the user124or an administrator.

Accordingly, the product analysis module116may enable complete or a more complete assessment of the product114than conventional systems. Moreover, the product analysis module116individually assesses the programs112, organizes results of tests of the programs112, and automatically remediates technical issues.

The operating environment100may include the first endpoint118A, a second endpoint118B, and an administrative device106that may communicate data and information via a network122. Each of these components are described in the following paragraphs.

The network122may include any communication network configured for communication of signals between the components (e.g.,118and106) of the operating environment100. The network122may be wired or wireless. The network122may have numerous configurations including a star configuration, a token ring configuration, or another suitable configuration. Furthermore, the network122may include a local area network (LAN), a wide area network (WAN) (e.g., the Internet), and/or other interconnected data paths across which multiple devices may communicate. In some embodiments, the network122may include a peer-to-peer network. The network122may also be coupled to or include portions of a telecommunications network that may enable communication of data in a variety of different communication protocols.

In some embodiments, the network122includes or is configured to include a BLUETOOTH® communication network, a Z-Wave® communication network, an Insteon® communication network, an EnOcean® communication network, a Wi-Fi communication network, a ZigBee communication network, a representative state transfer application protocol interface (REST API) communication network, an extensible messaging and presence protocol (XMPP) communication network, a cellular communications network, any similar communication networks, or any combination thereof for sending and receiving data. The data communicated in the network122may include data communicated via short messaging service (SMS), multimedia messaging service (MMS), hypertext transfer protocol (HTTP), direct data connection, wireless application protocol (WAP), e-mail, smart energy profile (SEP), ECHONET Lite, OpenADR, or any other protocol that may be implemented with the administrative device106and the endpoints118.

The administrator device106may include a hardware-based computing system. The administrator device106generally refers to a computing asset or device that is implemented to assess the product114on the endpoints118. For example, the administrator device106may include a remote product analysis module116B, which is remote to the endpoints118. The product analysis module116B may operate substantially similarly to the local product analysis module116A of the first endpoint118A or may implement one or more operations described with reference to the local product analysis module116B. For example, in some embodiments, the administrative device106may receive results data from the local product analysis module116A. The results data may be displayed locally at the administrative device106or may be stored. Additionally, the administrative device106may implement or initiate remediations on the endpoints118.

The administrative device106may provide management of one or both of the private networks120. The administrative device106might load the product114onto the first endpoint118A. For instance, the administrative device106may load the product114onto the first endpoint118A when the first endpoint118A is incorporated into the first private network120A. The diagnostic process may be implemented to ensure the product114is operational.

The endpoints118may be associated with users124A and124B (generally, user124or users124). The users124may include a person or set of people who regularly operate or are otherwise affiliated with the endpoints118. In some implementations, the users124may be uniquely associated with one of the endpoints118. The user124may operate one of the endpoints118to operate the product114following evaluation by the product analysis module116. Additionally, in some embodiments the user124may initiate a diagnostic process implemented by the product analysis module116. Additionally or alternatively, the user124may interact with the product analysis module116to review results, initiate remediations, etc.

The users124may be individuals within an association or entity. For instance, the users124may have a particular job or role within the entity such as an engineer, a business leader, or a contractor. Additionally, the users124may have particular security or access privileges, which may be associated with the role. A first user124A may have a separate or different role than a second user124B or may have a similar role to the second user124B.

The endpoints118may include hardware-based computing systems. The endpoints118generally refer to computing assets or devices such as a laptop computer, a desktop computer, a server device, a mobile device (e.g., tablet, smart phone, etc.), a printer, server devices, a smart screen, a telephone system, and the like.

In some embodiments, the endpoints118may be configured to manage other devices. For instance, the endpoints118may be communicatively coupled (e.g., via the network122) to other devices to perform some management functions. The management functions may include unified endpoint management (UEM), service management, patch or update management, data access management, and the like.

The endpoints118may be integrated into a private network120. The private networks120may be managed networks, which are configured to supervise and support devices of a particular entity such as an enterprise or government entity.

The product114may be configured to provide management functions related to the private network120. For instance, the product114may include a first program112A that is a patch management program, a second program112B that is a service management program, etc. The product114may be a software as a service (SAAS) product that is overseen or supported by the administrative device106.

The product114is loaded onto the endpoints118. The product analysis module116may be configured to test and assess the product114prior to operation by the user124or during operation by the user124. The product114may include multiple programs112. The programs112may be included in a single product114, but may be configured for analysis at least partially independently. For instance, the first program112A may provide UEM service and the second program112B may provide service management. The first and second programs112A and112B may operate based on common data (e.g., information and data related to the endpoint118and the users124) but may implement substantially separate operations. Accordingly, the first program112A may be tested independently of the second program112B by the product analysis module116.

The product analysis module116is configured to evaluate the product114deployed on the first endpoint118A. In some embodiments, the product analysis module116may identify the programs112that are included in the product114. The programs112may include a UEM program, a patch program, an information technology service management (ITSM) program, an endpoint management program (EPM) program, a cloud management program, a virtual private network (VPN) program, a patch update program, an asset management program, combinations thereof or another suitable program. InFIG.1, the agent208is separate from the programs112. However, in some embodiments, the agent208may be one of the programs112configured to operate on the first endpoint118A.

The endpoints118might also include the agent208. In some embodiments, the administrative device106may interface with the agent208. For instance, the agent208may have a high level of privilege on the endpoints118, which enables visibility of the product114as well as operational parameters related to or characterizing the product114. The agent208may be configured to exist on the endpoints118to support ongoing management of the endpoints118. The agent208may interface with local applications (e.g., the search feature) and may support communication of information back to the administrative device106.

The product analysis module116may implement a diagnostic process at the first endpoint118A or may implement the diagnostic process at a subset of the one or more endpoints118. In some embodiments, the user124of the first endpoint118A may initiate the diagnostic process. In some embodiments, the administrative device106may initiate the diagnostic process remotely.

The product analysis module116may include two or more subroutines. The subroutines may be directed to one of the programs112. For instance, a first subroutine may be directed to the first program112A and a second subroutine may be directed to the second program112B.

The subroutines each execute tests that assess proper installation and functional parameters of the programs112. In some embodiments a first set of installation and functional parameter tests of the first subroutine and a second set of installation and functional parameter tests of the first subroutine includes diagnostic different aspects of each program112of the product114. The product analysis module116may execute the subroutines and tests therein, which may generate results data. The product analysis module116may itemize result data. The results may be itemized according to a corresponding program112, subroutine, tests, or combinations thereof.

In response to the results of the first subroutine indicating that the program112is installed and operational at the first endpoint118A, the product analysis module116may output data indicating the program112successfully passed the test. In response to a test returning an unexpected result or invalid result, the product analysis module116may output data indicating the program112failed and/or details of the test and the unexpected result. Additionally, the product analysis module116may implement a remediation. The remediation may be specific to the unexpected result and/or the failed test. The remediation may be configured to modify the program112that failed the test and/or modify a state or a condition at the first endpoint118A to mitigate the unexpected result and enable the program112to pass the test.

The product analysis module116may cause display of data and information related to the diagnostic process. The product analysis module116may cause display of the data and information at the first endpoint118A and/or the administrative device106. For instance, the product analysis module116may cause display of a selectable instruction, icons that correspond to the programs112. The selectable instruction may be configured to initiate at least a portion of the remediation. A first icon may be associated with the first program112A and may indicate, for example, that the first program112A successfully passed a first subroutine. A second icon may be associated with the second program112B and may indicate, for example, that the data output by the second subroutine indicates that the second program112B failed the second subroutine.

The product analysis module116may determine whether the second endpoint118B is similar to the first endpoint118A. The second endpoint118B may be in the same private network120as the first endpoint118A or in a separate private network120. The product analysis module116may implement the remediation or some portion thereof at the second endpoint118B. The product analysis module116may implement the remediation without implementing the diagnostic process at the second endpoint118B. Implementation of the remediation without the diagnostic process may improve efficiencies in the operating environment100. For instance, instead of independently testing each endpoint118, the remediation is implemented based on characteristics of the endpoints118or users124. This is an improvement over agent-based diagnostics that submit results data from every endpoint118prior to remediations being implemented.

The product analysis modules116, the product114, the programs112, and components thereof may be implemented using hardware including a processor, a microprocessor (e.g., to perform or control performance of one or more operations), a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC). In some other instances, the product analysis modules116, the product114, the programs112, and components thereof may be implemented using a combination of hardware and software. Implementation in software may include rapid activation and deactivation of one or more transistors or transistor elements such as may be included in hardware of a computing system (e.g., the endpoints118or the administrative devices106). Additionally, software defined instructions may operate on information within transistor elements. Implementation of software instructions may at least temporarily reconfigure electronic pathways and transform computing hardware.

Modifications, additions, or omissions may be made to the operating environment100without departing from the scope of the present disclosure. For example, the operating environment100may include one or more administrative devices106, one or more networks122, one or more endpoints118, or any combination thereof. Moreover, the separation of various components and servers in the embodiments described herein is not meant to indicate that the separation occurs in all embodiments.

FIG.2depicts an example diagnostic process200that may be implemented in the operating environment100ofFIG.1or another suitable environment.FIG.2includes some components (e.g.,118,116,114,106, etc.) described with reference toFIG.1. Description of these components is not repeated. Additionally, althoughFIG.2does not include the network122ofFIG.1, it may be understood that some or all communications of the diagnostic process200may be via the network122or another suitable communication network.

In the embodiment ofFIG.1, the diagnostic process200is implemented at least partially by the local product analysis module116of the first endpoint118A. Accordingly, details of the local product analysis module116of the first endpoint118A are shown inFIG.2. In other embodiments, the remote product analysis module116B of the administrative device106may implement at least some portions of the diagnostic process200. In these embodiments, the remote product analysis module116B of the administrative device106may be substantially similar to the local product analysis module116A of the first endpoint118A as described with reference toFIG.2.

The diagnostic process200may be implemented after the product114is installed at the first endpoint118A. The diagnostic process200may be implemented to ensure the product114is functional and operational prior to allowing a user such as the user124ofFIG.1employs the product114. Additionally, the diagnostic process200may be implemented prior to installation of an agent208. Accordingly, the diagnostic process200may be able to test functionality of the agent208, the product114, as well as communication and functionality between the product114, the agent208, and the administrative device106.

In the depicted embodiment, the product analysis module116may include a diagnostic process module210. The diagnostic process200may begin by the diagnostic process module210identifying the programs112in the product114. For instance, the first program112A and the second program112B of the product114deployed on the first endpoint118A may be identified by the diagnostic process module210. In some embodiments, the programs112may include an agent such as the agent208or another program implemented on the first endpoint118A such as an UEM program, a patch program, a ITSM program, a EPM program, a cloud management program, some portion thereof, combinations thereof, or another suitable program

The diagnostic process module210may be specific to the product114. For instance, the diagnostic process module210includes subroutines212A and212B, which may correspond to the programs112of the product114. Specifically, a first subroutine212A may be directed to and test the first program112A and a second subroutine212B may be directed to and test the second program112B.

The subroutines212may include one or more tests214A and214B (generally, test214or tests214). The tests214are configured to evaluate a parameter, a feature, a characteristic, or combinations thereof of one of the programs112. For instance, the tests214may include programming instructions that perform one or more operations that evaluate functionality of a portion of one of the programs112. Some of the tests214may be configured to evaluate whether the parameter of the program112is properly installed, a security parameter of the program112, the function of the program112, etc.

For example, in the embodiment ofFIG.2, the first subroutine212A includes a first test214A and a second test214B. The first test214A may test functionality of a first particular element or feature of the first program112A. Similarly, the second test214B may test functionality of a second particular element or feature of the first program112A. Although not shown inFIG.2, the second subroutine212B may similarly include a set of installation and functional parameter tests directed to the second program112B.

Some examples of the tests214of the subroutines212may include verification of locations of files and data of one of the programs112(e.g., network location or local location of files and data), verification of a profile or role of a user (e.g., the user124) that implemented one of the programs112, identification of a time involved in carrying out a computing process in one of the programs112, verification of a version of one of the programs112, verification of enrollment of one of the programs112with a second system, verification of a source of one of the programs112, verification of a vendor of one of the programs112, other tests, or combinations thereof.

The subroutines212and the tests214may be generated according to a template such as a script template. The template enables relatively simple and uniform development and implementation of the subroutines212and the tests214. The template may provide a framework for data requests, commands, and instructions.

In some embodiments, the subroutines212may be scripts or be written using APPLE SWIFT™ or another suitable programming language. As used in the present disclosure, the scripts may include computing instructions implemented by another program or a scripting engine. An example of one of the tests214is depicted inFIG.8.

In some embodiments, the subroutines212or some portion thereof may be signed. For instance, the subroutines may be signed with a digital certificate of a developer of the subroutine212or an administrator evaluating the product114. Accordingly, in these and other embodiments, the subroutines212and/or one or more of the tests214may be signed scripts or signed APPLE SWIFT codes. Inclusion of a signature on the subroutines212may reduce or eliminate a security threat that may be introduced by implementation of the diagnostic process200.

The subroutines212may also include remediations216A and216B (generally, remediation216or remediations216). In some embodiments, the remediations216may correspond to the tests214. For instance, the first test214A may correspond to a first remediation216A. The diagnostic process module210may implement the first remediation216in response to a result (e.g., result206) that includes an unexpected or an erroneous value that indicates a failure of the first test214A. Alternatively, in response to the results206indicating an expected value that indicates a successful application of the first test214A, the first remediation216A may not be implemented.

Accordingly, in an example embodiment, the first subroutine212A may be directed to the first program112of the product114. The first subroutine212A may include the first test214A and the second test214B. When the first subroutine212A is executed during evaluation of the product114, the first test214A may evaluate a first parameter or function of the first program112A and the second test214B may evaluate a second parameter or function of the first program112A. Responsive to the first test214A returning an expected or valid value, the first remediation216A may not be implemented. Responsive to the second test214B returning an unexpected or invalid value, the second remediation216B may be implemented.

The remediations216may be specific to an unexpected or invalid value included in results206of one of the tests214. For instance, the remediation216may be configured to modify the program112or a condition at the first endpoint118A to mitigate the unexpected result, which may enable the program112to pass the test214.

In some embodiments, the remediation216includes a change or modification to the product114or the endpoints118to address an unexpected or invalid result of one or more of the tests. For example, the remediation216may include reinstallation of one or more of the programs112, downloading or reinstallation of the agent208, requesting or updating a certificate of a user, requesting an updated certificate of a user, communication of a notification to a user or an administrator, increasing a time to complete a computing process at the endpoints118, installation of a patch or product update, disabling the program112or a portion thereof.

Execution of the subroutines212may generate the results206. The results206may be itemized or organized. For instance, the results206may be itemized or organized according to the subroutine212and/or the test214that generated the results206or to the program112being evaluated. For instance, a first portion of the results206may be itemized to correspond to the first subroutine212A and a second portion of the results206may be itemized to correspond to the second subroutine212B. Itemization may enable the assessment of portions of the program112at a parameter- or feature-level of granularity.

In addition, the subroutines212or the diagnostic process module210may be configured to output results data. The results data output by the subroutines212may be related to testing and evaluation of the programs112. In some embodiments, the results data output by the subroutines212may be derived from and convey the results206of the subroutines212. For instance, in response to the results206of the first subroutine212A or one of the tests214therein indicating that the first program112is installed and operational at the first endpoint118A, the results data output may include data indicating the first program112successfully passed the first subroutine212A. The results data output may include details of the test214and the expected result. Similarly, in response to the first subroutine212A or one of the tests214therein returning an unexpected result, the results data output may include data indicating the first program112or one of the tests214therein failed the first subroutine212A. The results data output may include details of the test214and the unexpected result.

The results206may be communicated to a user interface (UX)228. The results206may be displayed in the UX228. For instance, the data and the instructions output by the diagnostic process200may be displayed in the UX228that may be presented to a user of the first endpoint118A or to the administrative device106. Some examples of the data and instructions that might be displayed at the UX228may include a selectable instruction configured to initiate at least a portion of the remediation216, a first icon associated with the one of the programs112that indicates that the program112successfully passed one of the subroutines212or tests214, a second icon associated with one of the programs112that indicates that the data output by one of the subroutines212or tests214indicates that the program112failed the subroutine212, indications of one or more similar endpoints118, other information and data, or combinations thereof. An example of the UX228is provided inFIG.5.

In some embodiments, the administrative device106may be configured to collect device information202A and202B (generally, device information202). For instance, the administrative device106may collect a first device information202A from the first endpoint118A and a second device information202B from the second endpoint118B. In these and other embodiments, an endpoint management operation (e.g., a unified endpoint management (UEM), a mobile device management (MDM) product, or an endpoint management (EPM) product, etc.) may be implemented by the administrative device106to obtain the device information202. The device information may alternatively be supplied by another device such as a cloud system that collects the device information202or from an agent such as the agent208on the endpoints118.

The device information202may include a device type, a role of users (e.g., the users124) associated with the endpoints118, a security status of the users, use of the endpoints118, status of applications at the endpoints118, inclusion in a managed network of the endpoints118, update policies applied to the endpoints118, locations (geographic location or network location), network connectivity type; other parameters, or combinations thereof.

In response to a determination that the second endpoint118B is similar to the first endpoint118A, the administrative device106may further implement the remediation216at the second endpoint118B. The administrative device106may implement the remediation216without implementing the diagnostic process200at the second endpoint118B.

FIG.3illustrates an example computing system300configured for product diagnosis according to at least one embodiment of the present disclosure. The computing system300may be implemented in the operating environment100ofFIG.1, for instance. Examples of the computing system300may include the endpoints118or the administrative device106or some combination thereof. The computing system300may include one or more processors310, a memory312, a communication unit314, a user interface device316, and a data storage304that includes the agent208, the product114, and the product analysis module116(collectively, modules208/114/116).

The processor310may include any suitable special-purpose or general-purpose computer, computing entity, or processing device including various computer hardware or software modules and may be configured to execute instructions stored on any applicable computer-readable storage media. For example, the processor310may include a microprocessor, a microcontroller, a digital signal processor (DSP), an ASIC, an FPGA, or any other digital or analog circuitry configured to interpret and/or to execute program instructions and/or to process data. Although illustrated as a single processor inFIG.3, the processor310may more generally include any number of processors configured to perform individually or collectively any number of operations described in the present disclosure. Additionally, one or more of the processors310may be present on one or more different electronic devices or computing systems. In some embodiments, the processor310may interpret and/or execute program instructions and/or process data stored in the memory312, the data storage304, or the memory312and the data storage304. In some embodiments, the processor310may fetch program instructions from the data storage304and load the program instructions in the memory312. After the program instructions are loaded into the memory312, the processor310may execute the program instructions.

The memory312and the data storage304may include computer-readable storage media for carrying or having computer-executable instructions or data structures stored thereon. Such computer-readable storage media may include any available media that may be accessed by a general-purpose or special-purpose computer, such as the processor310. By way of example, and not limitation, such computer-readable storage media may include tangible or non-transitory computer-readable storage media including RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory devices (e.g., solid state memory devices), or any other storage medium which may be used to carry or store desired program code in the form of computer-executable instructions or data structures and that may be accessed by a general-purpose or special-purpose computer. Combinations of the above may also be included within the scope of computer-readable storage media. Computer-executable instructions may include, for example, instructions and data configured to cause the processor310to perform a certain operation or group of operations.

The communication unit314may include one or more pieces of hardware configured to receive and send communications. In some embodiments, the communication unit314may include one or more of an antenna, a wired port, and modulation/demodulation hardware, among other communication hardware devices. In particular, the communication unit314may be configured to receive a communication from outside the computing system300and to present the communication to the processor310or to send a communication from the processor310to another device or network (e.g.,122ofFIG.1).

The user interface device316may include one or more pieces of hardware configured to receive input from and/or provide output to a user. In some embodiments, the user interface device316may include one or more of a speaker, a microphone, a display, a keyboard, a touch screen, or a holographic projection, among other hardware devices.

The modules208/114/116may include program instructions stored in the data storage304. The processor310may be configured to load the modules208/114/116into the memory312and execute the modules208/114/116. Alternatively, the processor310may execute the modules208/114/116line-by-line from the data storage304without loading them into the memory312. When executing the modules208/114/116the processor310may be configured to perform a participation verification process as described elsewhere in this disclosure.

Modifications, additions, or omissions may be made to the computing system300without departing from the scope of the present disclosure. For example, in some embodiments, the computing system300may not include the user interface device316. In some embodiments, the different components of the computing system300may be physically separate and may be communicatively coupled via any suitable mechanism. For example, the data storage304may be part of a storage device that is separate from a server, which includes the processor310, the memory312, and the communication unit314, that is communicatively coupled to the storage device. The embodiments described herein may include the use of a special-purpose or general-purpose computer including various computer hardware or software modules, as discussed in greater detail below.

FIG.4is a flow diagram of a method400of evaluating computer-implemented product that is deployed on one or more endpoints according to at least one embodiment. The method400may be implemented in a manages network another suitable environment such as the operating environment100. The operating environment implementing the method400may be included in a cloud-based networked system, an on-premises system, a managed network, managed subnetwork, or another suitable network computing environment.

The method may begin at block402in which one or more programs are identified. For instance, a first program and a second program of a product deployed on a first endpoint of one or more endpoints may be identified. The first and the second programs may include an agent that is configured to operate on the first endpoint, or another program implemented on the first endpoint such as an UEM program, a patch program, a ITSM program, a EPM program, a cloud management program, combinations thereof or another suitable program.

At block404, a diagnostic process may be implemented. The diagnostic process may be implemented and a subset of the one or more endpoints or individually at the first endpoint. In some embodiments, a user of the first endpoint may initiate the diagnostic process. In some embodiments, a cloud management system or an administrative system may implement the diagnostic process remotely. Additionally, the diagnostic process may be implemented prior to operation or installation of an agent on the first endpoint.

The diagnostic process may include two or more subroutines. The subroutines may each be directed to a program that is included in the evaluated product. The diagnostic process may be organized similar to a bill of sale or list of programs with associated subroutines that test operation thereof. For instance, in some embodiments, the diagnostic process may include a first subroutine and a second subroutine of the two or more subroutines. The first subroutine may be directed to the first program and the second subroutine may be directed to a second program. The first and second subroutines each execute a set of installation and functional parameter tests of the first and second programs, respectively. For instance, the first subroutine includes a first set of installation and functional parameter tests and the second subroutine includes a second set of installation and functional parameter tests. The first set of installation and functional parameter tests may include some or all of the tests of the second set of installation and functional parameter tests.

In some embodiments the first set of installation and functional parameter tests and the second set of installation and functional parameter tests include are different for each program of the product. Some examples of the tests that may be included in the set of installation and functional parameter tests may include verification of locations of files and data of the first or the second programs, verification of a profile of a user that implemented the first or the second programs, identification of a time involved in carrying out a computing process in the first and the second programs, verification of a version of the first or the second programs, verification of enrollment of the first or the second programs with a second system, verification of a source of the first or the second programs, verification of a vendor of the first or the second programs, other suitable tests, or combinations thereof. In some embodiments the first subroutine and the second subroutine as well as other subroutines of the diagnostic process may be generated according to a template such as a script template. Additionally, in some embodiments the subroutines of the diagnostic process such as the first subroutine and the second subroutine may be signed by an entity or individual that developed the subroutine. For instance, the first subroutine and the second subroutine may be signed scripts.

At block406, results of the two or more subroutines may be itemized. For instance, results of the first subroutine and the second subroutine may be itemized or organized. The results may be itemized according to a corresponding program. For instance, results of the first subroutine may be itemized or organized such that they are associated with the first program. Also, results of the second subroutine may be itemized or organized such that they are associated with the second program.

At block408, it may be determined whether results of each of the two or more subroutines indicate that a corresponding program is installed and operational at the first endpoint. For instance, it may be determined whether the results of the first subroutine and the results of the second subroutine indicate that the first program and the second program are installed and operational at the first endpoint. In response to the results of the first subroutine indicating that the first program is installed and operational at the first endpoint, the method400may proceed to block418. In response to the results of the second subroutine indicating that the second program is not installed and operational at the first endpoint, the method400may proceed to block410. For instance, in response to a first test of the set of installation and functional parameter tests of the second subroutine returning an unexpected result, the method400may proceed to block410. At block410, data indicating the second program failed may be output. In some embodiments, the data output may include details of the first test and/or the unexpected result.

At block412, a remediation may be implemented. The remediation may be specific to the unexpected result and/or to the first test. The remediation may be configured to modify the second program or a condition at the first endpoint to mitigate or resolve the unexpected result. Additionally or alternatively, the remediation may enable the second program to pass the first test. For instance, a change to the first test might be implemented. Some additional examples of the remediation may include reinstallation of the second program, download or an additional download of an agent, request a certificate of a user or an updated certificate of a user, communication of a notification to an administrator and/or a user, increase in a time to complete a computing process or the first test, installation of a patch or a product update, removal of the second program, disabling one or more portions of the second program, other remedial actions or combinations thereof.

At block414it may be determined whether a second endpoint of the one or more endpoints is similar to the first endpoint. Similarity between endpoints may include device type, role of users associated with the first and the second endpoints, function of the first and the second endpoints, products loaded on the first and the second endpoints, entities managing the first and the second endpoints, endpoint characteristics, network configuration associated with the endpoint characteristics, other parameters, or combinations thereof. In response to the second endpoint being similar to the first endpoint (“YES” at block414), the method400may proceed to block416. In response to the second endpoint being dissimilar to the first endpoint (“NO” at block414), the method400may proceed to block420.

At block416, the remediation may be implemented. The remediation may be implemented at the second endpoint without implementing the diagnostic process at the second endpoint. For instance, the diagnostic process may be used to determine technical issues at the first endpoint. It may be assumed that the technical issue may also exist at other endpoints that are similar to the first endpoint. Accordingly, the remediation may be implemented on one or more of these similar endpoints. The second endpoint may be included in a managed network with the first endpoint or may be in another managed network.

At block418, data and instructions output by the diagnostic process may be caused to be displayed. For instance, the data and the instructions output by the diagnostic process may be caused to be displayed in a user interface that may be presented to the user of the first endpoint or to administrative device. Some examples of the data and instructions that might be caused to be displayed may include a selectable instruction configured to initiate at least a portion of the remediation in the user interface; a first icon associated with the first program that indicates that the first program successfully passed the first subroutine, a second icon associated with the second program that indicates that the data output by the second subroutine indicates that the second program failed the second subroutine, indications of one or more similar endpoints, other information and data, or combinations thereof. At block420, the method400may end.

Further, modifications, additions, or omissions may be made to the method400without departing from the scope of the present disclosure. For example, the operations of the method400may be implemented in differing order. Furthermore, the outlined operations and actions are only provided as examples, and some of the operations and actions may be optional, combined into fewer operations and actions, or expanded into additional operations and actions without detracting from the disclosed embodiments.

FIG.5depicts a block diagram of an example of the UX228. The UX228is labeled a diagnostic tool UX and the information presented in the UX228is depicted in response to an overview icon524being selected. The UX228may include a refresh icon502. The refresh icon502may initiate the diagnostic process or a portion thereof.

The UX228may also include a test field542. The test field542may include a list of programs516A and516B (generally, program516or programs516) as well as tests518A-518G (generally, test518or tests518) that are implemented with the programs516. The programs516may correspond and may be substantially similar to the programs112. With each of the programs516, pass/fail icons544may be displayed. The pass/fail icons544may indicate whether the test518passed or failed. InFIG.5only one of the pass/fail icons544is shown. Additionally, with each of the tests518, details icons506A-506E are displayed. Selection of the details icons506A-506E provide details of the test518and/or details of results data generated by the test518. In tests518that fail, the pass/fail icons544may be displayed differently and a remediate icon504may be displayed. For instance in a sixth test518F, a VulScan test failed. Accordingly, the pass/fail icon544of the sixth test518F is black (instead of white) and the remediate icon504is displayed. Selection of the remediate icon504may provide details related to a remediation that addresses the failed test518F or the program on which the test518F is implemented. Additionally or alternatively, selection of the remediate icon504may initiate a remediation process.

FIG.6depicts a block diagram of an example window600that may be displayed. For instance, the window600may be displayed on the endpoint118or administrative device106. The window600may be displayed responsive to selection to a details icon such as one of the details icon506ofFIG.5. The window may provide information and data related to one of the tests implemented in a diagnosis process such as the diagnostic process200ofFIG.2. For instance, the window600provides versions of the agent and platform at an endpoint. Additionally, status information related to an agent and console may be provided. The installed engines at the endpoint may be further provided in the window600.

FIG.7depicts an example of diagnostic status data700that may be generated during a diagnostic process such as the diagnostic process200ofFIG.2. The diagnostic status data700may include a series of operations that are implemented as one or more tests are executed. In the diagnostic status data, time of events are provided followed by a description of the operations. In the diagnostic status data700ofFIG.7, multiple connections are tested at particular IP addresses and a certificate is checked among other tests.

FIG.8is example programming code800that may be executed to perform one of the tests of a diagnostic process such as the diagnostic process200ofFIG.2. The programming code800illustrates an example code developed in APPLE SWIFT to diagnose MDM enrollment. The programming code800provides readouts provided responsive to failure and successful execution of portions of the programming code800.

FIG.9is a high-level file structure900that may be implemented in a diagnostic process such as the diagnostic process200ofFIG.2. As shown in the high-level file structure900the “Diagnostic Test” may include subroutines that are directed to programs such as EPM and Neurons. Within the subroutines, multiple tests are included. For instance, within the EPM subroutine, an MDMEnrollTest, an EPMVersionTest, a ProxyHostTest, and a VulscanTest are included. Execution of the EPM subroutine includes execution of each of the tests included therein. Additionally, the DiagnosticTest may include a “Results” folder. In the Results folder the results data output by the subroutines may be aggregated and itemized. Additionally still, the DiagnosticTests folder includes templates and kits to develop tests.

The embodiments described herein may include the use of a special purpose or general-purpose computer including various computer hardware or software modules, as discussed in greater detail below.

Embodiments described herein may be implemented using computer-readable media for carrying or having computer-executable instructions or data structures stored thereon. Such computer-readable media may be any available media that may be accessed by a general purpose or special purpose computer. By way of example, and not limitation, such computer-readable media may include non-transitory computer-readable storage media including Random Access Memory (RAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-Only Memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory devices (e.g., solid state memory devices), or any other storage medium which may be used to carry or store desired program code in the form of computer-executable instructions or data structures and which may be accessed by a general purpose or special purpose computer. Combinations of the above may also be included within the scope of computer-readable media.

Computer-executable instructions may include, for example, instructions and data, which cause a general-purpose computer, special purpose computer, or special purpose processing device (e.g., one or more processors) to perform a certain function or group of functions. Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

As used herein, the terms “module” or “component” may refer to specific hardware implementations configured to perform the operations of the module or component and/or software objects or software routines that may be stored on and/or executed by general purpose hardware (e.g., computer-readable media, processing devices, etc.) of the computing system. In some embodiments, the different components, modules, engines, and services described herein may be implemented as objects or processes that execute on the computing system (e.g., as separate threads). While some of the system and methods described herein are generally described as being implemented in software (stored on and/or executed by general purpose hardware), specific hardware implementations or a combination of software and specific hardware implementations are also possible and contemplated. In this description, a “computing entity” may be any computing system as previously defined herein, or any module or combination of modulates running on a computing system.

The various features illustrated in the drawings may not be drawn to scale. The illustrations presented in the present disclosure are not meant to be actual views of any particular apparatus (e.g., device, system, etc.) or method, but are representations employed to describe embodiments of the disclosure. Accordingly, the dimensions of the features may be expanded or reduced for clarity. In addition, some of the drawings may be simplified for clarity. Thus, the drawings may not depict all of the components of a given apparatus (e.g., device) or all operations of a particular method.

Terms used in the present disclosure and the claims (e.g., bodies of the appended claims) are intended as “open” terms (e.g., the term “including” should be interpreted as “including, but not limited to,” the term “having” should be interpreted as “having at least,” the term “includes” should be interpreted as “includes, but is not limited to,” among others). Additionally, if a specific number of an introduced claim recitation is intended, such an intent will be explicitly recited in the claim, and in the absence of such recitation no such intent is present. For example, as an aid to understanding, the following appended claims may contain usage of the introductory phrases “at least one” and “one or more” to introduce claim recitations.

In addition, even if a specific number of an introduced claim recitation is explicitly recited, those skilled in the art will recognize that such recitation should be interpreted to mean at least the recited number (e.g., the bare recitation of “two recitations,” without other modifiers, means at least two recitations, or two or more recitations). Furthermore, in instances in which a convention analogous to “at least one of A, B, and C, etc.” or “one or more of A, B, and C, etc.” is used, in general such a construction is intended to include A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B, and C together, etc. Further, any disjunctive word or phrase presenting two or more alternative terms should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms. For example, the phrase “A or B” should be understood to include the possibilities of “A” or “B” or “A and B.”

However, the use of such phrases should not be construed to imply that the introduction of a claim recitation by the indefinite articles “a” or “an” limits any particular claim containing such introduced claim recitation to embodiments containing only one such recitation, even when the same claim includes the introductory phrases “one or more” or “at least one” and indefinite articles such as “a” or “an” (e.g., “a” and/or “an” should be interpreted to mean “at least one” or “one or more”); the same holds true for the use of definite articles used to introduce claim recitations.

The terms “first,” “second,” “third,” etc., are not necessarily used to connote a specific order or number of elements. Generally, the terms “first,” “second,” “third,” etc., are used to distinguish between different elements as generic identifiers. Absence a showing that the terms “first,” “second,” “third,” etc., connote a specific order, these terms should not be understood to connote a specific order. Furthermore, absence a showing that the terms “first,” “second,” “third,” etc., connote a specific number of elements, these terms should not be understood to connote a specific number of elements. For example, a first widget may be described as having a first side and a second widget may be described as having a second side. The use of the term “second side” with respect to the second widget may be to distinguish such side of the second widget from the “first side” of the first widget and not to connote that the second widget has two sides.

All examples and conditional language recited herein are intended for pedagogical objects to aid the reader in understanding the invention and the concepts contributed by the inventor to furthering the art and are to be construed as being without limitation to such specifically recited examples and conditions. Although embodiments of the present inventions have been described in detail, it should be understood that the various changes, substitutions, and alterations could be made hereto without departing from the scope of the invention.