Outlier software component remediation

Outlier software component remediation is disclosed. For example, a dependency evaluation module executes on a processor to determine an input stack including references to a first plurality of components incorporated in the program and a plurality of metadata tags associated with the first plurality of components. A plurality of selection filters is applied to the plurality of reference project manifests to select a reference project manifest as a reference stack. The first plurality of components identified in the input stack is compared with a second plurality of components identified in the reference stack. A component in the input stack is selected as an outlier component based on the reference stack lacking a reference to the component.

BACKGROUND

The present disclosure generally relates to developing and executing computer programs in computer systems. For rapid development and deployment of software, developers may often seek to integrate existing code modules into new software programs. Therefore, software programs are often dependent on multiple integrated pre-existing software components. Many software programs are also subject to fluctuations in usage demands, requiring more instances during peak usage. To address fluctuating usage demands while maximizing the utilization of physical hardware, many computer systems employ virtualized guests such as virtual machines and containers to execute computing tasks performed by the computing systems, such as for hosting software programs. Typically, guests such as containers and virtual machines may be launched to provide extra compute capacity of a type that the guest is designed to provide, while isolating compute resources used by different users and tenants away from those of other users. Employing guests enables rapid scaling of applications to the volume of traffic requesting the services provided by those applications. In addition, guests typically allow for increased deployment flexibility since guests may typically be deployed in a variety of hardware environments. Multiple guests may also be clustered together to perform more complex functions than the respective guests are capable of performing individually. For software applications intended to be deployed in virtualized environments, development, debugging, and/or testing of those software programs may be conducted in similar virtualized environments to mimic the environments where the software programs will execute.

SUMMARY

The present disclosure provides a new and innovative system, methods and apparatus for outlier software component remediation. In an example, a dependency evaluation module executes on a processor to determine an input stack including references to a first plurality of components incorporated in the program. A plurality of metadata tags associated with the first plurality of components is determined. A plurality of selection filters is applied to the plurality of reference project manifests to select a reference project manifest as a reference stack. The first plurality of components identified in the input stack is compared with a second plurality of components identified in the reference stack. A component in the input stack is selected as an outlier component based on the reference stack lacking a reference to the component.

Additional features and advantages of the disclosed method and apparatus are described in, and will be apparent from, the following Detailed Description and the Figures.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

In many computer systems, physical hardware may host guests such as virtual machines and/or containers. In an example, a virtual machine (“VM”) may be a robust simulation of an actual physical computer system utilizing a hypervisor to allocate physical resources to the virtual machine. In sharing physical computing resources, guests and/or a hypervisor controlling them, may also have access to shared components of the underlying host, for example, I/O devices (e.g., network interface cards (“NICs”), storage controllers, USB controllers, PS2 interfaces, etc.). Such access is typically restricted through a virtualization manager such as a hypervisor to ensure that virtual environments remain segregated and to prevent unauthorized access to other virtual environments on the same host, or to the host itself. In some examples, a container based virtualization system, for example, one managed by a container manager such as Red Hat® OpenShift® executing a containerization runtime environment such as Docker® may be implemented. Typically, a container will be hosted on a physical host or virtual machine that already has an operating system executing, and the container may be hosted on the operating system of the physical host or VM without executing an independent operating system of its own. Containers may be advantageous by being typically faster to deploy than virtual machines with independent operating systems and by typically incurring less computing overhead to execute than full virtual machines.

Software applications developed in a variety of programming languages are executed on physical hosts or virtual hosts (e.g., VMs and containers). Instead of redeveloping functionality of existing software programs, programmers typically seek to integrate new programs with these existing programs to utilize the features of the existing programs. However, to integrate program functionality piecemeal, elevated access to the source program may be required, for example, access to the source program's source code. Therefore, entire modules of existing code are typically integrated into new programs together (e.g., through the integration of shared libraries). Many online repositories of such modules for integration exist, however, without careful vetting of individual modules, potentially requiring access and understanding of the underlying source code of the module itself, it is often difficult to distinguish between high quality modules regularly updated for efficiency and security, and low quality modules that perform inefficiently and/or introduce security risks. Doing such investigations may significantly erode the advantages of using existing code modules rather than redeveloping certain functionality.

Typically, programs are designed to interoperate with other programs. Sharing incorporated modules (e.g., libraries) with related programs is often advantageous in ensuring interoperability. However, it is difficult at design time to necessarily ascertain the universe of past, present, and future applications that may interface with a program currently under development. A developer may select a library to incorporate into their application based on a written description of the library's features, without necessarily having investigated the actual implementation of the features in the library. That library may be perfectly satisfactory for the developer's intended functional purposes in the program being build. However, even a library that is currently state of the art, without being updated, could soon become antiquated or even ready for deprecation in fast moving technology fields. Therefore maintaining the modules incorporated into a given program as continually suitable for the program over the course of time as the program and its dependent incorporated modules (e.g., libraries) go through iterative releases is a significant task in an application development cycle.

The present disclosure simplifies integrated software component remediation by detecting modules incorporated into a software program that may be outliers for replacement through enabling a debugger to dynamically and automatically analyze the modules incorporated into the software program to identify comparable program projects represented in a reference database with similar designed functionality. The debugger may then analyze the identified comparable program projects to determine whether any of the modules incorporated into the software program are disused and/or identified as inefficient or unsecure, and therefore recommended for elimination and/or replacement. In an example, a reference database may be constructed by retrieving metadata manifests of the modules incorporated into software source code projects available to the maintainer of the reference database, for example, projects published in a public source code repository (e.g., GitHub®), or projects stored in a proprietary source code management system. Correlation of functionality between a software program and the reference manifests in the reference database may be drawn based on shared incorporated modules. By applying multiple filters, a preferred candidate is identified as the reference manifest against which the software program's modules are compared. Differences in incorporated modules may be selected, with additional filters applied (e.g., frequency of use) to determine outlier module candidates. Analysis may be performed on the source code of the software program to determine whether the identified outlier modules are utilized in the source code, allowing the module to be removed if it is unused. Alternatively, if the module is referenced in the source code, an alternative module (e.g., a more updated module) may be identified as offering similar replacement functionality to replace the selected outlier module that may soon be deprecated, or is already deprecated. In an example, outlier software component selection enables automated detection and remediation of unnecessarily included software modules. In addition, outlier software component selection allows for higher security in new and updated applications by ensuring that up to date (and therefore most likely security patched) components are integrated into the software program. Incorporating higher quality and more updated modules may typically improve processing efficiency for the host node on which the program executes, resulting in more instances of the software program potentially being able to execute on the same hardware environment, thereby increasing hardware utilization and reducing idle time.

FIG. 1is a block diagram of a system for outlier software component remediation according to an example of the present disclosure. The system100may include one or more physical host(s)110. Physical host110may in turn include one or more physical processor(s) (e.g., CPU112) communicatively coupled to memory device(s) (e.g., MD114) and input/output device(s) (e.g., I/O116). As used herein, physical processor or processors112refer to devices capable of executing instructions encoding arithmetic, logical, and/or I/O operations. In one illustrative example, a processor may follow Von Neumann architectural model and may include an arithmetic logic unit (ALU), a control unit, and a plurality of registers. In an example, a processor may be a single core processor which is typically capable of executing one instruction at a time (or process a single pipeline of instructions), or a multi-core processor which may simultaneously execute multiple instructions. In another example, a processor may be implemented as a single integrated circuit, two or more integrated circuits, or may be a component of a multi-chip module (e.g., in which individual microprocessor dies are included in a single integrated circuit package and hence share a single socket). A processor may also be referred to as a central processing unit (“CPU”).

As discussed herein, memory device114refers to volatile or non-volatile memory devices, such as RAM, ROM, EEPROM, or any other device capable of storing data. As discussed herein, I/O device(s)116refer to devices capable of providing an interface between one or more processor pins and an external device, the operation of which is based on the processor inputting and/or outputting binary data. For example, a network interface card may be an example of an I/O device through which physical host110and guests (e.g., guest122) hosted on physical host110communicates with external systems over a network. CPU(s)112may be interconnected using a variety of techniques, ranging from a point-to-point processor interconnect, to a system area network, such as an Ethernet-based network. Local connections within physical host110, including the connections between processor112and a memory device114and between processor112and I/O device116may be provided by one or more local buses of suitable architecture, for example, peripheral component interconnect (PCI).

In an example, physical host110may host one or more guests (e.g., guest122). In an example guests may be VMs and/or containers, which may additionally host additional nested layers of guests (e.g., where application150hosted on guest122is another guest). For example, application150may be another virtual guest nested inside of a virtual machine or container guest122. In an example, a container as referred to herein may be implemented with any form of operating system level virtualization, for example, Red Hat® OpenShift®, Docker® containers, chroot, Linux®-VServer, FreeBSD® Jails, HP-UX® Containers (SRP), VMware ThinApp®, etc. Containers may run directly on a host operating system or run within another layer of virtualization, for example, in a virtual machine. In an example, containers that perform a unified function may be grouped together in a container cluster that may be deployed together (e.g., in a Kubernetes® pod). In an example, guest122may execute on physical host110(e.g., on host OS121of physical host110). In an example, guest122may execute on a virtual host (e.g., a VM) executing on physical host110. In addition, containers and/or VMs may further host other guests necessary to execute their configured roles (e.g., a nested hypervisor or nested containers). In an example, a guest (e.g., guest122) further host additional guests such as a Java® Virtual Machine (“JVM”), for example, if execution of Java® code is necessary.

System100may run a software layer (e.g., hypervisor120) above the hardware and below the guests (e.g., guest122), as schematically shown inFIG. 1. In an example, the hypervisor120may be a component of host operating system121, which executes on physical host110. In another example, the hypervisor120may be provided by an application running on host operating system121. In an example, hypervisor120may run directly on physical host110without an operating system beneath hypervisor120. Hypervisor120may virtualize the physical layer, including processors, memory, and I/O devices, and present this virtualization to guests (e.g., guest122) as devices, including virtual central processing unit (“VCPU”)190, virtual memory devices (“VIVID”)192, virtual input/output (“VI/O”) device194, and/or guest memory195. Processor virtualization may be implemented by the hypervisor120scheduling time slots on physical processors112such that from the guest operating system's perspective those time slots are scheduled on a virtual processor190.

Guest122may run on any type of dependent, independent, compatible, and/or incompatible applications on the underlying hardware and host operating system121. In an example, application150running on guest122may be dependent on the underlying hardware and/or host operating system121. In another example, application150running on guest122may be independent of the underlying hardware and/or host operating system121. In an example, application150running on guest122may be compatible with the underlying hardware and/or host operating system121. Additionally, application150running on guest122may be incompatible with the underlying hardware and/or OS. The hypervisor120may manage memory for the host operating system121as well as memory allocated to guest122and guest operating system196such as guest memory195provided to guest OS196. In an example, guest122may execute on host OS121, and may be dependent on host OS121for memory and/or processor management. In an example, code executing on guest122may be required to be compatible with guest OS196but not with host OS121. In an example, any form of suitable network for enabling communications between computing devices, for example, a public network (e.g., the Internet), a private network (e.g., a local area network (LAN) or wide area network (WAN)), or a combination thereof may be employed to connect physical host110and/or guest122to other computer systems.

In an example, application150may execute on guest122(e.g., inside of a debugger145), where application150is dependent on some of the features of components151A,152A,153A,154A,155A, and156A. For example, components151A-156A may be shared libraries integrated into application150to provide certain functionalities (e.g., networking protocols, structured input/output file format parsing, database SQL integrations, etc.). A debugger145generally refers to a debugging tool that supports the debugging of a particular set of programming languages. In an example, application150may be executing in a debugger145, which may include a dependency evaluation module140. In other examples, application150and/or dependency evaluation module140may execute outside of debugger145, with debugger145configured to receive the inputs and outputs of application150and/or dependency evaluation module140. For example, if a timeout occurs during the debugging of application150, the debugger145may identify an error in the execution of application150. In an example, communications among physical host110, guest122, and other network nodes, devices, and programs may be routed through the debugger145. In an example embodiment, the debugger145may be able to debug one or more programming languages (e.g., C, C-sharp, jBPM, Visual Basic, SQL, Pascal, COBOL, etc.). In an example, debugger145and/or dependency evaluation module140may be implemented in any suitable programming language.

In an example, network105may be any form of suitable network for allowing communications between computing devices, for example, a public network (e.g., the Internet), a private network (e.g., a local area network (LAN) or wide area network (WAN)), or a combination thereof. In some examples, the dependency evaluation module140is associated with a reference database160implemented in any suitable type of database, for example, a relational database. The reference database160may be associated with a database management system (DBMS). A DBMS is a software application that facilitates interaction between the database and other components of the nodes110A-C. For example, a DMBS may have an associated data definition language describing commands that may be executed to interact with the database. Examples of suitable DMBS's include MariaDB®, PostgreSQL®, SQLite®, Microsoft SQL Server® available from MICROSOFT® CORPORATION, various DBMS's available from ORACLE® CORPORATION, various DBMS's available from SAP® AG, IBM® DB2®, available from the INTERNATIONAL BUSINESS MACHINES CORPORATION, etc. In an example, the reference database160may be organized as a formal database with a schema such as a relational schema with defined tables, indices, links, triggers, various commands etc. In some examples, the storage component of reference database160may not be organized as a formal database, but may instead be an alternative storage structure capable of holding the information and being queried in relation to reference project manifests (e.g., reference project manifests162,164, and166) and their respective associated components (e.g., components151B,152B,153B,154B,157B,158B,151C,154C,155C,157C,151D,153D,154D,155D,158D) in system100, including but not limited to a file, folder, directory, registry, etc. In some examples, the microservice registry160may reside over a network (not depicted), which may be, for example, a public network (e.g., the Internet), a private network (e.g., a local area network (LAN) or wide area network (WAN)), or a combination thereof. In an example, rather than storing the entirety of components associated with reference project manifests162,164, and166, reference database160may store a limited amount of metadata (e.g., name, hash value, checksum, modification date, etc.) representing each of components151B,152B,153B,154B,157B,158B,151C,154C,155C,157C,151D,153D,154D,155D,158D for remediation purposes.

In an example, reference database160is populated from one or more source code repositories (e.g., source code repositories170,175) located across network105from reference database160. In the example, source code repository170may be a public database of open source software projects (e.g., GitHub®), while source code repository175may be a private source code repository operated by the same entity implementing reference database160implemented with any suitable version control system (e.g., Subversion®, Concurrent Versions System®, Microsoft® Visual SourceSafe®, etc.). In an example, source code projects (e.g., reference projects) may be public and/or proprietary source code projects. In the example, reference database160is configured to retrieve metadata associated with reference projects180,181,182,183,184A-B,185,186A-B,187,188, and189and their associated integrated components. For example, a set of selection criteria may be implemented for project selection. In the example, reference projects184A and186A may be copies of reference projects184B and186B, and the presence of reference projects184A-B and186A-B may be a selection criteria for inclusion (e.g., based on a project being located in multiple trusted source code repositories).

In the example, metadata reference project manifest164is created with component references (e.g., components151C,154C,155C, and157C) corresponding to the components of reference projects184A-B. In an example, only the metadata references to each component are retrieved, for example, to save storage space. In an example, reference project manifest164is a metadata file retrieved from source code repository160. In another example, reference project manifest164is a custom metadata file generated in reference database160based on individual elements retrieved from source code repository160, potentially with additional modifications, for example, to conform reference project manifests retrieved from different source code repositories (e.g., source code repositories170and175) to a common standard and schema to allow more efficient queries. In an example, reference project manifest164is constructed with a custom schema that may be indexed in reference database160for efficient querying. In some examples, a copy of components151C,154C,155C, and157C may also be retrieved and stored either in reference database160, or in a separate component database (e.g., so the components may be readily added to a project under development in conjunction with debugger145). In an example, reference project manifest166corresponds to reference projects186A-B. In an example, reference project manifest162corresponds to reference project182, which may be selected for inclusion in reference database160for a variety of reasons. For example, selection criteria may include access rate metrics, download metrics, quality voting metrics, recommendation metrics, and/or integration metrics (e.g., how often a project is linked/integrated to other projects) stored by source code repository170that are queried by reference database160. In an example, highly rated, often cited projects are likely to be kept up to date, including their respective integrated components, and therefore they often serve as reference projects that may be advantageously used to determine a current state of the art set of modules for integration.

FIGS. 2A-Dare block diagrams of a system performing outlier software component filtering according to an example of the present disclosure. Example system200depicted inFIG. 2Adepicts an application150with integrated components151A,152A,153A,154A,155A, and156A being input into dependency evaluation module140for dependent module analysis. In the example, a first filter is applied by dependency evaluation module140(e.g., component intersection filter240). In the example, component intersection filter240retrieves each reference project manifest (e.g., reference project manifests162,164, and166) in reference database160, and finds the intersections between the components referenced by each of reference project manifests162,164, and166and application150generating common component lists262,264, and266. In an example, common component list262corresponding to reference project manifest162indicates that four components are shared between application150and reference project manifest162(e.g., component251B corresponding with components151A-B, component252B corresponding with components152A-B, component253B corresponding with components153A-B, and component254B corresponding with components154A-B). Similarly, common component list264(e.g., components251C,254C, and255C), and common component list266(e.g., component251D,253D,254D, and255D) are generated corresponding to reference project manifests164and166respectively. In the example, because common component lists262and266each have four shared components with application150, while common component list264only has three shared components, reference project manifest164is eliminated from contention as a reference stack for analyzing application150's integrated components151A-156A as being outliers. In an example, a quantity of shared components may be indicative of a likelihood of common configured purpose between application150and reference projects182and186A-B. In some examples, a shared size of components may additionally be factored in rather than relying on counts (e.g., a large shared component may provide ten times as many common features as a small shared component). In an example, sharing a networking protocol library (e.g., component151A,151B,151D) may be indicative of a shared status as network accessible applications. The set of integrated modules in an application may be referred to as a component stack of the application. In an example, a machine learning model with a web API (e.g., application150) may have a stack with libraries such as {“Deeplearning4j”, “Vertx-core”, “Vertx-web”, etc.}. An alternative stack for a similar program (e.g., a reference stack from reference database160) may be {“Weka”, “Vertx-core”, “Vertx-web”, etc.}.

Example system201depicted inFIG. 2Bshows a second filter being applied to the results of the first filter in example system200. For example, reference project manifests162and166which have not been eliminated by the component intersection filter240may be entered into a tag correlation filter242. In an example, various components and/or reference projects may be tagged with a functional purpose. In an example, each group of tags220A-J,221A-F,222A-F,223A-B,224A-B,225A-C, and226A corresponds to a respective descriptor for a component (e.g., component151A,151B,151D;152A,152B;153A,153B,153D;154A,154B,154D;155A,155D;156A;157B; and158B,158D) associated with the respective tags. For example, in the case of the networking protocol library (e.g., component151A,151B,151D), tag220A, E, H may represent “networking” and tag222A, C, E may represent “encryption”. In the machine learning web API example, “Deeplearning4j” and “Weka” may both be tagged with [machine learning] and [deep learning], “Vertx-core” may be tagged with [vertx], [web], and [core], while “Vertx-web” may be tagged with [vertx], and [web]. In an example, tag correlation filter242compares each of the different tags represented in the components of application150, with each of the respective different tags represented in reference project manifests162and166to generate common tag lists272and276. In an example, only unique tags are considered, so a list of unique tags for each of application150(e.g., tags220A-D,221A-C,222A-B,223A,224A), reference project manifest162(e.g., tags220E-G,221D-E,222C-D;223B,225A-B,226A) and reference project manifest166(e.g., tags220H-K,221F,222E-F,224B,225B-C) may first be generated. In an example, common tags between application150and reference project manifest162as represented by common tag list272include tags220E′,221D′,222C′, and223B′, in which case the notation ′ denotes the first copy of a common tag encountered by dependency evaluation module140. In an example, reference project manifest166shares four common tags (e.g., tags220H′,221F′,222E′, and224B′ in common tag list276) with application150which is the same common tags shared between reference project manifest162and application150. In the example, since the shared count of common tags is the same, neither reference project manifest162nor reference project manifest166is eliminated as a potential reference stack for application150. In various examples, tags may be curated for greater matching efficiency (e.g., tags may be standardized to improve matches by avoiding typographical errors or different names for the same feature). In an example where common tag list272and common tag list276have different counts of shared tags, the reference project manifest with the higher count of shared tags is selected as a reference stack for application150.

Example system202depicted inFIG. 2Cillustrates a third filter (e.g., an occupancy filter244) being applied to the previously generated common component lists262and266for each of the remaining reference stack candidates. In the example, common component list264(e.g., components251C,254C, and255C), and common component list266(e.g., component251D,253D,254D, and255D) each contain four components shared between application150and reference project manifests162and166respectively. In an example, occupancy filter244takes common component lists262and266as inputs and compares them against total component count282(e.g., of reference project manifest162) and total component count286(e.g., of reference project manifest166) to calculate respective occupancy values of reference project manifests162and166. In an example, an occupancy value is calculated based on a ratio of common components to total components, whether by size, count, or weighted otherwise (e.g., by rating, count of functions within a given component, etc.). In illustrated example system202, occupancy value284is calculated based on four common components (e.g., components251B,252B,253B, and254B) out of six total components (e.g., components251B,252B,253B,254B,257B,258B) resulting in a formula of 4/6=⅔ or approximately 0.666. Similarly, occupancy value288is calculated based on four common components (e.g., components251D,252D,253D, and255D) out of five total components (e.g., components251D,252D,253D,255D,258D) resulting in a formula of ⅘ or approximately 0.8. In the example, reference project manifest166is selected as a reference stack for application150based on occupancy value288being higher than occupancy value284. In an example, instead of a count of components, an aggregate size of components may be implemented to calculate occupancy.

Illustrated system203depicted inFIG. 2Dshows how a given component is selected as an outlier component in application150based on a comparison with selected the previously selected reference stack (e.g., reference project manifest166). In an example, potential outlier selector246receives the component list (e.g., components151A-156A) of application160as an input stack and the component list (e.g., components151D,153D,154D,155D, and158D) of reference project manifest166as a reference stack. In the example, a potential outlier component list290is generated, including components154A′ (e.g., a reference to or copy of component154A) and156A′ (e.g., a reference to or copy of component156A) based on reference project manifest166lacking any reference to components corresponding to component154A or156A. In an example, potential outlier component list290is presented in its entirety as outlier modules for evaluation by debugger145. In another example, potential outlier component list290is further submitted as an input into a component frequency filter248to select for a subset of outlier modules to be presented as outlier components. For example, component frequency filter248calculates the prevalence of each module among all reference project manifests (e.g., reference project manifests162,164, and166) in reference database160. In the example, the number of occurrences of each distinct module is tabulated and ranked. For example, component151B,151C,151D=3 references; component152B=1 reference; component153B,153D=2 references; component154B,154C,154D=3 references; component155C,155D=2 references; component156A=0 references; component157B,157C =2 references; component158B,158D=2 references. In example system203, component152A′ (e.g., 1 references) is therefore more frequently included in the reference database160, than component156A′ (e.g., 0 reference), and therefore component frequency filter248selects component156A′ as the identified outlier component295in application150. In an example, remediation for outlier component156A′ may include identifying whether any function calls to component156A are actually invoked in application150to determine whether incorporating component156A may affect the functionality of application150. In an example, an alternative module may be recommended to replace component156A, for example, based on commonality of tags with tags221A and/or223A.

In various examples, applying component intersection filter240, tag correlation filter242, occupancy filter244, component frequency filter248and/or other filters for selecting a reference stack for application150out of the reference project manifests in reference database160in various differentiated orders may be advantageous. For example, component frequency filter248may be applied as a second filter after the component intersection filter240to select those reference project manifests with high component intersection counts that also share highly (or lowly) used components. In an example, matching more rarely used components may be indicative of a better functional match, for example, the more rarely used components may be more likely correlated with a designed purpose of application150, while more commonly used shared components may be indicative of popular modules incorporated in many different projects for generic purposes (e.g., networking protocols, XML libraries, database drivers, etc.). In such examples, matching low component frequency components may be a high value selection criterion for a reference stack. In an example, after each filter (e.g., component intersection filter240, tag correlation filter242, occupancy filter244, component frequency filter248, etc.) is applied, the remaining candidate reference project manifests that were not eliminated are retained as a temporary working set of reference project manifests, on which the next filter is applied until one reference project manifest remains as the selected reference stack for application150.

FIGS. 3A-Bare block diagrams of a system performing outlier software component selection according to an example of the present disclosure. Example system300depicted inFIG. 3Ais an alternative operation for selecting a reference stack for application150, where instead of submitting the working set of reference project manifests162and166from system200to tag correlation filter242as depicted in system201onFIG. 2B, the working set of reference project manifests162and166is instead submitted into a component frequency filter340. In the example, common component list262and common component list266have respective component frequency rankings332and336computed based on the relative frequency of occurrence of components251B,252B,253B,254B,251D,253D,254D, and255D in reference database160. In the example, components251B,252B,253B,254B,251D,253D,254D, and255D are references to components151B, C, D;152B;153B, D;154B, C, D;155C, D. In the example, component151B,151C,151D=3 references; component152B=1 reference; component153B,153D=2 references; component154B,154C,154D=3 references; component155C,155D=2 references. In an example, because component255D (e.g., referencing component155C, D) appears twice in reference database160while component252B (e.g., referencing component152B) appears only once in reference database160, reference project manifest162is selected as the closer match to application150. For example, a configured function of application150may be machine learning, while application150is also a web server. In the example, component252B (e.g., a reference to component152A and152B) may be a machine learning library that is less frequently incorporated than component255B which may be a web service component used in many web services not necessarily having anything to do with machine learning. In such cases, perhaps counter-intuitively, a negative correlation between frequency and quality of reference stack match may be observed.

Example system301depicted inFIG. 3Bshows the input stack of components151A-156A of application150being submitted into potential outlier selector342along with reference project manifest162. Potential outlier selector342outputs a list of components of application160that are not present in reference project manifest162(e.g., component155A′ representing to component155A, and component156A′ representing component156A). In the example, potential outlier component list390is sent into component frequency filter344, which may be similar to component frequency filter340, but configured with different frequency selection criteria. For example, component frequency filter340is configured to preferentially select for low frequency components to increase match quality, while component frequency filter344is configured to select for high frequency components to be eliminated as potential outliers. In the example, component155A′ is referenced twice in reference database160, while component156A′ is not referenced in reference database160, so component156A′ is selected as identified outlier component395. In an example, different selection criteria identified the same outlier component156A′ in example system203and in example system301. In some implementations, multiple combinations of automated outlier software component remediation tests may be executed on a given application150, and the results of the various tests may be combined to obtain one or more consensus outlier software modules.

FIG. 4is a flowchart illustrating an example of outlier software component filtering and outlier remediation according to an example of the present disclosure. Although the example method400is described with reference to the flowchart illustrated inFIG. 4, it will be appreciated that many other methods of performing the acts associated with the method400may be used. For example, the order of some of the blocks may be changed, certain blocks may be combined with other blocks, and some of the blocks described are optional. The method400may be performed by processing logic that may comprise hardware (circuitry, dedicated logic, etc.), software, or a combination of both. In an example, the method400is performed by a dependency evaluation module140.

Example method400may begin with determining an input stack including references to a first plurality of components incorporated in a program (block410). For example, application150being debugged in debugger145may have application150's list of incorporated modules (e.g., libraries) submitted to dependency evaluation module140, which list of modules may be interpreted as an input stack of components151A-156A incorporated in application150. A plurality of metadata tags associated with the first plurality of components is determined (block415). In an example, each component of components151A-156A may be additionally associated with one or more tags (e.g., tags220AD,221AC,222AB;223A;224A). In an example, the combination of the component manifest (e.g., references to components151A-156A) of application150with the list of tags (e.g., tags220AD;221AC;222A, B;223A;224A) associated with application150form a basis with which dependency evaluation module140assesses various reference project manifests for similarity in functionality with application150. Tags may be automatically computed, for example, based on comments in the source code of various components, or based on recognized external calls (e.g., API calls) performed by the various components. Tags may also be manually attributed to a given component. In an example, when a given project is submitted to a source code repository, predetermined tags may be made available for labeling the project and/or its components (e.g., in a drop down menu) to ensure uniform tag naming for ease of search and matching.

A plurality of selection filters is applied to a plurality of reference project manifests to select a first reference project manifest of the plurality of reference project manifests as a reference stack (block420). In an example, the plurality of reference project manifests is populated from a first source code repository storing a plurality of source code projects accessible over a network. For example, reference database160may be a selected subset of reference projects181-189from source code repositories170and175. In an example, source code repositories170and/or175gather and maintain various metrics associated with each of reference projects181-189including incorporation metrics (e.g., how often a project is incorporated into or referenced by other projects), review metrics (e.g., commentary and/or ratings regarding the quality of a given project), code complexity metrics (e.g., calculated metrics relating to the human readability and understandability of a given project), supportability metrics (e.g., metrics relating to how well the source code of a project is documented and/or commented, and how likely a change in the code will result in unintended consequences), other code quality metrics (e.g., extensibility metrics for how difficult it would be to add new functionality to the project), and/or dead code metrics (e.g., deprecated code branches that can't be reached during execution that have not been removed from the code) that are evaluated by reference database160before a given reference project is incorporated into reference database160as a reference project manifest (e.g., reference project manifests162,164,166). In some examples, projects from a public database (e.g., source code repository170) may be graded more stringently than projects from a developer's own organization (e.g., source code repository175). In an example, finding a match as a reference stack within the developer's own organization is more likely to result in a unity of functionality and design purpose. In addition, encouraging common module usage within an organization may improve the overall supportability of the organization's code base by ensuring that known issues may be resolved throughout the code base by updating individual dependencies rather than testing for whether a given issue exists in various competing integrated components, and then developing remedies for all of the various affected components.

In an example, a series of selection filters may be applied until one reference project manifest from reference database160is selected as a reference stack to which the input stack of application150is compared. For example, a combination of component intersection filter240, tag correlation filter242, occupancy filter244, component frequency filter248and/or other filters may be applied for selecting a reference stack for application150out of the reference project manifests in reference database160. In an example, after each filter is applied the remaining reference project manifests resulting from the filter are stored as a working set of reference project manifests that is updated after each subsequent filter is applied until only one reference project manifest remains (e.g., the reference stack). In an example, the component intersection filter240compares each of the plurality of reference project manifests (e.g., reference project manifests162,164, and166) in reference database160(or those reference project manifests still remaining in a working set if the component intersection filter240is not the first filter applied) with the input stack of components (e.g., components151A-156A) to select at least one selected reference project manifest with a highest count of shared components with components151A-156A (e.g., reference project manifests162and166with four shared components a piece), and the working set of reference project manifests is updated with the selected reference project manifests162and166. In an example, reference project manifests162and166become the inputs into the next selection filter.

In an example, a second selection filter is tag correlation filter242, which compares the working set of reference project manifests (e.g., reference project manifests162and166) with a plurality of metadata tags associated with components151A-156A (e.g., tags220AD,221AC,222AB,223A,224A) to select at least one selected reference project manifest with a highest count of shared metadata tags with components151A-156A (e.g., both reference project manifest162and166sharing 4 tags so neither is eliminated), and the working set of reference project manifests is updated with reference project manifests162and166, which become the inputs into the next selection filter. In an example, as illustrated in system201inFIG. 2B, tag correlation filter242may be implemented second, after component intersection filter240is used to narrow down the membership of reference database160to include only reference project manifests with a high degree of correlated components, so that tag correlation filter242may identify potential alternative components (e.g., modules and libraries) to those unmatched components of application150(e.g., component156A) by factoring in the tags of components in reference project manifests162and166that do not match any component in application150(e.g., components157B and158BD). In an example, if an outlier component is identified, a component from the reference stack sharing tags with the outlier component may be suggested as a substitute component to incorporate.

In an example, occupancy filter244may be applied to the working set resulting from applying component intersection filter240and/or tag correlation filter242. In an example, occupancy filter244may also be applied directly to the reference database160where reference database160is of limited size, as certain implementations of occupancy filter244may be natural extensions of component intersection filter240. In an example, occupancy filter244compares reference project manifests162,164, and/or166with an occupancy value of each respective reference project manifest, where the occupancy value is calculated based on a ratio of (i) shared components between a given reference project manifest and components151A-156A to (ii) a count of total components in the given reference project manifest to select at least one selected reference project manifest with a highest occupancy score, and the working set of reference project manifests is updated with the one or more selected reference project manifests. For example, reference project manifest162includes six component references while reference project manifest166includes only five component references, so where both match four of application150's components, reference project manifest166includes a higher ratio of matching components (e.g., a higher occupancy value) and therefore reference project manifest166may be selected as a reference stack for application150.

In an example, component frequency filter248may be implemented as an alternative filter to tag correlation filter242or occupancy filter244. In another example, component frequency filter248may be implemented as a selection mechanism for selecting a subset of components in a potential outlier component list290that includes all unmatched components between input stack (e.g., components of application150) and reference stack (e.g., reference project manifest166). In an example, component frequency filter248generates a ranking of each component represented reference database160by frequency of occurrence, and then filters reference project manifests162,164, and/or166based on the ranking of the components represented in each of reference project manifests162,164, and/or166. In an example, the ranking of component occurrence frequencies may be pre-calculated, for example, as part of a batch update sequence for importing new reference project manifests into reference database160. In an example, reference database160may only be updated periodically (e.g., monthly, quarterly, etc.) since new reference projects added to source code repositories170and/or175may require a period of time before the project quality metrics utilized by reference database160gather enough data on a given project to produce meaningful results. In an example, a project may be considered for inclusion in reference database160only after a given time has elapsed after submission. In some examples, popular projects garnering high usage statistics may bypass the waiting period for incorporation due to sufficiency of quality metric data.

In some exemplary embodiments, component intersection filter240, then tag correlation filter242, then occupancy filter244, and finally component frequency filter248is applied to the reference project manifests in reference database160until only one reference project manifest remains in order to select a reference stack. In an example, any combination of filters may be applied until a match is identified. The first plurality of components identified in the input stack is compared with a second plurality of components identified in the reference stack (block425). In the example, dependency evaluation module140(e.g., via potential outlier selector246), compares the input stack of application150components to the reference stack of reference project manifest166components to identify the components in the input stack that are absent in the reference stack. In an example, a component in the input stack is selected as an outlier component based on the reference stack lacking a reference to the component (block430). For example, components152A′ and156A′ are absent from reference project manifest166. In an example, both components152A′ and156A′ may be selected as outlier components. In another example, a limit may be configured for how many components in a given input stack may be identified as outlier based on the total count of components in the input stack. In such examples, various potential outlier components are evaluated for a likelihood of being a true outlier, for example, based on the application of component frequency filter248to the potential outlier component list290. In an example, component156A′ is selected as an identified outlier component295. In the example, component158D may be recommended as a potential replacement for component156A′ based on component158D being included in the selected reference stack. In an example, a copy of component158D may be stored in its entirety (rather than as a metadata entry in reference project manifest166). In the example, a component repository may be associated with debugger145allowing for immediate incorporation and testing of the recommended replacement component158D. In an example, the component repository stores respective copies of each component represented in the reference database160.

FIG. 5is a flowchart illustrating an example of outlier software component selection and remediation according to an example of the present disclosure. Although the example method500is described with reference to the flowchart illustrated inFIG. 5, it will be appreciated that many other methods of performing the acts associated with the method500may be used. For example, the order of some of the blocks may be changed, certain blocks may be combined with other blocks, and some of the blocks described are optional. The method500may be performed by processing logic that may comprise hardware (circuitry, dedicated logic, etc.), software, or a combination of both. In an example, the method500is performed by a dependency evaluation module140.

Example method500may begin with generating a ranking of each component represented in a plurality of reference project manifests by frequency of occurrence (block510). In an example, the plurality of reference project manifests is populated from a first source code repository storing a plurality of source code projects accessible over a network. For example, reference database160is populated from source code repositories170and/or175over network105. In the example, reference database160is regularly updated from source code repositories170and/or175, both for adding new reference project manifests as well as for removing deprecated reference project manifests (e.g., projects that have passed a threshold without update activity). In an example, reference projects (e.g., reference projects181-189) in source code repositories170and175are put through selection criteria by reference database160prior to a subset of reference projects181-189being selected for inclusion in reference database160(e.g., as reference project manifests162,164, and166). In an example, rather than storing entire software projects in reference database160, only metadata is stored in reference database160, for example, references (e.g., names, hash values, version numbers, etc.) to reference projects182,184A-B, and186A-B in the form of reference project manifests162,164, and166. In an example, rather than storing the contents of reference projects in reference database160, components151B,152B,153B,154B,157B,158B,151C,154C,155C,157C,151D,153D,154D,155D, and158D are stored as metadata references to those respective components.

An input stack that includes references to a first plurality of components incorporated in a program is determined (block515). For example, an input stack of application150into dependency evaluation module is determined to be a list of references to components151A-156A incorporated in application150. In an example, the frequency ranking may be calculated based on the frequency of occurrence of each component in reference database160. For example, component151B,151C,151D=3 references; component152B=1 reference; component153B,153D=2 references; component154B,154C,154D=3 references; component155C,155D=2 references; component156A=0 references; component157B,157C=2 references; component158B,158D=2 references. The first plurality of components is sorted based on the ranking (block520). In the example, the sorted order of components151A-156A would be151A (e.g. 3 references),154A (e.g. 3 references),153A (e.g. 2 references),155A (e.g. 2 references),152A (e.g. 1 reference), and finally156A (e.g. 0 references).

A plurality of component correlation manifests is generated from the first plurality of components (block525). For example, based on the input stack of references to components151A-156A, the combinatorics of possible matching subsets of the six components151A-156A may be generated to be compared with reference project manifests for an a matching reference stack, with the selection criteria weighted towards matching components with higher or lower frequencies of occurrence in reference database160. For example, each combination of two or more of components151A-156A may be generated (e.g.,151A and152A;151A,153A, and155A;151A,153A,154A, and155A, etc.). In an example, rather than generating and testing each possible combination, a heuristic may be used, for example, where any combinations that do not include component151A, which is the most frequently incorporated component in reference database160are ignored. In an example, instead of generating numerous failed matches, a selection filter may be applied first to limit the universe of potential reference stacks. For example, component intersection filter240may be implemented first before the plurality of component correlation manifests are generated, such that the generated component correlation manifests are all potential matches. In the example, by first applying component intersection filter240, a list of the reference project manifests matching a highest count of components with application150may be identified as a working set, and then a component frequency ranking score of each member of the working set (e.g., reference project manifests162and166) may be evaluated to determine a best matching reference stack for the input stack. In an example where an application being debugged has numerous components, a selection filter (e.g., component frequency filter340) may be applied to the input stack first, to identify a subset of components in the input stack to attempt to match to a reference project manifest. For example, the highest frequency, lowest frequency, or a combination of the highest and lowest frequency components may be selected, up to a maximum count of components to avoid introducing an un-scalably high potential number of combinations for matching. In an example, combinatorics based matching may be directly performed without first applying component intersection filter240, for example, by working in reverse starting with, for example, combinations of 2 components, and iteratively adding another component to the requested matching set until a limit is reached where no more components may be added and still yield a match. The components selected for testing matches may be selected based on frequency of occurrence. In an example, any type of filter may be applied as an initial selection criterion before applying component frequency filter340, including the component intersection filter240, tag correlation filter242, and/or occupancy filter244disclosed herein.

The plurality of component correlation manifests is compared with the plurality of reference project manifests to select the first reference project manifest as a reference stack (block530). In an example, each component correlation manifest is associated with a respective correlation value. For example, each component represented in reference database160has a calculated frequency ranking. Based on these calculated frequency rankings, each combination of components that results in a match has a calculated weighted correlation value, with higher or lower frequency being a primary factor in determining the weighting of a matching a particular component for correlation value calculation purposes. In an example, additional factors may be considered such as size of component, number of functions in the component, and last update time of the component. In an example, a match of three highly weighted components may result in a higher correlation value than a match of four lowly weighted components. The first plurality of components identified in the input stack is compared with a second plurality of components identified in the reference stack (block535). After one reference project manifest (e.g., reference project manifest162) is selected as the reference stack for the input stack of the components of application150, the input stack and reference stack are compared to identify components in the input stack missing from the reference stack. These components (e.g., components155A′ and156A′) are identified as potential outliers in a potential outlier component list390.

A first component in the input stack is selected as an outlier component based on the reference stack lacking a reference to the first component (block540). In an example, component156A′ is selected as an outlier component based on component156A′ being missing from reference project manifest162. In the example, because there is no analog to component156A′ in reference database160at all, it has no correlation value and no frequency ranking. In such a scenario, since component156A′ is never referenced in the curated reference database160, it is identified as an outlier component. In various scenarios, different factors may contribute towards selecting identified outlier components from potential outlier component list390. For example, tag based selection may be implemented where a component with a high correlation value or frequency score is identified as an outlier based on being missing from the reference stack. For example, component155A′ shares tag220C with components151A,154A, and156A. Since an analogue to component155A is missing from reference project manifest162, even though component155A′ appears in both other reference project manifests164and166, it may be identified as an outlier based on potentially being redundant in light of the other components of application150. In an example, a maximum and/or minimum count of outlier components (e.g., outlier thresholds) may be configured to avoid an overabundance of results for applications with many components or a lack of results for applications with few components. In projects with many components (e.g., 25 components) but with a highest matched count of components being much lower (e.g., 10 components), rather than identifying 15 outliers, component frequency filter344may be applied to select components in the identified potential outlier component list that are more likely to be outliers than the others (e.g., based on lower frequency of use in reference database160). A maximum outlier threshold therefore protects against over-identifying outlier modules based on an input stack having a higher than average count of integrated components or modules. Similarly, if every component in an application is matched by a reference stack, analysis may be performed to identify potentially redundant components (e.g., based on tag correlation filter242). A minimum outlier threshold therefore protects against failing to identify any outlier modules where, for example, an input stack has a lower than average count of integrated components or modules. In some examples where every component is matched by a reference stack, additional modules from the reference stack may be recommended for inclusion. In an example, having a reference stack that matches every component in the input stack may be indicative of the source application150having custom, bespoke implementations of industry standard features, which may be potential security risks or inefficient code that may be more easily supported and execute more efficiently if replaced with a commonly used external library that is regularly updated. In an example, in addition to identifying outlier components, dependency evaluation module140may additionally recommend replacement modules based on a reference stack. For example, a substitute component with similar tags may be recommended as a replacement for an outlier component. In an example, a component repository stores each component referenced by reference database160to allow debugger145to quickly retrieve a recommended component for integration.

FIG. 6is flow diagram of an example of outlier software component filtering with automated outlier remediation according to an example of the present disclosure. Although the examples below are described with reference to the flowchart illustrated inFIG. 6, it will be appreciated that many other methods of performing the acts associated withFIG. 6may be used. For example, the order of some of the blocks may be changed, certain blocks may be combined with other blocks, and some of the blocks described are optional. The methods may be performed by processing logic that may comprise hardware (circuitry, dedicated logic, etc.), software, or a combination of both. In illustrated example600, a debugger145requests dependency evaluation module140to perform outlier software component remediation with reference data stored in reference database160to identify and remediate outlier code.

Example method600may begin with debugger145requesting evaluation of the integrated components151A-156A of application150executing in debugger145for potential outlier components that may require remediation (block610). In the example, dependency evaluation module140generates an input stack representing the components151A-156A of application150from the request (block612). Dependency evaluation module140then queries reference database160with the components of the input stack to retrieve reference project manifests sharing common components with the input stack (block614). In the example, reference database compiles common component lists for each potential reference stack, for example, each reference project manifest with over a certain threshold of common components with application150(block616). In some examples, a reference project manifest with a lesser count of common components may be selected as a reference stack if, for example, the matched components are more significant components (e.g., larger size, more function calls) or the matched components are more closely related to the design function of application150(e.g., based on metadata tags or lower incorporation frequency in reference database160). In a straight forward example such as the one illustrated in system200onFIG. 2A, common component lists262,264, and266are generated based on all three of reference project manifests162,164, and166being within one component match of the maximum (e.g., 4 component matches by reference project manifests162and166). In the example, without further distinguishing characteristics (e.g., none of the matched components for reference project manifest164are unique to reference project manifest164), reference project manifest164is eliminated from contention as a reference stack for application150. In the example, reference project manifests162and166with 4 matching components each are selected as a working set based on their highest count of matched components (block618).

In the example, dependency evaluation module140applies a second filter with the goal of narrowing down the potential reference stacks for application150down to one selection. To do so, dependency evaluation module140queries reference database160for the metadata tag information related to reference project manifests162and166, to determine whether reference project manifests162and166may be distinguished based on a comparison of their metadata descriptor tags with those of components151A-156A of application150(block620). In an example, reference database160responds by providing a count of shared tags for each reference project manifest in the selected subset working set (e.g., reference project manifests162and166) (block622). In an example, dependency evaluation module140selects the reference project manifests with the highest count of shared tags (block624). For example, as illustrated system201onFIG. 2B, both reference project manifests162and166have common tag lists of four tags apiece (e.g., common tag lists272and276), and therefore neither is eliminated in this case by the tag correlation filter242as a potential reference stack for application150. In an example, occupancy filter244is next applied to calculate occupancy values284and288of reference project manifests162and166(block626). In an example, occupancy values284and288are values indicative of the ratio of matching components to total components in a given reference stack candidate (e.g., reference project manifests162and166). In an example, an occupancy value may be computed based on a simple count of shared components to total components in a given reference project manifest. Occupancy values may also be calculated factoring in weighting criteria beyond counting references. For example, the size of a component, metadata tags of a component, function calls of a component (both total available function calls and invocations of those function calls within a source code project), ratings a component, last update of a component, etc. may all be factored into an occupancy value to aid in computing a functional match between a reference project manifest and application150. In an example, occupancy calculations may be more expensive than correlation calculations in terms of time and CPU cycles, so occupancy filters factoring in more inputs may be reserved for filtering after other filters have narrowed down the choices for a reference stack to a few final candidates.

In an example, dependency evaluation module140selects reference project manifest166as a reference stack for application150based on reference project manifest166having a higher occupancy value288than reference project manifest162's occupancy value284(block628). In an example, dependency evaluation module140reports that component156A is an outlier module based on reference project manifest166lacking any reference corresponding to component156A (block630). In some examples, component152A may also be identified as an outlier module. In an example, component156A may be identified as a more likely outlier than component152A due to reference database160completely lacking any reference to component156A, and therefore component156A having a nonexistent frequency of incorporation within the reference project manifests162,164, and166of reference database160.

Debugger145receives the report that component156A is identified as an outlier component, and requests analysis on where component156A is used in the source code of application150(block632). In an example, dependency evaluation module140analyzes application150finding that component156A is not actually functionally used by application150(block634). For example, component156A may never be invoked (e.g., no function calls are made) by application150. In another example, component156A may be referenced in the source code of application150, but the reference may be in a logical branch that is deprecated (e.g., the code cannot actually go down that logical path in execution). In an example, debugger145removes outlier component156A from the source code of application150in response to the finding that component156A is unused (block636). In the example, debugger145requests application150to be recompiled as an updated version without the outlier component156A (block638).

FIG. 7is flow diagram of an example of component frequency based component selection outlier software component according to an example of the present disclosure.FIG. 7is flow diagram of an example of dynamic integration of command line utilities according to an example of the present disclosure. Although the examples below are described with reference to the flowchart illustrated inFIG. 7, it will be appreciated that many other methods of performing the acts associated withFIG. 7may be used. For example, the order of some of the blocks may be changed, certain blocks may be combined with other blocks, and some of the blocks described are optional. The methods may be performed by processing logic that may comprise hardware (circuitry, dedicated logic, etc.), software, or a combination of both. In illustrated example700a debugger145requests dependency evaluation module140to perform outlier software component remediation with frequency based component selection using reference data stored in reference database160to identify outlier code.

Example method700may begin with reference database160executing a periodic update to import new reference project manifests from source code repository170(block710). In the example, reference database160recalculates the rankings of all of the components in the reference project manifests stored in reference database160by frequency of representation after importing new reference project manifests (block712). In various examples, updates of reference database160may be triggered by different circumstances. For example, a timer may be employed, or a counter for new projects in a given source code repository. In an example, different source code repositories may be on different update cycles. For example, where source code repository170is a public database, updates may be processed only monthly or quarterly. However, with an internal trusted database (e.g., source code repository175operated by the same entity as reference database160), updates may be nightly or live.

In an example, debugger145executes an application150, and sends a request for the evaluation of components151A-156A of application150to dependency evaluation module140(block720). In the example, dependency evaluation module140generates an input stack of component references from the request from debugger145(block722). Dependency evaluation module140then queries reference database160with the input stack to discover reference project manifests with common components (block724). In an example, dependency evaluation module140in conjunction with reference database160(e.g., by way of a query) compiles a common component list (e.g., common component lists262,264, and266) for each potential reference stack out of the reference project manifests162,164, and166stored in reference database160(block726). In an example, reference database160selects reference project manifests162and166as a working set of manifests with a highest count (e.g., four) of matched components when compared with the input stack (block728). In an example, dependency evaluation module140receives the common component lists262and266, and subsequently queries reference database160for the current component frequency ranking data that was recently updated by reference database160(block730). In the example, reference database160provides the updated component frequency ranking data (block732).

Dependency evaluation module140, upon receiving the component frequency ranking data, calculates a component usage frequency ranking for each of the selected common component lists262and266(block734). In the example, dependency evaluation module140selects the reference project manifest from the working set that has the lowest component usage frequency ranking as a reference stack (block736). For example, reference project manifest162has shares a lesser used component (e.g., component152B) with application150than component155D shared by reference project manifest166. In the example, component152B is more likely to be a specialized functional component relating to application150's designed functionality than a more broadly shared component, which is more likely to be for a generic programming purpose (e.g., networking, data storage, etc.). In an example, dependency evaluation module140ranks the components151A-156A of the input stack for application150by usage frequency (block738). In the example, component156A which is not used by any of the reference project manifests in reference database160is selected as an outlier component based on its lowest usage frequency (block740). In the example, debugger145requests analysis of outlier component156A to determine if any form of automated remediation is possible (block742). For example, if component156A is tagged with certain tags shared by another component, that other component may be recommended as a replacement component. If component156A is not utilized by application150, component156A may be dereferenced and removed. In addition, places in the source code of application150that reference functions of component156A may be highlighted and reported.

FIG. 8is a block diagram of an example system implementing outlier software component filtering according to an example of the present disclosure. Example system800includes source code repository870storing source code projects881,882, and883, where source code repository870is accessible over a network805by reference database860that stores reference project manifests851and852populated from source code repository870. Host810has processor812and memory814storing a program850, with dependency evaluation module840executing on processor812to determine an input stack855with reference856to component851and reference857to component852respectively incorporated into program850. Metadata tags871and872associated with components851and852are determined. Selection filters891and892are applied to reference project manifests862and864to select reference project manifest862as reference stack865. Components851and852identified by reference856and857in input stack855are compared with components885B and886B identified in reference stack865. Component851is selected as outlier component895based on reference stack865lacking a reference to component851.

FIG. 9is a block diagram of an example system implementing outlier software component selection according to an example of the present disclosure. Example system900includes source code repository970storing source code projects981,982, and983, where source code repository970is accessible over a network905by reference database960that stores reference project manifests951and952populated from source code repository970. Host910has processor912and memory914storing a program950, with dependency evaluation module940executing on processor912to generate ranking930of components995A,996A,997, and998represented in reference project manifests962and964by frequency of occurrence. An input stack955is determined, with reference956to component951and reference957to component952respectively incorporated into program950. Components951and952are sorted based on ranking930. Component correlation manifests971and972are generated from components951and952. Component correlation manifests971and972are compared with reference project manifests962and964to select reference project manifest962as reference stack965, where each of component correlation manifests971and972is associated with a respective correlation value (e.g., correlation value975and976). Components951and952identified by reference956and957in input stack955are compared with components985B and986B identified in reference stack965. Component951is selected as outlier component995based on reference stack965lacking a reference to component951.

Outlier software component remediation enables more efficient identification and remediation of potential problem components in software projects by dynamically discovering known and trusted reference software projects that are closely related in functionality, and through comparative analysis, discovering whether a given integrated component is an outlier among other related components. Outliers exhibit behaviors that often cause maintenance, compatibility, and security issues. For example, software code that heavily employs outliers may be difficult for other software programs to integrate with due to using non-standard interfaces. In addition, outlier modules may be lacking in security and efficiency updates. Increased difficulty in integrating with external applications and decreased security often result in increased maintenance required to keep the application running, which results in increased downtime and therefore lower computing utilization. By employing outlier software component remediation as disclosed herein, computing utilization and efficiency is therefore increased while increasing the security of the computing environment.

Aspects of the subject matter described herein may be useful alone or in combination with one or more other aspects described herein. In a 1st exemplary aspect of the present disclosure, a system comprises: a first source code repository storing a plurality of source code projects accessible over a network; a reference database storing a plurality of reference project manifests populated from the first source code repository, including a first reference project manifest; a host with a processor and a memory, wherein the memory stores a program; and a dependency evaluation module executing on the processor to: determine an input stack including references to a first plurality of components incorporated in the program; determine a first plurality of metadata tags associated with the first plurality of components; apply a plurality of selection filters to the plurality of reference project manifests to select the first reference project manifest as a reference stack; compare the first plurality of components identified in the input stack with a second plurality of components identified in the reference stack; and select a first component in the input stack as an outlier component based on the reference stack lacking a reference to the first component.

In accordance with a 2nd exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 1st aspect), wherein a second component in the reference stack is selected as a replacement component for the first component. In accordance with a 3rd exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 1st aspect), wherein a working set of reference project manifests is generated from the plurality of reference project manifests based on applying a first selection filter of the plurality of selection filters, and the working set of reference project manifests is updated after applying each subsequent selection filter of the plurality of selection filters. In accordance with a 4th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 3rd aspect), wherein a component intersection filter compares one of the plurality of reference project manifests and the working set of reference project manifests with the first plurality of components identified in the input stack to select at least one selected reference project manifest with a highest count of shared components with the first plurality of components, and the working set of reference project manifests is updated with the at least one selected reference project manifest. In accordance with a 5th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 3rd aspect), wherein a tag correlation filter compares one of the plurality of reference project manifests and the working set of reference project manifests with the first plurality of metadata tags to select at least one selected reference project manifest with a highest count of shared metadata tags with the first plurality of components, and the working set of reference project manifests is updated with the at least one selected reference project manifest. In accordance with a 6th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 3rd aspect), wherein an occupancy filter compares one of the plurality of reference project manifests and the working set of reference project manifests with an occupancy value of each respective reference project manifest, wherein the occupancy value is calculated based on a ratio of (i) shared components between a given reference project manifest and the first plurality of components to (ii) a count of total components in the given reference project manifest to select at least one selected reference project manifest with a highest occupancy score, and the working set of reference project manifests is updated with the at least one selected reference project manifest. In accordance with a 47th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 3rd aspect), wherein a component frequency filter generates a ranking of each component represented in the plurality of reference project manifests by frequency of occurrence in the plurality of reference project manifests, and filtering one of the plurality of reference project manifests and the working set of reference projects based on the ranking of the components represented in each respective reference project manifest. In accordance with an 8th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 3rd aspect), wherein applying the plurality of selection filters includes first applying a component intersection filter, then applying a tag correlation filter to a results of the component intersection filter, then applying an occupancy filter to a results of the tag correlation filter, and finally applying a component frequency filter to a results of the occupancy filter.

In accordance with a 9th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 1st aspect), wherein a maximum count of outlier components is configured based on a total count of components in the first plurality of components. In accordance with a 10th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 1st aspect), wherein the plurality of reference project manifests is populated from the first source code repository based on evaluating at least one of incorporation metrics, review metrics, code complexity metrics, supportability metrics, code quality metrics, and dead code metrics of the plurality of source code projects in the first source code repository. In accordance with an 11th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 1st aspect), wherein the plurality of reference project manifests is populated from a plurality of source code repositories including the first source code repository. In accordance with a 12th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 1st aspect), wherein a component repository stores respective copies of each component represented in the reference database.

Aspects of the subject matter described herein may be useful alone or in combination with one or more other aspects described herein. In a 13th exemplary aspect of the present disclosure, a method comprises: determining an input stack including references to a first plurality of components incorporated in a program; determining a first plurality of metadata tags associated with the first plurality of components; applying a plurality of selection filters to a plurality of reference project manifests to select a first reference project manifest of the plurality of reference project manifests as a reference stack, wherein the plurality of reference project manifests is populated from a first source code repository storing a plurality of source code projects accessible over a network; comparing the first plurality of components identified in the input stack with a second plurality of components identified in the reference stack; and selecting a first component in the input stack as an outlier component based on the reference stack lacking a reference to the first component.

Aspects of the subject matter described herein may be useful alone or in combination with one or more other aspects described herein. In a 14th exemplary aspect of the present disclosure, a system comprises: a means for determining an input stack including references to a first plurality of components incorporated in a program; a means for determining a first plurality of metadata tags associated with the first plurality of components; a means for applying a plurality of selection filters to a plurality of reference project manifests to select a first reference project manifest of the plurality of reference project manifests as a reference stack, wherein the plurality of reference project manifests is populated from a first source code repository storing a plurality of source code projects accessible over a network; a means for comparing the first plurality of components identified in the input stack with a second plurality of components identified in the reference stack; and a means for selecting a first component in the input stack as an outlier component based on the reference stack lacking a reference to the first component.

Aspects of the subject matter described herein may be useful alone or in combination with one or more other aspects described herein. In a 15th exemplary aspect of the present disclosure, a computer-readable non-transitory storage medium storing executable instructions, which when executed by a computer system, cause the computer system to: determine an input stack including references to a first plurality of components incorporated in a program; determine a first plurality of metadata tags associated with the first plurality of components; apply a plurality of selection filters to a plurality of reference project manifests to select a first reference project manifest of the plurality of reference project manifests as a reference stack, wherein the plurality of reference project manifests is populated from a first source code repository storing a plurality of source code projects accessible over a network; compare the first plurality of components identified in the input stack with a second plurality of components identified in the reference stack; and select a first component in the input stack as an outlier component based on the reference stack lacking a reference to the first component.

In accordance with a 16th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 13th, 14th, or 15th aspects), wherein a second component in the reference stack is selected as a replacement component for the first component. In accordance with a 17th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 13th, 14th, or 15th aspects), further comprises: generating a working set of reference project manifests from the plurality of reference project manifests based on applying a first selection filter of the plurality of selection filters; and updating the working set of reference project manifests after applying each subsequent selection filter of the plurality of selection filters. In accordance with an 18th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 17th aspect), wherein a component intersection filter compares one of the plurality of reference project manifests and the working set of reference project manifests with the first plurality of components identified in the input stack to select at least one selected reference project manifest with a highest count of shared components with the first plurality of components, and the working set of reference project manifests is updated with the at least one selected reference project manifest. In accordance with a 19th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 17th aspect), wherein a tag correlation filter compares one of the plurality of reference project manifests and the working set of reference project manifests with the first plurality of metadata tags to select at least one selected reference project manifest with a highest count of shared metadata tags with the first plurality of components, and the working set of reference project manifests is updated with the at least one selected reference project manifest. In accordance with a 20h exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 17th aspect), wherein an occupancy filter compares one of the plurality of reference project manifests and the working set of reference project manifests with an occupancy value of each respective reference project manifest, wherein the occupancy value is calculated based on a ratio of (i) shared components between a given reference project manifest and the first plurality of components to (ii) a count of total components in the given reference project manifest to select at least one selected reference project manifest with a highest occupancy score, and the working set of reference project manifests is updated with the at least one selected reference project manifest. In accordance with a 21st exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 17th aspect), wherein a component frequency filter generates a ranking of each component represented in the plurality of reference project manifests by frequency of occurrence in the plurality of reference project manifests, and filtering one of the plurality of reference project manifests and the working set of reference projects based on the ranking of the components represented in each respective reference project manifest. In accordance with a 22nd exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 17th aspect), further comprises: first applying a component intersection filter; then applying a tag correlation filter to a results of the component intersection filter; then applying an occupancy filter to a results of the tag correlation filter; and finally applying a component frequency filter to a results of the occupancy filter.

In accordance with a 23rd exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 13th, 14th, or 15th aspects), further comprises: configuring a maximum count of outlier components based on a total count of components in the first plurality of components. In accordance with a 24th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 13th, 14th, or 15th aspects), further comprises: populating the plurality of reference project manifests from the first source code repository based on evaluating at least one of incorporation metrics, review metrics, code complexity metrics, supportability metrics, code quality metrics, and dead code metrics of the plurality of source code projects in the first source code repository. In accordance with a 25th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 13th, 14th, or 15th aspects), further comprises: populating the plurality of reference project manifests from a plurality of source code repositories including the first source code repository. In accordance with a 26th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 13th, 14th, or 15th aspects), further comprises: storing respective copies of each component represented in the plurality of reference project manifests in a component repository.

Aspects of the subject matter described herein may be useful alone or in combination with one or more other aspects described herein. In a 27th exemplary aspect of the present disclosure, a system comprises: a first source code repository storing a plurality of source code projects accessible over a network; a reference database storing a plurality of reference project manifests populated from the first source code repository, including a first reference project manifest; a host with a processor and a memory, wherein the memory stores a program; and a dependency evaluation module executing on the processor to: generate a ranking of each component represented in the plurality of reference project manifests by frequency of occurrence; determine an input stack including references to a first plurality of components incorporated in the program; sort the first plurality of components based on the ranking; generate a plurality of component correlation manifests from the first plurality of components; compare the plurality of component correlation manifests with the plurality of reference project manifests to select the first reference project manifest as a reference stack, wherein each component correlation manifest is associated with a respective correlation value; compare the first plurality of components identified in the input stack with a second plurality of components identified in the reference stack; and select a first component in the input stack as an outlier component based on the reference stack lacking a reference to the first component.

In accordance with a 28th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 27th aspect), wherein a selection filter is applied to at least one of the first plurality of components and the plurality of reference project manifests. In accordance with a 29th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 28th aspect), wherein the selection filter is one of (i) a component intersection filter, (ii) a tag correlation filter, (iii) an occupancy filter, and (iv) a component frequency filter. In accordance with a 30th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 29th aspect), wherein the component intersection filter compares the plurality of reference project manifests with the first plurality of components identified in the input stack to select a plurality of selected reference project manifests based on respective counts of shared components with the first plurality of components. In accordance with a 31st exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 29th aspect), wherein the occupancy filter calculates respective occupancy values of reference project manifests, wherein an occupancy value includes a ratio of (i) shared components between the respective reference project manifest and the first plurality of components to (ii) a count of total components in the respective reference project manifest, and wherein the occupancy filter selects at least one selected reference project manifest with a highest occupancy score. In accordance with a 32nd exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 29th aspect), wherein the component frequency filter generates a ranking of each component represented in the plurality of reference project manifests by frequency of occurrence in the plurality of reference project manifests, and ranks a set of reference project manifests based on the ranking of the components represented in each respective reference project manifest of the set of reference project manifests. In accordance with a 33rd exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 29th aspect), wherein the component intersection filter is applied to the plurality of reference project manifests prior to generating the plurality of component correlation manifests, each of the plurality of component correlation manifests corresponds to a subset of the first plurality of components, and each of the plurality of component correlation manifests matches at least one reference project manifest of the plurality of reference project manifests. In accordance with a 34th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 33rd aspect), wherein a respective usage frequency grade is calculated for each of the plurality of component correlation manifests, and the first reference project manifest is selected as the reference stack based on a usage frequency grade associated with a component correlation manifest corresponding to the first reference project manifest.

In accordance with a 35th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 27th aspect), wherein at least one of a maximum count of outlier components and a minimum count of outlier components is configured based on a total count of components in the first plurality of components. In accordance with a 36th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 27th aspect), wherein the plurality of reference project manifests is populated from the first source code repository based on evaluating at least one of incorporation metrics, review metrics, code complexity metrics, supportability metrics, code quality metrics, and dead code metrics of the plurality of source code projects in the first source code repository. In accordance with a 37th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 27th aspect), wherein the plurality of reference project manifests is populated from a plurality of source code repositories including the first source code repository. In accordance with a 38th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 27th aspect), wherein a component repository stores respective copies of each component represented in the reference database.

Aspects of the subject matter described herein may be useful alone or in combination with one or more other aspects described herein. In a 39th exemplary aspect of the present disclosure, a method comprises: generating a ranking of each component represented in a plurality of reference project manifests by frequency of occurrence, wherein the plurality of reference project manifests is populated from a first source code repository storing a plurality of source code projects accessible over a network; determining an input stack including references to a first plurality of components incorporated in a program; sorting the first plurality of components based on the ranking; generating a plurality of component correlation manifests from the first plurality of components; comparing the plurality of component correlation manifests with the plurality of reference project manifests to select the first reference project manifest as a reference stack, wherein each component correlation manifest is associated with a respective correlation value; comparing the first plurality of components identified in the input stack with a second plurality of components identified in the reference stack; and selecting a first component in the input stack as an outlier component based on the reference stack lacking a reference to the first component.

Aspects of the subject matter described herein may be useful alone or in combination with one or more other aspects described herein. In a 40th exemplary aspect of the present disclosure, a system comprises: a means for generating a ranking of each component represented in a plurality of reference project manifests by frequency of occurrence, wherein the plurality of reference project manifests is populated from a first source code repository storing a plurality of source code projects accessible over a network; a means for determining an input stack including references to a first plurality of components incorporated in a program; a means for sorting the first plurality of components based on the ranking; a means for generating a plurality of component correlation manifests from the first plurality of components; a means for comparing the plurality of component correlation manifests with the plurality of reference project manifests to select the first reference project manifest as a reference stack, wherein each component correlation manifest is associated with a respective correlation value; a means for comparing the first plurality of components identified in the input stack with a second plurality of components identified in the reference stack; and a means for selecting a first component in the input stack as an outlier component based on the reference stack lacking a reference to the first component.

Aspects of the subject matter described herein may be useful alone or in combination with one or more other aspects described herein. In a 41st exemplary aspect of the present disclosure, a computer-readable non-transitory storage medium storing executable instructions, which when executed by a computer system, cause the computer system to: generate a ranking of each component represented in a plurality of reference project manifests by frequency of occurrence, wherein the plurality of reference project manifests is populated from a first source code repository storing a plurality of source code projects accessible over a network; determine an input stack including references to a first plurality of components incorporated in a program; sort the first plurality of components based on the ranking; generate a plurality of component correlation manifests from the first plurality of components; compare the plurality of component correlation manifests with the plurality of reference project manifests to select the first reference project manifest as a reference stack, wherein each component correlation manifest is associated with a respective correlation value; compare the first plurality of components identified in the input stack with a second plurality of components identified in the reference stack; and select a first component in the input stack as an outlier component based on the reference stack lacking a reference to the first component.

In accordance with a 42nd exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 39th, 40th, or 41st aspects), further comprises: applying a selection filter to at least one of the first plurality of components and the plurality of reference project manifests. In accordance with a 43rd exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 39th, 40th, or 41st aspects), wherein the selection filter is one of (i) a component intersection filter, (ii) a tag correlation filter, (iii) an occupancy filter, and (iv) a component frequency filter. In accordance with a 44th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 43rd aspect), wherein the component intersection filter compares the plurality of reference project manifests with the first plurality of components identified in the input stack to select a plurality of selected reference project manifests based on respective counts of shared components with the first plurality of components. In accordance with a 45th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 43rd aspect), wherein the occupancy filter calculates respective occupancy values of reference project manifests, wherein an occupancy value includes a ratio of (i) shared components between the respective reference project manifest and the first plurality of components to (ii) a count of total components in the respective reference project manifest, and wherein the occupancy filter selects at least one selected reference project manifest with a highest occupancy score. In accordance with a 46th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 43rd aspect), wherein the component frequency filter generates a ranking of each component represented in the plurality of reference project manifests by frequency of occurrence in the plurality of reference project manifests, and ranks a set of reference project manifests based on the ranking of the components represented in each respective reference project manifest of the set of reference project manifests. In accordance with a 47th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 43rd aspect), further comprises: applying the component intersection filter to the plurality of reference project manifests prior to generating the plurality of component correlation manifests, wherein each of the plurality of component correlation manifests corresponds to a subset of the first plurality of components, and each of the plurality of component correlation manifests matches at least one reference project manifest of the plurality of reference project manifests. In accordance with a 48th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 43rd aspect), further comprises: calculating a respective usage frequency grade for each of the plurality of component correlation manifests; and selecting the first reference project manifest as the reference stack based on a usage frequency grade associated with a component correlation manifest corresponding to the first reference project manifest.

In accordance with a 49th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 39th, 40th, or 41st aspects), further comprises: configuring at least one of a maximum count of outlier components and a minimum count of outlier components based on a total count of components in the first plurality of components. In accordance with a 50th exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 39th, 40th, or 41st aspects), further comprises: populating the plurality of reference project manifests from the first source code repository based on evaluating at least one of incorporation metrics, review metrics, code complexity metrics, supportability metrics, code quality metrics, and dead code metrics of the plurality of source code projects in the first source code repository. In accordance with a 51st exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 39th, 40th, or 41st aspects), further comprises: populating the plurality of reference project manifests from a plurality of source code repositories including the first source code repository. In accordance with a 52nd exemplary aspect of the present disclosure, which may be used in combination with any one or more of other aspects described herein (e.g., the 39th, 40th, or 41st aspects), further comprises: storing respective copies of each component represented in the reference database in a component repository.

To the extent that any of these aspects are mutually exclusive, it should be understood that such mutual exclusivity shall not limit in any way the combination of such aspects with any other aspect whether or not such aspect is explicitly recited. Any of these aspects may be claimed, without limitation, as a system, method, apparatus, device, medium, etc.