CYBERSECURITY STRATEGY ANALYSIS MATRIX

A business poly-intelligence application enabling the secure collection, warehousing, analysis, and reporting of manually shared and publicly sourced business strategy data is presented with systems, methods, and computer-readable media with a specific focus on crowdsourced cybersecurity strategy development.

TECHNICAL FIELD

The present disclosure relates generally to the fields of crowdsourced knowledge, online data mining, big data analytics, data analytics, data analytics visualizations, business management, information security, information security strategy, and business management strategy.

BACKGROUND

Data analytics and the more common application of data analytics to strategic business management (known as business intelligence) are increasingly adopted as critical decision support tools around the world. Over the past twenty years, business intelligence has evolved from being a niche-but-powerful concept used by the largest businesses to its current status as a standard component or operational goal across nearly every industry and within companies of every size. Among the many drivers of business intelligence and data analytics' explosive growth are the increases in computing power available for data processing, improvements in data processing algorithms (such as artificial intelligence and machine learning), improvements in data visualizations and reporting, and the data analytics industry's shift toward self-service analysis capabilities that lower the entry bar for smaller companies that lack data scientists.

Under the best circumstances, modern companies invest in business intelligence initiatives to create enterprise-wide data analytics environments where internal and external data sources may be combined and processed for decision support and strategic planning. Analysis of data trends in areas such as sales, finance, operations, human resources, capital and operations spending, accounts receivable, and marketing allows corporate executives to base their strategic plans and tactical decisions on enterprise data instead of on intuition and/or general industry best practices.

But there is gap in the application of business intelligence capabilities beyond the scope of any single business. There is no established way for companies to access what could be called “poly-intelligence”, the results of collecting data from many companies in a given industry vertical and conducting BI-like analytics to determine best practices, enterprise strategies, and specific tactics based on real-world results. Tens of thousands of companies worldwide collect useful data on their operations, but these data are only analyzed locally within each enterprise.

SUMMARY

In accordance with the principles of the present disclosure, methods and systems are provided herein for the following aspects of the disclosure:

In some embodiments, a computer-implemented method for analyzing cybersecurity data may be provided. The method may be implemented via one or more local or remote processors, networks, servers, memory units, and/or other electronic or electrical components. In some instances, the method may include: (1) anonymously gathering and/or parameterizing manually-shared multi-enterprise cybersecurity/business strategy (cybersecurity best practices) data and cyber program outcomes; (2) gathering and/or parameterizing manually-shared, attributed cybersecurity/business strategy (cybersecurity best practices) data and cyber program outcomes from individual organizations; (3) autonomously and/or manually gathering and/or parameterizing multi-source academic research data on cybersecurity best practices and outcomes; (4) autonomously gathering and/or parameterizing open internet data on cybersecurity program design and implementation (best practices) and cyber program outcomes; (5) categorizing, transforming, and storing the data retrieved as a result of any of the foregoing steps in a common data warehouse; (6) categorizing, transforming, and/or storing the data retrieved as a result of any of the foregoing steps in a data warehouse; (7) performing business poly-intelligence analytics upon data resulting from any of the foregoing steps using descriptive and predictive analysis algorithms via business intelligence tools and proprietary analytic algorithms; (8) performing business intelligence analytics upon data resulting from any of the foregoing steps using descriptive and predictive analysis algorithms via business intelligence tools and proprietary analytic algorithms; (9) delivering analytic results in the form of reports and/or data visualizations as a result of analyses performed to provide insights into the relative strengths of an organization's cyber strategy current state and decision support/recommendations on domain-specific cyber strategy improvements; and/or (10) delivering analytic results in the form of reports and/or data visualizations as a result of analyses performed to provide threat-based predictive cyber strategy recommendations and decision support.

The foregoing aspects reflect a variety of the embodiments explicitly contemplated by the present application. Those of ordinary skill in the art will readily appreciate that the aspects below are neither limiting of the embodiments disclosed herein, nor exhaustive of all of the embodiments conceivable from the disclosure above, but are instead meant to be exemplary in nature.

These aspects may combine to create methods and systems for an end-to-end information lifecycle capability that transforms cybersecurity strategy plans and outcomes from many sources into descriptive and predictive analytic results for optimal (from both a security and financial perspective) cybersecurity program design and program efficacy evaluation for various enterprises and organizations.

Additionally, these aspects may also combine to create methods and systems for an end-to-end information lifecycle capability that transforms cybersecurity strategy plans and outcome histories from individual organizations into descriptive and predictive analytic results for optimal (from both a security and financial perspective) cybersecurity program design.

Finally, these aspects may also combine to create methods and systems for an end-to-end information lifecycle capability that transforms cybersecurity threat trends into descriptive and predictive analytic results for strategic cybersecurity program decision support (considering both security and financial aspects) in response to threat evolution.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

The Cybersecurity Strategy Analysis Matrix (CSAM) is a system of systems that may provide an independent gathering place for parameterized cybersecurity best practices information and related cyber outcomes from multiple anonymous sources; academic research, the open internet (which may include news and social media sources), and organizations (companies and government bodies). This real-world cybersecurity strategy data may be stored in a data warehouse for analysis using data analytics (which may include artificial intelligence/machine learning) algorithms. The results of these analyses may include of the aforementioned business poly-intelligence information, correlations emerge between specific cybersecurity program decisions, and related best practices and specific cyber results. Trends emerge between particular plans, actions, technologies, operations, policies, and plans and actual cybersecurity results. In addition, cost data may be captured or calculated to represent the organizational investments that may be utilized to implement specific cybersecurity strategies. Simultaneously, negative cybersecurity outcomes (losses) may be captured or calculated. As a result, business poly-intelligence analytics may be used to actively calculate the return on investment (ROI) of specific cybersecurity strategies. Perhaps most powerfully, predictive analytics may provide insights into what is likely to occur given a given set of implemented cybersecurity program practices, and what the costs and associated ROI profiles of future investments might be.

These crowd-sourced analytic results may be arranged into business intelligence reports containing dashboards, scorecards, predictive outcome summaries, ROI summaries, and other outputs presented as data visualizations. These output products may provide databased insights to both general and specific cyber strategy questions and may provide cybersecurity planning information for questions no one knew to ask. These analytic reports may be made available to cybersecurity support organizations, corporations, and government organizations to provide never-before-seen databased decision support in the war against cyber attackers. In addition, these reports may provide data analytics to support cybersecurity program evaluation and review from an efficacy and ROI perspective.

In addition to these crowd-sourced analytic products, the CSAM architecture may also support helping individual organizations better manage their own internal cybersecurity strategies based on their individual, internal cybersecurity practices and outcomes over time. For this capability, commercial and government organizations may provide the same parameterized cyber practices and outcomes information mentioned previously, but in this case there is no need for anonymity. This attributed cyber strategy information may remain segmented from any other input data and is accumulated over time in a separate data warehouse. From there, business poly-intelligence analytics using the aforementioned algorithms may be performed on the organization's data alone. The resulting trend analyses, correlation metrics, ROI estimates, predictive analytics, and other output products may be provided to the organization for using in their security optimization efforts. This capability, providing individualized cybersecurity strategy decision support to commercial and government organizations seeking to improve their security posture, is critical, among other systems and features described herein, to realizing optimized cybersecurity results.

Another aspect of the CSAM capability suite may involve the analysis of data from academic sources, public internet sources, and both public and private organizations regarding cybersecurity threat trends and correlations that relate to cybersecurity strategy. Based on data consumed from these input sources, the CSAM data warehouse may include accumulated information on cyber threat evolution historically and presently. The BI poly-intelligence capability may analyze these data points to search for correlations and trends related to specific industries and to specific cyber strategy alignment in order to produce predictive cyber threat alerts. These alerts may be designed to alert threat-focused customers to the optimal ways to shift their domain-specific cyber strategy characteristics to proactively address specific new threat trends before attackers strike, and may include cost/benefit analyses (ROI calculations) in support of final decision markers.

The present disclosure may provide an end-to-end information lifecycle that transforms cybersecurity strategy plans and outcomes from many sources into descriptive and predictive analytic results for optimal (from both a security and cost perspective) cybersecurity program design for various enterprises and organizations. Multiple communication protocols based on internet communication services, cloud-based data management techniques, business intelligence toolsets, and data warehousing/master data management technologies may be leveraged in the various aspects of the disclosure as described in the descriptions below.

An Overall View of the End-to-End Information Lifecycle from Multi-Part Source System Data Retrieval and Processing to Analytic Result and Visualization Delivery to External Parties.

FIG.1depicts a summary view of the end-to-end information lifecycle proposed in the current disclosure. Note that the present disclosure is encapsulated within the block labeled “Cybersecurity Strategy Analysis Matrix (CSAM)”, and that in contrast the three symbols to the left of the block represent input source systems and the three symbols to the right of the block represent consumers of analytic results.

For details on the specific nature of each aspect of the current disclosure as depicted inFIG.1, please reference the following detailed descriptions forFIG.2-6.

Architecture for Autonomously and Manually Gathering Multi-Source Cybersecurity Program Design and Outcome Academic Research Data and Integrating Said Data into a Data Warehouse.

FIG.2depicts the first of three categories of poly-intelligence source systems for data retrieval and processing into the CSAM analytic architecture, the scholarly research data retrieval and processing path. There is an ever-expanding universe of peer-reviewed scholarly research into cybersecurity best practices and their outcomes that forms a readily available resource for poly-intelligence cybersecurity strategy analysis. The CSAM architecture supports the retrieval and processing of academic community cyber research data, the capture of research results across many different cybersecurity domains (see Table 1 for a list of in-scope cybersecurity domains) in temporary data lakes, and the transformation and loading of research data to a data warehouse for storage until needed for analytic processes.

TABLE 1Cybersecurity Domains for CSAM Strategy Analytics1. Program Administration & Planning2. Policies, Plans, & Procedures Management3. Identity & Access Management4. Endpoint Protection5. Perimeter/Cloud Protection6. Network Security7. Risk Management8. Training9. Data Governance10. Email and Communications Security11. Secure Business Continuity12. Executive/Key Person Security13. IT Disaster Recovery14. Vulnerability Management15. Incident Response16. Mobile Device Management17. Change and Configuration Management18. Physical Cybersecurity19. IT Asset Management20. Monitoring & Log Management21. Vendor Management22. Secure Application Development23. Internal Threat ModelingReserved for Future UseReserved for Future UseReserved for Future UseReserved for Future UseReserved for Future Use

Scholarly research sources in the Academic Community Cyber Sources101cloud may include EBSCO general and premium scholarly research databases, JSTOR scholarly research articles, and university-specific research collections from around the world. The nonhomogeneous nature of research products in the academic community leads to the bifurcated input path set depicted inFIG.2; Step1“Research Data—Manual Retrieval” and Step2.

“Research Data-Automatic Retrieval”. Note that Steps1and2ofFIG.2may be asynchronous and, therefore, may occur at any time or simultaneously.

The Step1“Research Data-Manual Retrieval” input path from Academic Community Cyber Sources101may involve cybersecurity analysts manually entering relevant, cyber domain-specific best practices scholarly research information into tables for storage in a manual retrieval Data Lake201. Cybersecurity research data captured within the Step1“Research Data—Manual Retrieval” input path may be unstructured and semi-structured data that is not suited for automatic retrieval and processing via the Data Retrieval Engine203.

On a per-cyber domain basis, Step1data retrieval and processing from Academic Community Cyber Sources101to Data Lake201may occur in alignment with Table 2. Note that the Best Practices (BP) information elements in Table 2, marked with an asterisk, may include summary elements that are themselves made up of many domain-specific data elements captured in Table 3. Table 3 also may contain the BP Definition data collection. Table 4 defines the information elements for the related cybersecurity outcomes.

The standardization of summarized research data may result within Data Lake201may provide the foundation for the continuation of Step1via extraction, transformation, and loading (ETL)202processing resulting in data loads to the Data Warehouse301. The logic built into ETL202, particularly the transformations that may be required to meet the analytic aspects of Business Intelligence Engine302and its proprietary analytic algorithms, supports the common master data architecture that may be required to integrate Step1research data retrieval and processing with the source system data provided via ETL206and209. In other words, ETL202,206, and209may feature parallel design elements that support the common Data Warehouse301architecture despite being supplied by different source systems.

The Step2“Research Data-Automatic Retrieval” input path from Academic Community Cyber Sources101may involve Data Retrieval Engine203establishing electronic interfaces directly with research results databases within the academic community. Step2is therefore a multi-interface, multi-source system input path for unstructured research publications. Data Retrieval Engine203, which may be configured to locate and accept completed and peer-reviewed cybersecurity academic research, may serve as a collection and routing point to manage the various Step2source systems and route the data through the Step5“Multi-Source Cyber Info” path to the Text Mining Engine204. The Text Mining Engine204may be itself an instantiation of a data analytics tool similar to the Business Intelligence Engine302, but with limited scope designed to perform textual analysis on the unstructured data collected.

The Text Mining Engine204may be configured to automatically identify and capture, to the greatest extent possible, the information elements in Tables 2, 3, and 4 from the research sources. The results may continue along the Step5“Multi-Source Cyber Info” path to Data Lake205for storage and both automatic and manual curation by data administrators. The data curation process may address any data quality issues resulting from Text Mining Engine204's automated processing to complete data alignment with the information elements in Tables 2, 3, and 4.

Step5“Multi-Source Cyber Info” may continue from Data Lake205to ETL206, previously described as featuring parallel design elements that support the common Data Warehouse301architecture. Data loads leveraging ETL206may populate Data Warehouse301via both automated and manually triggered loading processes.

The Step2“Research Data-Automatic Retrieval” input path from Academic Community Cyber Sources101may involve a search/web crawler-based Data Retrieval Engine203leveraging electronic interfaces to academic data sources to capture relevant, cyber domain-specific best practices scholarly research information. The nature of automatic search/web crawler-based data retrieval and processing may rely on the availability of semi-structured and structured research data results within academic research sources, but may also support the retrieval and processing of unstructured data.

Regardless of the level of structure, automatic data retrieval driven by Data Retrieval Engine203, which may proceed through the Step5“Multi-Source Cyber Info” path, may be processed via the Text Mining Engine204. The text mining engine may leverage standard and novel textual analytics to derive information elements aligned with Tables 2, 3, and 4.

Step5may continue with the Text Mining Engine204output information elements (aligned with Tables 2, 3, and 4) stored in Data Lake205(a mirror of Data Lake201). As is the case in Step1, Step5may culminate with the standardized research data results within the Data Lake205supporting extraction, transformation, and loading (ETL)206(a mirror of ETL202) processing and loading into Data Warehouse301.

Architecture for Autonomously Gathering Open Internet Data on Cybersecurity Program Design and Outcomes and Integrating Said Data into a Data Warehouse.

FIG.3depicts the second of three categories of poly-intelligence source systems for data retrieval and processing into the CSAM analytic architecture, the open internet data retrieval and processing path. The open internet data retrieval path is the most complex data retrieval and processing path since the vast range of internet publications, articles, blog posts, and social media discussions pose a tremendous challenge to any big data analytics pursuit and to relevant data quality maintenance. Allowing success in the management of this compound source system is, among other systems and features described herein, the configurable nature of Data Retrieval Engine203and the manual and automatic data curation processes established along the Step5“Multi-Source Cyber Info” path.

The CSAM architecture may support the retrieval and processing of open source intelligence (OSINT) cybersecurity and cyber strategy commentary, news articles, social media alerts, and general discussions from Internet and Social Media Cyber Sources102through the Step3“Public Cyber Data” path. The content of Step3“Public Cyber Data” may be unstructured cybersecurity practice information within which may include both positive and negative best practices architecture and outcome information as well as related implementation cost information. The capture of related cybersecurity best practice information from public sources may occur within Data Retrieval Engine203for each defined cybersecurity domain (see Table 1 for a list of in-scope cybersecurity domains). The nature and definition of accepted public cybersecurity source systems may be manually determined and configured by CSAM administrators and features incremental and iterative source identification and acceptance throughout the CSAM data lifecycle.

From that point forward, the data processing path may proceed to Step5“Multi-Source Cyber Info”.

Architecture for Anonymously Gathering Manually Shared Multi-Enterprise (Government and Corporate) Cybersecurity/Business Strategy and Cyber Program Operational Results Data and Integrating Said Data into a Data Warehouse.

FIG.4depicts the third of three categories of poly-intelligence source systems for data retrieval and processing into the CSAM analytic architecture, the multi-enterprise cybersecurity/business strategy path. This is the Corporate Sources103data retrieval and processing path, where partner companies provide cybersecurity program design and outcome information destined for the Data Warehouse301. Note that both public/governmental and private sector organizations may be included within the Corporate Sources103cloud.

This source system path may begin with the submission of Step4“Cyber Experience Data” from organizations within Corporate Sources103via CSAM-internal Web Portal207. As with prior source systems, Step4“Cyber Experience Data” may be structured in alignment with the cybersecurity BP design and outcome information elements in Tables 2, 3, and 4. Web Portal207may be designed with both anonymity and security controls in place; no connecting corporate or organizational identifiers, logical or electronic, are stored within Step6“Anonymized Corporate Data”.

Web Portal207may feature end-to-end encryption via TLS1.2and organization-specific login access leveraging multi-factor authentication and session security management based on short-lived sessions. Cybersecurity best practice design and outcome (e.g., implementation costs and cybersecurity-related losses) data entry within the web portal may be accomplished via either wizard-based domain-by-domain manual entry or via upload of completed best practice.csv table/spreadsheet (which may be downloaded from the web portal's entry dashboard). Web Portal207also may feature progress tracking and email-based notifications for incomplete submissions, as well as automated email reminders requesting regular best practice design and outcome updates.

Data Lake208, previously noted as being a structural mirror of Data Lakes205and202, may store Step6“Anonymized Corporate Data”. As is the case with Steps1and5, Step6may culminate with the anonymized corporate data within the Data Lake208being subject to extraction, transformation, and loading processes via ETL209(a mirror of ETL206and202) with a final destination of the Data Warehouse301.

Architecture for Gathering Attributed, Manually Shared Organizational (Government and Corporate) Cybersecurity/Business Strategy and Cyber Program Operational Results Data and Storing Said Data into a Secondary Data Warehouse.

FIG.5depicts the fourth category for data retrieval and processing into the CSAM analytic architecture, the attributed data, internal analytics path. This is the attributed Corporate Sources103data retrieval and processing path, where organizations provide historical cybersecurity program design and outcome information (e.g., implementation costs and cybersecurity-related losses) without anonymity destined for the Data Warehouse2305. Note that both public/governmental and private sector organizations may be included within the Corporate Sources103cloud.

This source system path may begin with the submission of Step4“Cyber Experience Data” from organizations within Corporate Sources103via CSAM-internal Web Portal207. As with prior source systems, Step4“Cyber Experience Data” may be structured in alignment with the cybersecurity BP design and outcome information elements in Tables 2, 3, and 4. In this case, however, the Web Portal207anonymity controls may be bypassed, and the organization-specific cyber program information may proceed along the Step12“Attributed Cyber Info” path. The other Web Portal207capabilities previously described may also apply here.

Data Lake210, a structural mirror of Data Lakes208,205and202, may store Step12“Attributed Cyber Info”. This data may be subject to extraction, transformation, and loading processes via ETL211(a mirror of ETL209,206and202) with a final destination of the Data Warehouse2305.

Architecture for Leveraging the Data Warehouse to Perform Business Poly-Intelligence Analytics Using Descriptive and Predictive Analysis Algorithms Via Business Intelligence Tools and Proprietary Analytic Algorithms to Produce Crowdsourced Analytic Results on Cybersecurity Strategy.

For details on the specific nature of each aspect of the current invention as depicted inFIG.1, please reference the following detailed descriptions forFIG.2-6.

FIG.6depicts the CSAM design for the capture and storage of integrated cybersecurity best practices design and outcome data within Data Warehouse301, as well has how the aggregated data is leveraged by poly-intelligence analytics to generate crowdsourced analytic results.

ETL202,206, and209may provide data loads to the Data Warehouse301. Data Warehouse301may be a cloud-based, dynamic, multi-part data management system that may include both relational and dimensionally modeled components. The structure of the warehouse iteratively changes along with the adaptive nature of the detailed data elements as well as the summary information elements in Tables 2, 3, and 4. A strict data governance process and agile management approach are in place to maintain Data Warehouse301as a “source of truth” for CSAM analytics.

From the Data Warehouse301, the Step7“Aggregated Cyber Data” path allows the Business Intelligence Engine302to request/extract specific datasets for analysis using proprietary analytic algorithms.

In some embodiments, the analytic algorithm type employed within Business Intelligence Engine302may be direct correlation computation based on simple and/or advanced regression analyses across the multidimensional surfaces from per-domain and cross-domain BP data elements and cybersecurity outcomes as defined in Table 4 (this analytic algorithm also may be described as a “descriptive analytic algorithm” herein). As the volume of data available in Data Warehouse301increases, strong and statistically significant correlations between best practices design and specific cyber outcome details emerge with increasing levels of correlation confidence.

In these embodiments, the cybersecurity outcomes as defined in Table 4 may be dependent variables and each of the multidimensional surfaces from per-domain and cross-domain BP data elements may be independent variables. A machine learning module may generate a machine learning model as an equation, which most closely approximates the cybersecurity outcomes as defined in Table 4 from the multidimensional surfaces from per-domain and cross-domain BP data elements. In some embodiments, an ordinary least squares method may be used to minimize the difference between the value of the guessed cybersecurity outcomes and the actual cybersecurity outcomes using the machine learning model.

Additionally, the differences between the values of each of the multidimensional surfaces from per-domain and cross-domain BP data elements (ŷi) using the machine learning model and actual cybersecurity outcomes as defined in Table 4 (yi) may be aggregated and/or combined in any suitable manner to determine a mean square error (MSE) of the regression. The MSE may be used to determine a standard error or standard deviation(s) in the machine learning model, which may in turn be used to create confidence intervals. For example, assuming the data is normally distributed, a confidence interval which may include about three standard deviations from the guessed cybersecurity outcomes using the machine learning model (ŷi−3σε−ŷi+3σε) may correspond to 99.5 percent confidence. A confidence interval which may include about two standard deviations from the recommended vehicle seat using the machine learning model (ŷi-2σε−ŷi+2σε) may correspond to 95 percent confidence. Moreover, a confidence interval which may include about 1.5 standard deviations from the recommended vehicle seat using the machine learning model (ŷi−1.5σε−ŷi+1.5σε) may correspond to 90 percent confidence.

In some other embodiments, the analytic algorithm type employed within Business Intelligence Engine302may be machine learning-based predictive analytics. More specifically, the accumulated data within Data Warehouse301may be used to train machine learning algorithms in support of decision modeling. The inputs from the various parameterized best practices may represent hundreds of specific decisions intended to generate specific outcomes. The outcome information, also parameterized, may be combined with the best practice inputs to train machine learning algorithms on the most likely outcomes aligned with the input decisions and investment profiles. Once again, as the volume of data available in Data Warehouse301increases the accuracy and confidence levels of decision model predictions increases.

The machine learning algorithms may also be tested to determine accuracy. In some embodiments, the testing data may be from the same collection of data as the training data. In these embodiments, the training data is divided into a ratio of training data and testing data (e.g., 20% training data and 80% testing data). Once divided, the training data generates the machine learning model and the testing data determines the accuracy of the model. When the machine learning module is correct more than a predetermined threshold amount, the machine learning model may be used for generating the specific outcomes. However, if the machine learning module is not correct more than the threshold amount, the machine learning module may continue obtaining sets of training data and/or testing data for further training and/or testing.

The aforementioned algorithms may be based on the application of Evidence-based Weighting (EBW) for specific factors in best practices design. EBW also may take into account non-parameterized inputs such as corporate culture information, source reliability, human factors issues in specific industries and organizational types, and other indirect factors discovered during source data evaluation and industry analysis. These EBW factors may be iteratively applied to both regression and decision modeling datasets to account for non-parameterized factors. The EBW impacts themselves are cross-analyzed against non-weighted input sets to increase the accuracy of the factors in future iterations. This allows the ever-evolving current state of individual organizational cybersecurity strategy, industry-level cybersecurity strategy, and general cybersecurity strategy to be more accurately reflected in the analytic results.

These analytic results may proceed through Step8“BI Engine Output”, flowing from Business Intelligence Engine302to Crowdsourced Analytic Results303. Crowdsourced Analytic Results303may be a results repository within the BI stack for the storage of initial, intermediate, and final analytic results from regression and decision modeling activities within Business Intelligence Engine302. Initial and intermediate results may be staged for re-analysis via the same or different analytical approaches or for iterative re-analysis using adapted EBW factors.

Architecture for Leveraging the Secondary Data Warehouse to Perform Business Intelligence Analytics Using Descriptive and Predictive Analysis Algorithms Via Business Intelligence Tools and Proprietary Analytic Algorithms to Produce Individual Organization Analytic Results on Cybersecurity Strategy.

FIG.7depicts the CSAM design for the capture and storage of individual organization cybersecurity best practices design and outcome data within Data Warehouse2305, as well has how the aggregated data is analyzed to generate organization-specific cyber strategy analytic results.

Data Warehouse2305, a structural mirror of Data Warehouse301, may be a cloud-based, dynamic, multi-part data management system which may include both relational and dimensionally modeled components. The structure of the warehouse may iteratively change along with the adaptive nature of the detailed data elements as well as the summary information elements in Tables 2, 3, and 4. A strict data governance process and agile management approach may be in place to maintain Data Warehouse2305as a “source of truth” for individual organization CSAM analytics.

From Data Warehouse2305, the Step13“Attributed Cyber Data” path may allow the Business Intelligence Engine302to request/extract specific datasets for analysis using proprietary analytic algorithms. Business Intelligence Engine302may leverage the same per-domain and cross-domain BP data elements and cybersecurity outcomes previously mentioned. As the volume of data available in Data Warehouse2305increases, strong and statistically significant correlations between best practices design and specific cyber outcome details emerge with increasing levels of correlation confidence for specific organizations. Initial and intermediate results may be staged for re-analysis within Business Intelligence Engine302via the same or different analytical approaches or for iterative re-analysis. The other key capabilities previously described for Business Intelligence Engine302apply.

These analytic results may proceed through Step14“Attributed Results”, flowing from Business Intelligence Engine302to Reporting and Visualization Engine304.

Architecture for Leveraging the Crowdsourced Analytic Results within a Reporting and Visualization Engine (Supported by the Business Intelligence Platform) to Create Cybersecurity Strategy and Insight Deliverables Tailored to Specific Information Consumers.

FIG.8depicts the CSAM design for leveraging the crowdsourced analytic results within a reporting and visualization engine to create cybersecurity strategy and insight deliverables tailored to specific information consumers.

Crowdsourced Analytic Results303may contain analytic results data that may require direct intervention by human cybersecurity strategy experts to validate and verify applicability and completeness before processing into cybersecurity strategy and insight deliverables. This cultivated set of outputs represent a core component of business poly-intelligence analyses performed by the CSAM invention.

After verification and validation, these cultivated output datasets may flow through the Step9“Analytic Results” path to Reporting and Visualization Engine304. Cultivated output analytic results may be organized into many potential reporting and visualization types leveraging the visualization engine, based on both information consumer requests and on internal CSAM cybersecurity strategy expert directive. The reporting and visualization options may include dash boards, individual graphs and charts, scorecards, and narrative reports that may accompany visualizations, include visualizations, or may stand alone. Regardless of the medium, Reporting and Visualization Engine304may be leveraged to create organized summaries of trends, correlations, and predictions for cybersecurity strategy based on the many potential combinations of input best practices data from academic, open internet, and corporate sources.

A first type of output from Reporting and Visualization Engine304may be the Cyber Insight Analysis. Cyber Insight Analyses may proceed along the Step10“Cyber Insight Analyses” path to Cybersecurity Support Organizations104. Cyber Insight Analyses may be not organization or company specific, but rather contain industry-specific, size-specific, and strategic approach-specific analytic results for use by organizations in need of increased clarity into the databased best practices approach in some or all of the22cybersecurity domains. The information consumers for Cyber Insight Analyses, Cybersecurity Support Organizations104, may include law firms, insurance providers, educational/academic bodies, managed services providers, and perhaps most commonly organizations that provide cybersecurity consulting to multiple other independent organizations.

A second type of output from Reporting and Visualization Engine304may be Cyber Strategy & Optimization Intelligence. Cyber Strategy & Optimization Intelligence may proceed along the Step11“Cyber Strategy & Optimization Intelligence” path to Corporate Cyber Practitioners105and Government Cyber Decision Makers106. Cyber Strategy & Optimization Intelligence may be much more specific than Cyber Insight Analyses and may provide detailed analytic results and visualizations for specific organizations based on their alignment with cybersecurity strategy best practices, calculated cybersecurity strategy ROI, and the analytic results/predictions from the CSAM process. In many cases, Cyber Strategy & Optimization Intelligence may provide answers to specific strategic questions posed by Corporate Cyber Practitioners105and Government Cyber Decision Makers106information consumers. In others, CSAM cyber strategy experts proactively determine critical correlations or predictions and offer the corresponding results and visualizations to these information consumers.

The delivery of all three categories of output products via Step10“Cyber Insight Analyses” and Step11“Cyber Strategy & Optimization Intelligence” may be cyclical, iterative, and/or recursive in nature, reflecting the every-changing nature of cybersecurity best practices and their real-world outcomes in the many different sizes and types of organizations worldwide. Also note that, by design, many of the entities acting as information consumers in Cybersecurity Support Organizations104, Corporate Cyber Practitioners105, and Government Cyber Decision Makers106may be the same entities providing input to the CSAM process within Academic Community Cyber Sources101and Corporate Sources103.

Architecture for Leveraging Individual Organization Analytic Results within a Reporting and Visualization Engine (Supported by the Business Intelligence Platform) to Create Cybersecurity Strategy and Optimization Deliverables.

FIG.9depicts the CSAM design for leveraging individual organization analytic results within a reporting and visualization engine to create cybersecurity strategy and optimization deliverables for each organization.

Crowdsourced Analytic Results303contains analytic results data that may require direct intervention by human cybersecurity strategy experts to validate and verify applicability and completeness before processing into cybersecurity strategy and insight deliverables. This cultivated set of outputs represent a core component of business poly-intelligence analyses performed by the CSAM invention.

Reporting and Visualization Engine304may be leveraged as previously described but for individual organizations. The output products for individual organizations may include the Step13“Cyber Optimization Intelligence” flowing to Corporate Cyber Practitioners105and Government Cyber Decision Makers106. As with other reports previously defined, Step13“Cyber Optimization Intelligence” may be cyclical, iterative, and/or recursive in nature, reflecting the every-changing nature of cybersecurity best practices and their real-world outcomes within each individual organization.

Architecture for Leveraging Crowdsourced Analytic Results of Threat Data and Threat Trends within a Reporting and Visualization Engine (Supported by the Business Intelligence Platform) to Create Cybersecurity Strategy Recommendations/Alerts Based on Threat Trends.

FIG.10depicts the CSAM design for leveraging crowdsourced analytic results within a reporting and visualization engine to create cybersecurity strategy recommendations and alerts based on threat trends. Native to the outcome information captured from academic, open internet, and organizational sources may include significant threat data. In addition, threat model domain information may be captured as a part of overall cybersecurity strategy information capture. Each of these sets of cybersecurity threat information may be analyzed within the CSAM analytics engine to create ROI-focused threat alerts targeting strategic cybersecurity program changes.

The analytic algorithms described herein for use within Business Intelligence Engine302may involve correlation computations based on simple regression analyses across the multidimensional surfaces from per-domain and cross-domain BP data elements and cybersecurity outcomes as defined in Table 4. This same approach may be used to support correlation analyses of for threat information, resulting in correlation statistics for specific types of cybersecurity threats that correlate with successful attacks given specific previously implemented cybersecurity strategies.

Similarly, as described above, Business Intelligence Engine302may use machine learning-based predictive analytics in support of decision modeling. The same process may apply here for threat alert generation. The inputs from the various parameterized threat data sets may be combined with outcome information to train machine learning algorithms on the most likely outcomes that a given threat will trigger given a specific previously implemented cybersecurity strategy. As a result, strategic decisions that are likely to result in negative outcomes may be used to trigger cyber strategy alerts.

Both types of threat-based analytic results may be stored within Crowdsourced Analytic Results303. Reporting and Visualization Engine304may be leveraged to create organized summaries of trends, correlations, and predictions for cybersecurity strategy based on these threat analytics. As previously, direct intervention by human cybersecurity strategy experts may be required to validate and verify applicability and completeness before processing into Strategic Cyber Threat Alerts14. The completed Strategic Cyber Threat Alerts14may be delivered to Threat-Focused Organizations107to complete the information lifecycle.

It should be appreciated that the foregoing processes, methods, and/or techniques described herein need not be performed in any specific order and/or need not be performed by specific architecture (e.g., a singular component may be both the Text Mining Engine204and the Data Lake205, more than two data warehouses may be utilized, etc.). Further, processes, methods, and/or techniques calling for iterative, incremental, cyclical, and/or recursive processing techniques may be interchangeably performed by any one or more of iterative, incremental, cyclical, and/or recursive processing where appropriate.

Exemplary Computing Devices and Systems

FIG.11depicts a block diagram of an exemplary computing system400to implement any of the foregoing systems, methods, and/or techniques in accordance with described embodiments.

The computing system400may include one or more processors402(e.g., a programmable processor, a programmable controller, a GPU, a DSP, an ASIC, a PLD, an FPGA, an FPLD, etc.), one or more memories (e.g., random access memory (RAM)414, read only memory (ROM)416, cache, etc.)404, one or more program memories406, one or more input units410, and/or one or more output units412, all of which may be interconnected via an address/data bus420. The one or more program memories406may store software and/or computer-executable instructions, which may be executed by the one or more processors402.

The one or more program memories406may include one or more memories404that may store software and/or computer-executable instructions. The software and/or computer-executable instructions may be stored on separate non-transitory computer-readable storage mediums or disks, or at different physical locations.

In some embodiments, the one or more processors402may also include, or otherwise be communicatively connected to, one or more databases408or other data storage mechanism (one or more hard disk drives, optical storage drives, solid state storage devices, CDs, CD-ROMs, DVDs, Blu-ray disks, etc.). In some examples, the one or more databases408store a set of training/testing data.

The one or more input units410and/or the one or more output units412may include any number of different types of input and/or output units and/or combined I/O circuits and/or components that enable the one or more processors402to communicate with peripheral devices. The peripheral devices may be any desired type of device such as a keyboard, a display (a liquid crystal display (LCD), a cathode ray tube (CRT) display, touch, etc.), a navigation device (a mouse, a trackball, a capacitive touch pad, a joystick, etc.), a speaker, a microphone, a button, a communication interface, an antenna, etc. The one or more input units410and/or the one or more output units412may include any number of different network transceivers418. The network transceivers118may be a Wi-Fi transceiver, a Bluetooth® transceiver, an infrared transceiver, a cellular transceiver, an Ethernet network transceiver, an asynchronous transfer mode (ATM) network transceiver, a digital subscriber line (DSL) modem, a cable modem, etc.

The one or more program memories106and/or the one or more memories404may be implemented in any known form of volatile or non-volatile computer storage media, including but not limited to, semiconductor memories, magnetically readable memories, and/or optically readable memories, for example, but does not include carrier waves.

As used herein, a non-transitory computer-readable storage medium or disk may be, but is not limited to, one or more of a hard disk drive (HDD), an optical storage drive, a solid-state storage device, a solid-state drive (SSD), a read-only memory (ROM), a random-access memory (RAM), a compact disc (CD), a compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a Blu-ray disk, a cache, a flash memory, and/or any other storage device or storage disk in which information may be stored for any duration (e.g., permanently, for an extended time period, for a brief instance, for temporarily buffering, for caching of the information, etc.).

It should be appreciated that the computing system400may include multiple nodes (computers) comprising of multiple processors402, multiple memories404, multiple program memories406, multiple databases408, multiple input units410, and/or multiple output units412in the form of computing clusters where a cluster is in the form or one or more of these nodes.

It should be appreciated that while specific elements, components, and/or devices are described as part of computing system400, other elements, components, and/or devices are contemplated.

Exemplary Machine Learning Training Module and Scoring Module

FIG.12depicts a diagram of an exemplary machine learning training module500. The machine learning training module500may include a training module510, training/testing data512, a machine learning engine514, a testing module516, a model validation module518, a machine learning model520, a scoring module530, and/or a scoring engine532.

The training module510may include the machine learning engine514, the testing module516, and/or the model validation module518. The training/testing data512may store any number of prior multidimensional surfaces from per-domain and cross-domain BP data elements and/or cybersecurity outcomes as defined in Table 4 which may be stored on any number or type(s) of non-transitory machine-readable storage medium or disk using any number or type(s) of data structures. The scoring module530may include the scoring engine532.

The training module510, the machine learning engine514, the testing module,516, the model validation module518, the machine learning model520, the scoring module530, and/or the scoring engine532, may be, or may include, a portion of a memory unit (e.g., the one or more program memories406ofFIG.11) configured to store software and/or computer-executable instructions that, when executed by a processing unit (e.g., the one or more processors402ofFIG.11), may cause the one or more of the aforementioned components to generate, develop, train, test, deploy, and/or validate the machine learning model520for generating one or more resulting outputs of cybersecurity outcomes. The training module510, the machine learning model520and/or the scoring module530may be executed for use as a machine learning module550. There may be one or more machine learning models520.

In operation, the input module501may initially access the machine learning training module500. The machine learning training module500may form input vectors from the training/testing data512and may be passed through the machine learning engine514to form test cybersecurity outcomes. Similarly, the machine learning training module500may pass prior multidimensional surfaces from per-domain and cross-domain BP data elements and/or cybersecurity outcomes to the testing module516and/or to the model validation module518. The developing machine learning model within the machine learning engine514may be trained using supervised learning.

The testing module516may compare the resulting outputs of cybersecurity outcomes by the machine learning engine514to the actual cybersecurity outcomes of the input training data to determine an error rate that may be used to develop and/or update the machine learning model520. The machine learning engine514may generate, develop, deploy, and/or update the machine learning model520by using, for example, gradient boosting machine learning, a neural network, deep learning, a regression technique, etc.

The developing machine learning model within the machine learning engine514may be validated by the model validation module518. The model validation module may statistically validate the developing machine learning model, for example, by using k-fold cross-validation. In these embodiments, the training/testing data512may be randomly split into k parts, and the developing machine learning model may be trained using k−1 of the k parts of the training/testing data512which represent prior multidimensional surfaces from per-domain and cross-domain BP data elements and/or cybersecurity outcomes.

The developing machine learning model may be evaluated using the remaining one part of the training/testing data512which represent the multidimensional surfaces from per-domain and cross-domain BP data elements and/or cybersecurity outcomes, which the machine learning engine514has not yet been exposed to. Results of the developing machine learning model for generating resulting outputs of cybersecurity outcomes are compared to the actual cybersecurity outcomes by the model validation module518to determine the performance and/or convergence of developing machine learning model. Performance and/or convergence may be determined by, for example, identifying when a metric computed over the previously determined error rate (e.g., a mean-square metric, a rate-of-decrease metric, etc.) satisfies a criteria (e.g., a metric is less than a predetermined threshold, such as a root mean squared error).

The resulting machine learning model520may be further evaluated by the scoring module530. The scoring engine532of the scoring module530may be used to generate simulated input data from sample data from the training/testing data512. The simulated input data may include multidimensional surfaces from per-domain and cross-domain BP data elements and/or cybersecurity outcomes, etc.

In some alternative embodiments, the scoring module530may develop, deploy, and/or update the machine learning model520without the training module510. In these embodiments, the scoring module530uses sample data from the training/testing data512to generate a plurality of simulated input data. The input data may be used as the training data and/or the testing data in the development of the machine learning model520.

The foregoing processes may repeat until the results of the machine learning model520produce a desirable error rate. The machine learning model520may be updated from parallel machine learning engines514and/or scoring engines532. It should be appreciated that while specific elements, processes, devices, and/or components are described as part of example machine learning training module500, other elements, processes, devices and/or components are contemplated and/or the elements, processes, devices, and/or components may interact in different ways and/or in differing orders, etc. Additionally, the machine learning models described herein may utilize any artificial intelligence techniques including, but not limited to, such as gradient boosting, neural networks, deep learning, linear regression, polynomial regression, logistic regression, support vector machines, decision trees, random forests, nearest neighbors, and/or any other suitable machine learning technique, some of which are described in more detail herein.

Exemplary Methods and Processes

FIG.13depicts an exemplary computer-implemented method600for generating cybersecurity outcomes using automated data capturing and machine learning algorithms. The method600depicted inFIG.13may employ any of the techniques, methods, and systems described herein with respect toFIGS.1-12.

The method600may begin at block602by training, by one or more processors, a first machine learning model using a first training dataset related to at least one area of interest of cybersecurity, the first training dataset comprising outcome information and one or more of: (i) academic training data, (ii) open internet training data, and/or (iii) corporate training data. A machine learning module (e.g., machine learning module550) may generate a machine learning model based upon training data from previously generated cybersecurity outcomes. The training data may include, for each multidimensional surfaces from per-domain and cross-domain BP data elements and/or cybersecurity outcomes as defined in Table 4.

The machine learning module may test the machine learning model generated. In some embodiments, the test may be conducted using the machine learning technique used to generate the model (e.g., gradient boosting, neural networks, deep learning, linear regression, polynomial regression, support vector machines, decision trees, random forests, nearest neighbors, and/or any other suitable machine learning technique). Further, in some embodiments, the testing data may be from the same collection of data as the training data. In these embodiments, the training data may generate the machine learning model and the testing data may determine the accuracy of the model. When the machine learning module is correct more than a predetermined threshold amount, the machine learning model may be used generating cybersecurity outcomes. However, if the machine learning module is not correct more than the threshold amount, the machine learning module may continue obtaining sets of training data and/or testing data for further training and/or testing.

The method600may proceed to block604by storing, by the one or more processors, the first machine learning model in one or more memories.

The method600may proceed to block606by retrieving, by the one or more processors, a first collection of data, the first collection of data including one or more of academic data, open internet data, and/or corporate data, and the first collection of data is related to the at least one area of interest of cybersecurity. As described in detail above, the academic data may include peer-reviewed academic research, the open internet data may include one or more of one or more news sources, one or more blogs, one or more forum posts, and/or one or more social media sources, and the corporate data may include one or more of anonymized corporate data and/or attributed corporate data. Any of the first collection of data may be collected by the Data Retrieval Engine203and/or the Web Portal207. Further, any of the first collection of data may be retrieved manually and/or automatically (e.g., by using artificial intelligence techniques and/or algorithms). In addition, the area of interests of cybersecurity may include one or more of: ransomware attacks, denial of service attacks, social engineering attacks, password attacks, cloud attacks, near misses, and/or threat trends

The method600may proceed to block608by analyzing, by the one or more processors using the first machine learning model stored in the one or more memories, the first collection of data. Analysis of data described herein may include one or more of descriptive analysis algorithms, predictive analysis algorithms, and/or statistical modeling algorithms.

The method600may proceed to block610by generating, by the one or more processors based upon the analysis, a resulting output, the resulting output including one or more of: a strength of a cybersecurity strategy of an organization, a recommendation of a change to a cybersecurity strategy of an organization, or a predicted outcome given a cybersecurity strategy of an organization. This resulting output may then be further processed (e.g., visualization data may be generated, etc.) and/or may be provided to one or more Cyber Support Organizations104, Threat-focused Organizations107, Corporate Cyber Practitioners105, and/or Government Cyber Decision Makers106.

The method600may have more or less or different steps and/or may be performed in different orders of steps. For example, the method600may also include (i) training, by the one or more processors, a second machine learning model using a second training dataset related to at least one area of interest of cybersecurity, the second training dataset comprising outcome information and one or more of: (a) the academic training data, (b) the open internet training data, and/or (c) the corporate training data; (ii) storing, by the one or more processors, the second machine learning model in the one or more memories; (iii) identifying, by the one or more processors using the second machine learning model stored in the one or more memories, a second collection of data, the second collection of data including one or more of academic data, open internet data, and/or corporate data, and the second collection of data is related to the at least one area of interest of cybersecurity; (iv) reducing, by the one or more processors, the percent rate of error of generating the resulting output by calculating one or more of: (a) the ordinary least squares of the difference between the generated resulting output and the actual resulting output of the first training data set, and/or (b) the ordinary mean square of an aggregation of results between the generated resulting output and the actual resulting output of the first training data set; and/or (v) generating, by the one or more processors, a confidence interval based upon one or more of: (a) the generated resulting output, (b) the actual resulting output of the first training data set, and/or (c) one or more standard deviations from the aggregated result.

Exemplary Best Practice Data Elements Per-Domain

The following set of tables are a non-exhaustive list of data elements that may be used throughout various aspects of this description. Note that the following detailed per-domain best practice data elements are designed to change and grow, adapting to the changing nature of cybersecurity best practices and the iteratively discovered best approaches to identify and parameterize cybersecurity strategy.

TABLE A3Domain Information Elements—Identity and Access ManagementDomain Information ElementData TypeIAM Policy DefinedYes/NoIAM Policy Coverage LevelPercentageCentral IAM Solution In PlaceYes/NoMFA, Critical System CoveragePercentageMFA, Overall System CoveragePercentageAccount Audit/Review Cadence (per year)NumericHR Integration Confidence LevelPercentageLeast Privilege Controls In PlaceYes/NoPAM Access Controls in PlaceYes/No/PartialPAM Access Confidence LevelPercentageEndpoint Local Admin RestrictedYes/No/PartialDefault Cloud Super-User Accounts DisabledYes/No/PartialUser Permissions Removal Confidence LevelPercentageTemporary Access Management Controls inYes/NoPlaceTemporary Access Management ConfidencePercentageLevelSeparation of Duties in PlaceYes/No/PartialSeparation of Duties Confidence LevelPercentageAccount Creation/Deletion Confidence LevelPercentageRBAC CoveragePercentageAccess Logging CoveragePercentageShared Access AllowedYes/NoShared Access Confidence LevelPercentagePassword Strength RatingCategorical(weak, moderate,strong, very strong)Policy Exceptions Allowed (per year)NumericSingle Sign-On Access in PlaceYes/NoSingle Sign-On Business Systems CoveragePercentagePolicy Compliance Violations (per year)Numeric

TABLE A6Domain Information Elements—Network SecurityDomain Information ElementData TypeNetwork Security (NS) Policy DefinedYes/NoNS Policy Coverage LevelPercentageIntegrated NS Solution in PlaceYes/NoThird Party NS and/or IDS/IPS Service inYes/NoPlaceIDS/IPS Controls in PlaceYes/NoISD/IPS Centrally ManagedYes/NoPhysical Connection Controls in PlaceYes/NoNetwork Segmentation in PlaceYes/NoNumber of Operating SegmentsNumericNS Configuration Review CadenceNumeric(minimum times per year)

TABLE A8Domain Information Elements—TrainingDomain Information ElementData TypeCybersecurity Training (CT) Policy DefinedYes/NoCT Policy Coverage LevelPercentageCT Training Cadence—StaffNumeric(minimum times per year)CT Training Cadence—ITNumeric(minimum times per year)Customized TrainingPercentageGeneral Cyber Training Solution in PlaceYes/NoGeneral Cyber Testing Solution in PlaceYes/NoPhishing Training Solution/Service in PlaceYes/NoPhishing Testing Solution/Service in PlaceYes/NoGeneral Cyber Awareness Confidence LevelPercentagePhishing/Social Engineering ConfidencePercentageLevelExecutive Training in PlaceYes/NoExecutive Training CadenceNumeric/Not applicable(minimum times per year)Key Person Training in PlaceYes/NoKey Person Training CadenceNumeric/Not applicable(minimum times per year)Privileged Access Management Training inYes/NoPlacePrivileged Access Management TrainingNumeric/Not applicableCadence (minimum times per year)General Data Governance Training in PlaceYes/NoGeneral Data Governance Training TypeIntegrated/Standalone/Custom/Not applicableInsider Threat Training in PlaceYes/NoInsider Threat Training TypeIntegrated/Standalone/Custom/Not applicableStaff Incident Response (IR) Training inYes/NoPlaceStaff IR Training CadenceYes/No(minimum times per year)Staff IR Training TypeIntegrated/Standalone/Custom/Not applicableWhistleblower Training in PlaceYes/NoWork-from-Home Training in PlaceYes/No/Not applicableWork-while-Traveling Training in PlaceYes/No/Not applicableCT Compliance/Efficacy Review CadenceNumeric/Not applicable(minimum times per year)

TABLE A9Domain Information Elements—Data GovernanceDomain Information ElementData TypeData Governance (DG) Policy DefinedYes/NoDG Policy Coverage LevelPercentageDG Committee EstablishedYes/NoDG Committee Operating CadenceNumeric/Not(minimum times per year)applicableData ClassesNumericData Protection Defined by ClassYes/NoData Protection Confidence LevelPercentageData Retention Plan in PlaceYes/NoData Retention Plan CompliancePercentageData Loss Prevention Solution in PlaceYes/NoEncryption-at-RestYes/NoEncryption-at-Rest CompliancePercentageEncryption-in-MotionYes/NoEncryption-in-Motion CompliancePercentageEnterprise Data Model DefinedYes/NoEnterprise Data Model Confidence LevelPercentageData Ownership Confidence LevelPercentageData Definition Confidence LevelPercentageDG Compliance/Efficacy Review CadenceNumeric/Not(minimum times per year)applicable

TABLE A16Domain Information Elements—Mobile Device ManagementDomain Information ElementData TypeMobile Device Management (MDM) PolicyYes/NoDefinedMDM Policy Coverage LevelPercentageCompany-owned Mobiles UsagePercentageBYOD Mobile UsagePercentageCentralized MDM Solution in PlaceYes/NoRemote Wipe SupportedYes/No/PartialGeolocation SupportedYes/No/PartialEmail-only MDM Solution in PlaceYes/No/PartialMobile Application Management in PlaceYes/No/PartialLaptop MDM Controls in PlaceYes/No/PartialMDM Compliance/Efficacy Review CadenceNumeric/Not(minimum times per year)applicable

TABLE A17Domain Information Elements—Change and Configuration ManagementDomain Information ElementData TypeChange and Configuration ManagementYes/No(CCM) Policy DefinedCCM Policy Coverage LevelPercentageChange Management Procedures DefinedYes/No/PartialChange Advisory Board TypeCategorical (Virtual,Live, Mixed, Text/List-based)Centralized Change Tracking Tool in PlaceYes/No/PartialRisk-based Change Tracking in PlaceYes/No/PartialChange TestingYes/No/ConditionalChange Management Confidence LevelPercentageChange Detection Tools in PlaceYes/No/PartialConfiguration BackupsYes/No/PartialConfiguration Backup Cadence (minimumNumerictimes per month)Centralized Config Backup Tool(s) in PlaceYes/No/PartialEndpoint Baseline Config MaintainedYes/NoEndpoint Baseline Config HardeningPercentageConfidence LevelServer Config Baseline MaintainedYes/NoServer Baseline Config HardeningPercentageConfidence LevelCCM Compliance/Efficacy Review CadenceNumeric/Not applicable(minimum times per year)

TABLE A18Domain Information Elements—Physical CybersecurityDomain Information ElementData TypePhysical Cybersecurity (PC) Policy DefinedYes/NoPC Policy Coverage LevelPercentageOn-Premise Equipment Access Controls inYes/No/PartialPlaceOn-Premise Equipment Video Monitoring inYes/No/PartialPlaceOn-Premise Physical Security Audit CadenceNumeric(minimum times per year)On-Premise Equipment Intrusion Alarms inYes/No/PartialPlaceColo/Remote Site Physical Security AuditNumericCadence (minimum times per year)Colo/Remote Site Equipment Access ControlsYes/No/Partialin PlaceColo/Remote Site Equipment VideoYes/No/PartialMonitoring in PlaceColo/Remote Site Equipment IntrusionYes/No/PartialAlarms in PlaceEndpoint Clear Screen Controls in PlaceYes/No/PartialClear Desk Review & Enforcement in PlaceYes/No/PartialFacility Electronic Access Controls in PlaceYes/No/PartialFacility Human-based Access Controls inYes/No/PartialPlaceFacility Alarms in PlaceYes/No/PartialFacility Video Monitoring in PlaceYes/No/PartialWork-from-Home Security Audit/Review inYes/No/PartialPlaceWork-while-Traveling Security Audit/ReviewYes/No/Partialin PlacePC Compliance/Efficacy Review CadenceNumeric/Not(minimum times per year)applicable

Additional Considerations

The detailed description is to be construed as exemplary only and does not describe every possible embodiment since describing every possible embodiment would be impractical. Numerous alternative embodiments may be implemented, using either current technology or technology developed after the filing date of this application, which would still fall within the scope of the claims.

Examples of computer code include machine code, such as produced by a compiler, and files containing higher-level code that are executed by a computer using an interpreter or a compiler. For example, an embodiment of the disclosure may be implemented using Java, C++, or other object-oriented programming language and development tools. Additional examples of computer code include encrypted code and compressed code. Moreover, an embodiment of the disclosure may be downloaded as a computer program product, which may be transferred from a remote computer (e.g., a server computer) to a requesting computer (e.g., a client computer or a different server computer) via a transmission channel. Another embodiment of the disclosure may be implemented in hardwired circuitry in place of, or in combination with, machine-executable software instructions.

Some embodiments of the disclosure relate to a non-transitory computer-readable storage medium having instructions/computer-readable storage medium thereon for performing various computer-implemented operations. The term “instructions/one or more computer-readable media” is used herein to include any medium that is capable of storing or encoding a sequence of instructions or computer codes for performing the operations, methodologies, and techniques described herein. The media and computer code may be those specially designed and constructed for the purposes of the embodiments of the disclosure, or they may be of the kind well known and available to those having skill in the computer software arts. Examples of computer-readable storage media include, but are not limited to: magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and holographic devices; magneto-optical media such as optical disks; and hardware devices that are specially configured to store and execute program code, such as ASICs, programmable logic devices (“PLDs”), and ROM and RAM devices.

This description provided herein is to be construed as exemplary only and does not describe every possible embodiment, as describing every possible embodiment would be impractical, if not impossible. One may be implement numerous alternate embodiments, using either current technology or technology developed after the filing date of this application. While the present disclosure has been described and illustrated with reference to specific embodiments thereof, these descriptions and illustrations do not limit the present disclosure. It should be understood by those skilled in the art that various changes may be made and equivalents may be substituted without departing from the true spirit and scope of the present disclosure as defined by the appended claims. The illustrations are not necessarily drawn to scale. There may be distinctions between the artistic renditions in the present disclosure and the actual apparatuses and/or systems due to manufacturing processes, tolerances and/or other reasons. There may be other embodiments of the present disclosure which are not specifically illustrated. Modifications may be made to adapt a particular situation, material, composition of matter, technique, or process to the objective, spirit and scope of the present disclosure. All such modifications are intended to be within the scope of the claims appended hereto. While the techniques disclosed herein have been described with reference to particular operations performed in a particular order, it will be understood that these operations may be combined, sub-divided, or re-ordered to form an equivalent technique without departing from the teachings of the present disclosure. Accordingly, unless specifically indicated herein, the order and grouping of the operations are not limitations of the present disclosure.

Those of ordinary skill in the art will recognize that a wide variety of modifications, alterations, and combinations may be made with respect to the above described embodiments without departing from the scope of the invention, and that such modifications, alterations, and combinations are to be viewed as being within the ambit of the inventive concept. The systems and methods described herein are directed to an improvement to computer functionality, and improve the functioning of conventional computers.