Analyzing and explaining a temporal evolution of policies and suggesting next steps

A system and method for analyzing and explaining the temporal evolution of policies. The analysis of the temporal evolution of policies includes determining semantically related changes in the text of the corresponding policies over time and across the different versions of the (two or more) policy documents. An explanation of the temporal evolution of policies is provided consisting of human interpretable information relating each change to one or more events and/or contextual data. A next steps policy change is also suggested that includes a set of policy conditions based on potential future changes of similar policies and contextual data and event data. The suggestions are related to the policy relevant to the target cohort or individual. The system adapts machine learned models by receiving user feedback provided on an analysis of the correctness of the temporal evolution of policies, corresponding explanations and suggestions of the next actions.

FIELD

The present invention relates to applications of Natural Language Processing and Machine Learned models for solving a problem of contextually and semantically understanding legislation changes and the evolution of policies over time to adapt to current external events.

BACKGROUND

Generally, policies are textual documents describing principles to guide decisions and achieve outcomes. Such principles can usually be described in the form of business rules. Examples of policies include car insurance policies, home insurance policies, private health policies, public/state health policies describing coverage and eligibility criteria for its beneficiaries, financials compliance regulations, etc.

A change in a regulation may lead to an updated version of an existent policy, or new bills that replace previous legislation. Often changes in a state policy are published in the form of bulletins describing the amendments done in some of the text or sections in a policy.

Systematic comparison on large policy documents, and in particular, understanding the changes on policy and implications is inherently difficult and time consuming. The regulation and criteria described and the changes in response to an even may vary greatly across local government, and policies in other states, even if for the same service.

SUMMARY

According to an aspect, the present disclosure provides for a system and a method for rapidly and automatically suggesting legislative or regulatory policy changes across multiple dimensions.

The system and method for suggesting policy changes and generating rapid changes in legislation responsive to external events, targets the problem of contextually and semantically understanding legislation changes and the evolution of policies over time to adapt to external events, such as a lockdown or limited gathering law resulting from a public health emergency, e.g., a pandemic (Covid, SARS), a natural disaster (such as a hurricane, tsunami, or earthquake) etc., that can impact an entity, e.g., an individual, cohort or a whole population.

In an embodiment, the method and system supports stakeholders such as service providers, policy makers, and insurers (payers) beneficiaries, in: evaluating response (changes) during the preparedness, response and recovery phases including post-crisis follow up and analysis.

According to an aspect, there is provided a computer-implemented method of automatically generating policy for governing an entity. The method comprises: receiving, by a hardware processor, textual content associated with one or more policies over a period of time, the one or more policies comprising different versions of a policy governing an individual or cohort type over the time period; comparing, by the hardware processor, text content of at least two policy versions over a given time frame; determining, by the hardware processor, based on the comparing, semantic differences between the text content of the at least two policy versions that amount to changes of the at least two policy versions over the period of time; receiving, by the hardware processor, one or more focus events effecting the individual or cohort type over the given period of time; determining, by the hardware processor, based on the semantic differences, a causal link explaining a reason for the changes between the at least two policy versions in the context of a focus event; and presenting, by the hardware processor, an explanation or reason for the changes via a user interface device.

According to one aspect, there is provided a computer-implemented system for automatically generating policy for governing an entity. The system comprises: a memory storage device for storing a computer-readable program, and at least one processor adapted to run said computer-readable program to configure the at least one processor to: receive textual content associated with one or more policies over a period of time, the one or more policies comprising different versions of a policy governing an individual or cohort type over the time period; compare text content of at least two policy versions over a given time frame; determine, based on said comparing, semantic differences between said text content of said at least two policy versions that amount to changes of said at least two policy versions over the period of time; receive one or more focus events effecting the individual or cohort type over the given period of time; determine based on said semantic differences, a causal link explaining a reason for the changes between the at least two policy versions in the context of a focus event; and present an explanation or reason for the changes via a user interface device.

In a further aspect, there is provided a computer program product for performing operations. The computer program product includes a storage medium readable by a processing circuit and storing instructions run by the processing circuit for running a method. The method is the same as listed above.

DETAILED DESCRIPTION

According to an embodiment, the present disclosure provides for a system and a method for rapidly and automatically suggesting legislative policy changes, regulatory policy changes, or other policies/guideline changes of a particular domain across multiple dimensions. The system and method targets the problem of contextually and semantically understanding legislation/regulatory/guideline policy changes and the evolution of such policies over time to adapt to external events, such as a lockdown or limited gathering law resulting from a pandemic (Covid, SARS), a natural disaster (such as a hurricane, tsunami, or earthquake etc.) that can impact a population.

For example, in response to external events such as a case of a public health emergency (such as a lockdown engendered by a pandemic or Covid-19 situation or opioid use disorder crisis) rapid changes in legislation is generated or suggested, for instance in response to how care is delivered (devices, tele-medicine), relaxed medical policy requirements, changes on social services program payments, generation of daily corpus updates globally, etc.

In an embodiment, the method and system supports stakeholders such as service providers, policy makers, and insurers (payers) beneficiaries, in: evaluating response (changes) during the preparedness, response and recovery phases including post-crisis follow up and analysis.

In an embodiment, data processing modules of a computer system compare policies over a given time, identify and explain the changes in the given context of an external event.

In an embodiment, the data processing modules of the computer system compares the evolution of one policy or multiple policies (e.g., comparing updates on policies across two different states since the introduction of a new act) and supports for stakeholders the planning of the next steps or actions to take based on that comparison.

As shown inFIG.1, in the context of automatically suggesting legislative or policy changes across multiple dimensions according to one embodiment, a tool100implementing systems and methods to analyze and explain the temporal evolution of policies, is a computer system, a computing device, a mobile device, or a server. In some aspects, computing device100may include, for example, personal computers, laptops, tablets, smart devices, smart phones, or any other similar computing device.

Computing system100includes one or more hardware processors152A,152B, a memory150, e.g., for storing an operating system, application program interfaces (APIs) and program instructions, a network interface156, a display device158, an input device159, and any other features common to a computing device. In some aspects, computing system100may, for example, be any computing device that is configured to communicate with one or more web-sites125including a web- or cloud-based server120over a public or private communications network99. For instance, a web-site may include text of current and past (historical) legislative policies, e.g., including different versions of legislative policies relating to a particular domain, e.g., healthcare, social services, insurance, national research policies, energy, environment, education. These current and past legislative policies can relate to those promulgated by a government or agency of a particular entity or jurisdiction including but not limited to: countries, provinces, states, cities, counties, towns, municipalities, at all levels. The current and past legislative policies may be structured/unstructured electronic-based textual documents that may be accessed from a pre-determined institution that records/stores such policy and law/legal information over an Internet-based connection. Each current and past legislative policy may include a time-stamp or time indication of when it issued and/or indication that it is a changed version or next iteration of the particular policy or rule. Such structured/unstructured electronic-based textual documents may be stored in a policy or rule database130.

Further, the computing device may further access via a web- or cloud-based server120, electronic documents or text, e.g., electronic document/newspaper article repositories, knowledgebases,135, etc., relating to events that have impacted a particular area or particular jurisdiction, e.g., events that impact the world, or specifically have occurred within or have impacted in any meaningful way a particular country, province, state, city, county, town, municipality, at all governmental level. Such meaningful impact refers to an ability of an occurred event or combination of events to impact a target population such as to have engendered a change in a particular policy or rule of law governing that population in a particular domain in a particular area or jurisdiction.

Further, as shown as part of system100, there is provided a local memory useful for a data processing framework which may include an attached memory storage device160, or a remote memory storage device, e.g., a database, a lexical database, an ontology, accessible via a remote network connection for input to the system100.

In the embodiment depicted inFIG.1, processors152A,152B may include, for example, a microcontroller, Field Programmable Gate Array (FPGA), or any other processor that is configured to perform various operations. Additionally shown are the communication channels140, e.g., wired connections such as data bus lines, address bus lines, Input/Output (I/O) data lines, video bus, expansion busses, etc., for routing signals between the various components of system100. Processors152A,152B are configured to execute method instructions as described below. These instructions may be stored, for example, as programmed modules in a further associated memory storage device150.

Memory150may include, for example, non-transitory computer readable media in the form of volatile memory, such as random access memory (RAM) and/or cache memory or others. Memory150may include, for example, other removable/non-removable, volatile/non-volatile storage media. By way of non-limiting examples only, memory150may include a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

Network interface156is configured to transmit and receive data or information to and from a web-site server120, e.g., via wired or wireless connections. For example, network interface156may utilize wireless technologies and communication protocols such as Bluetooth®, WIFI (e.g., 802.11a/b/g/n), cellular networks (e.g., CDMA, GSM, M2M, and 3G/4G/4G LTE, 5G), near-field communications systems, satellite communications, via a local area network (LAN), via a wide area network (WAN), or any other form of communication that allows computing device100to transmit information to or receive information from the server120.

Display158may include, for example, a computer monitor, television, smart television, a display screen integrated into a personal computing device such as, for example, laptops, smart phones, smart watches, virtual reality headsets, smart wearable devices, or any other mechanism for displaying information to a user. In some aspects, display158may include a liquid crystal display (LCD), an e-paper/e-ink display, an organic LED (OLED) display, or other similar display technologies. In some aspects, display158may be touch-sensitive and may also function as an input device.

Input device159may include, for example, a keyboard, a mouse, a touch-sensitive display, a keypad, a microphone, or other similar input devices or any other input devices that may be used alone or together to provide a user with the capability to interact with the computing device100. In an embodiment, through the user interface, the user can enter specific text versions of policies to be semantically compared, enter a target entity such as an individual user/cohort (e.g., an elderly population over 60 years old) or a whole population impacted by the policies, and/or enter a focus event(s), e.g., a time occurrence of a pandemic that may have engendered a policy change.

With respect to configuring the computer system as an analysis and explanatory tool for detecting the temporal evolution of policies, the local or remote memory160may be configured for temporarily storing or using text data or information162including, but not limited to: specific events, target entity (user/cohorts/populations), policy versions, and other domain knowledge or context data captured from the web server(s).

These data is stored as a database and accessed for use in conducting the analysis and explanation of a temporal evolution of policies in a particular domain, e.g., healthcare, social services, insurance, banking, etc.

The captured data162can be data mined from information stored in the electronic databases130,135or other data sources (not shown). This data may alternately be stored in a separate local memory storage device attached to the computer system100.

As shown inFIG.1, memory150of computer system100further stores processing modules that include programmed instructions adapted to configure the tool to provide an analysis and explanation of a detected temporal evolution of policies for stakeholders (e.g., providers, policy makers, beneficiaries).

In one embodiment, one of the programmed processing modules stored at the associated memory150include a data mining/ingestion module165that provide instructions and logic for operating circuitry to access/read large amounts of data (e.g., structured or unstructured policy text documents, policy versions data, and other domain knowledge or context data captured, from databases, lexical databases, ontologies, and the like) and rendering them in a form for use by other modules that process the data according to the embodiments of the invention. The domain knowledge data or context data for use in understanding what the policy is about is further extracted and used as input for semantic comparison. Such context can include, for example, a benefit, a restriction, a population, a characteristic (e.g., demographic such as gender, age, race, income) or user profile of individuals in the population/cohort, target user/cohorts data, an economy metric, a statistical data about a cohort or population, a relevant time period, or other variables mentioned in the policy for purposes of presentation and semantic comparison.

In an embodiment, a policy alignment module170provides instructions and logic for operating circuitry to receive input text content relating to policies and versions thereof for a relevant time period, compare the extracted text content and align versions of the policies to be compared for evolution determination. In an example embodiment, promulgated policies may not be aligned and happen at different times. Such alignment can involve finding such policies timestamps at the different times and aligning them temporally. For example, state legislative bodies publish bulletins however do not necessarily publish policy updates, amendments or changes at the same time, i.e., one state may implement a change within a delta time interval t1and another state may publish a change to a similar policy at a next delta time interval t2(e.g., 2 months later).

In an embodiment, another programmed processing module stored at the associated memory150of system tool100includes a Natural Language Processing (NLP) tool175to implement operations for analyzing textual, natural language data for use by the other system components. For example, alignment module170invokes the NLP tool175for determining, from the textual content of different policies stored in a database(s) of a particular domain from one or more jurisdictions, each version(s) of a particular policy and its(their) relevant issuance timestamp, and aligning them temporally. The extracted text of policies and versions thereof are associated along a time dimension and pairs or triplets of policies, for example, any number of policies to be compared over a certain time period, are obtained for the given time periods for comparison purposes. A semantic analysis of the text extracted from each policy can be used to determine a similarity of the policies for alignment and semantic comparison purposes.

Another programmed processing module stored at the associated memory150of system tool100include a Semantic Comparator180employing logic and instructions for operating circuitry to compare the semantics of given policy conditions for a target cohort or an individual based on contextual data. Module180in particular compares the conditions of the policies based on the semantics of the corresponding texts. The semantic comparing is focused on the conditions that are relevant for a target entity (e.g., population, cohort or an individual), based on using and understanding contextual data. In an embodiment, the contextual data consists of features describing the target entity (e.g., population, cohort or the individual); such features describing may be collected from a variety of data sources including historical claim data, census data, personal profiles, medical/healthcare records, claims, etc. With this additional contextual information, the features can be mined that can navigate the semantic comparison, i.e., who is user being targeted, and understand the semantics of the policy/version for the target group/user for purposes of comparison.

In one embodiment the policy semantic comparator module180may be implemented as specified in commonly-owned, co-pending U.S. patent application Ser. No.——————entitled “A method/system to contextually compare the semantics of policies conditions for a target cohort or an individual” the whole content and disclosure of which is incorporated by reference as if fully set forth herein.

Another processing module stored at the associated computer memory150includes a temporal differences composer/explainer module190employing logic and instructions for operating circuitry using NLP and machine learned models to infer the link and causality of the changes in policy(ies) in time (between identified delta time intervals t0-t3) based on the events happened around the time of change. The events may be user-specified “focus events” if provided as an input.

Another programmed processing module stored at the associated memory150of system tool100employs instructions to configure the system to build and run supervised (or unsupervised) machine learning (ML) models192for use by the other system components. For example, explainer module190invokes both NLP tool175and a machine learned model for learning differences and/or explanations for different policy changes of one or more policies. The machine learned models192can include, but are not limited to: regression models, knowledge graphs, or neural networks. One of the ML models, can be trained to provide causality explanations or suggestion next action suggestions, based on trained examples that have found a linking of policy changes with respective events or focus events. In an embodiment, supervised learning (classification) techniques using a regression model are used to predict a causality (explanation) of a policy change difference or a next action suggestion associated with an occurrence of a particular event or event type, and/or unsupervised learning (clustering) techniques.

As further shown inFIG.1, memory150includes a supervisory program110having instructions for configuring the computing system100to call each of the program modules and invoke operations of the analysis and explanatory tool100of the temporal evolution of policies described herein. In an embodiment, such supervisory program110calls methods and provides application program interfaces for inputting and aligning of policies, the determining of identified policy changes, the obtaining of context or domain knowledge for a target individual group or cohorts subject of the policies, the building and updating of causality models for linking, using the trained models, policy changes to occurrence of external events and to suggest a next policy change or next action suggestion for a stakeholder. At least one application program interface is invoked to receive input data from a user, e.g., a focus event specification while conducting an analysis of particular policies.

FIG.2Adepicts a system block diagram of a first embodiment of a system200A run by computing system according to the supervisory program110for processing data to analyze and to explain determined temporal evolution of policies using the system tool ofFIG.1.

InFIG.2A, in the first embodiment of the analysis and explanatory tool of the temporal evolution of policies, there is performed the receipt of input data210including a set of text documents or text passages each describing one or more policies to be compared at the policy alignment module170. The set of policies data210in the input documents may be obtained by conducting a search via computer system100, e.g., a keyword search in a particular knowledgebase. This input data210includes versions of policies associated with one or more jurisdictions. The policy alignment module170performs an aligning of the versioned policies211an alignment based on policy information, e.g., its stated topic or purpose, its service or action type and/or any time stamp information or other temporal features data relating to the issuance of the policy version. For each policy document, the policy alignment module170outputs an analysis212of the changes of the policy conditions (e.g., changes in a policy topic or purpose, its service or action type) over time across the different versions of the one policy document.

FIG.3Ashows an example application of the policy alignment module170when aligning several policies. In response to conducting a search, or whether user-specified policies are received input, there is obtained a first set301of versions of a policy P1 issued over time including a first policy P1 text version P1V1305, a second policy P1 text version P1V2306, and a third policy P1 text version P1V3307. The approximate issuance times of each successive policy version P1V1305, P1V2306and version P1V3307is shown within delta time intervals t0, t1, and t2. As shown inFIG.3A, the policy alignment module170aligns several policies by first receiving or determining time intervals351,361,371shown along the alignment time line plot350. For example, as shown inFIG.3B, when comparing time stamps corresponding to their respective issuance times for changes/evolution of the same policy P1, e.g., policies P1V1, P1V2, P1V3 of set301these respective policies are binned according to time intervals351,361,371. The length of time of each time interval351,361,371is configurable or user-specified.

As further shown inFIG.3Adepicting aligning of several policies by application of the policy alignment module170, there is similarly obtained a second set302of versions of a policy P2 including a first policy P2 text version P2V1315and a second policy P2 text version P2V2316binned within respective delta time intervals361,371. A third policy P3 is obtained having a single text version P3V1317is shown binned within time interval371.

FIG.6shows an example automated method implemented600for policy alignment at policy alignment module170. At a first step602, there is received as input, one or policies (e.g, policies301-303ofFIG.3A) who's evolution of changes over time are to be compared. These policies are received as text (data) relating to plural legislative policies associated with a jurisdiction or areas. A keyword or natural language processing search may be used to obtain the relevant policy and versions thereof. As the alignment may be implemented by first determining preconfigured delta time intervals351,361,371, etc. as shown inFIGS.3A,3B. The preconfigured time intervals351,361,371within which to bin the policies obtained from the search is shown along the alignment time line plot350indicating to which time interval a policy(ies) is(are) to be entered for purposes of comparison. The delta time intervals can each be equal in time length or vary in time, e.g., according to variable intervals computed according to an algorithm. Continuing at604,FIG.6there is made a determination as to whether the system received as input a time evolution specification indicating a predetermined time span and/or a specification of a number and length of time intervals delineating relevant time periods within which policies are to be aligned for semantic comparison. For example, if such a time span/time line division scheme specification has been obtained, the method continues to608. Otherwise, if no time evolution input specification has been received at604, the method continues to606where a determination of a time interval delta/span for purposes of aligning policies is made based on relevant context including focus and/or events mined by the event mining module or policy features information. Then, the process returns to steps608,FIG.6.

In one embodiment, at step608, the policy alignment module170can optionally obtain semantic content from text of each policy versions using Policy Semantic Comparator module for use in aligning policies versions within a time interval(s). For example, performed at step608,FIG.6, in a further embodiment, the policy alignment module170aligns the policies versions using an artificial intelligence (AI) system implementing machine-learned models that can take into account “focus events”. For example, states in the United States do not publish changes at the same time, i.e., one state may implement a policy change: “schools closure” at the time t1and another state may publish a change: “the schools have to reduce the number of students to X” at t2(e.g., 2 months later) where both are caused by the focus event (e.g., a pandemic). Thus, the policy alignment module170may need information from policy semantic comparison to identify the time intervals for purposes of aligning these example state policies. In such an embodiment, as shown inFIG.4, policy alignment module170obtains at410a semantic comparison of versions for each pair of policies (from Policy Semantic Comparator module180), e.g., to identify the services/topics of changes and group them in a time interval at415, based on the service/action type/topic/timestamp.

Returning to610,FIG.6, the module performs aligning of policies/policy versions to within respective time intervals using policy features or AI algorithms (e.g., transformers, deep neural networks) and NLP tools for alignment. Additionally, an algorithm based on semantic comparison of versions based on the focus event features can be employed for alignment. That is, in an embodiment, the policy alignment is driven by a focus event or another “important” event mined by external event mining module160. For example, if a focus event is defined, alignment process would consist of a) sending different pairs of policies versions that are close on the time line (varying the version combinations) to the semantic comparator module along with focus event contextual data, and then b) based on the semantic comparison results, selecting the pairs of policies that suit most focus event features. Alignment may include obtaining the policy version every predetermined time period in the time line, or alignment may include obtaining the relevant policy versions for a period of time intervals before and after a time of a determined focus event occurrence. It is the case that a policy may not necessarily change in two or more consecutive time intervals.

An example policy alignment performed at step610is shown in the example depicted inFIG.3Ashowing that while the initial first policy P2 version P2V1315was issued at or near time interval delta t1such that the initial alignment of first policy version P2V1315is within time interval361, according to comparison of an extracted policy feature(s) such as its service, action type, stated topic or purpose, it is determined that policy P2V1 can also align with first policy version P1V1 in time interval351given the importance of the received or specified delta time intervals for purposes of determining a temporal evolution over a time span. Thus, for better comparison of its content to the content of first policy P1 version P1V1305, the alignment by policy alignment module170is performed by binning the content of existing first policy P2 version P2V1315as a new first policy P2 version P2V1325in first time alignment interval351including delta time interval t0. Similarly, while the initial first policy P3 version P3V1317was issued at or near delta time interval t2such that the initial alignment of existing first policy version P3V1317is within time interval371, according to its context or policy feature, e.g., stated topic or purpose, policy P3V1, can also align for content comparison purposes with the content of the second version of first policy P1, i.e., P1V2306and also existing policy P2V1315in time interval361, given the importance of the given delta time intervals for purposes of determining a temporal evolution. Thus, the alignment performed by policy alignment module170performs aligning the existing first policy P3 version P3V1317to within the same interval of policy version P1V2 and P2V1 as new policy P3 version P3V1327in second time alignment interval361delta t1.

Whether aligning for comparing changes/evolutions of several policies (FIG.3A) using a semantic comparison of versions for each policy, or for comparing changes/evolution of a single policy (FIG.3B), the alignment time line plot350indicates a time of the event or focus event380which may be determined to have engendered each of the subsequent policy version changes within a span of successive time intervals.

Returning to system diagram ofFIG.2A, the output212of the policy alignment module170includes the set of policies to compare and their corresponding respective time intervals, with each policy potentially having several versions evolving over time as shown in a time line plot350. This set of policies and time intervals is input to the policy semantic comparator module180.

Additionally received as input to the policy semantic comparator module180is data215representing a relevant target user, e.g., a single user302, a cohort of users304, to which the policies and versions thereof are directed.

Additionally received as input to the policy semantic comparator module180is data217representing policy domain knowledge. The policy domain knowledge data217is data relevant to policies and versions thereof such as data from ontologies, knowledgebases, terminologies, and cost of services covered by policies, etc., relating to implementations of the policy/version(s). Further policy domain knowledge data217can include data related to the events, e.g., ontologies, Knowledge Graphs (KG), terminologies, rules, etc. and can include temporal background data, e.g. historical event records, descriptions, or domain knowledge associated with the events, that can assist in understanding the semantics of the policy or domain of the policy (e.g., auto insurance (car insurance domain), medical insurance domain, etc.)

Additionally received as input to the policy semantic comparator module180is any relevant contextual data219that can be used to explain any relevant policy changes. The relevant contextual data219can include data associated with the particular target user. For example, for the cohort (population), relevant contextual data219can include a claim and/or other operational data for the cohort, census and/or other statistical data describing population, e.g., of a state or a country. For a single user, relevant contextual data219can include claims, operational data, user profile, preferences, or other type of contextual data related to the user. Contextual data can include any further data relating to a user or cohort that permits semantic comparisons and temporal inferencing.

For policy comparison purposes, a focus event(s) is a description of an event(s) the system must focus on in addition to user/cohort contextual data when comparing the policies. Such events or focus events can include a natural disaster or a pandemic.

FIG.7shows an embodiment of an example automated method700carrying out operations to semantically compare the policies aligned within each time interval of a temporal evolution time span. The steps ofFIG.7are performed at each time interval702of a time span. At a first step704, there is received the input data, e.g., policy domain knowledge, any relevant contextual data, any focus events, and the set of policies to compare, which may include identifiers of the particular policy and/or version thereof that has been aligned in the current time interval. Then, at707,FIG.7, there is performed a search within each policy being compared in the selected time interval to identify relevant sections for semantic matching. Once potential sections are identified, at710there is performed, at the NLP data processing tool175, attempts to semantically match the identified sections. The matching may entail representing the identified policy sections as two graph-like structures (e.g., in a data base or ontology) and implement a matching operator which identifies those nodes in the two structures that correspond semantically to each other. For example, applied to policy, section or title, the operator can identify a word “wage” is semantically equivalent to another policy section indicating word “salary” because they are synonyms. This information may be taken from a lexical database.

In an embodiment, the policy semantic comparator module180at step710further compares the semantics of given policies conditions for a target cohort or an individual based on the contextual data. The comparison of the policies conditions is based on the semantics of the corresponding (policy) texts211, and it is focused on the conditions that are relevant for a target cohort or an individual, based on using and understanding contextual data. The contextual data consists of features describing the target cohort or the individual. Such describing features may be collected from a variety of data sources including historical claim data, census data, e.g., for policies governing a population, personal profiles, medical records, e.g., claims data for policies governing car insurance domain, etc. . . . . The contextual comparison of policies conditions is associated with a human interpretable explanation of why such comparison/conditions are relevant/important for the given target cohort or individual, where the explanation is based on the semantics of the text describing the policies conditions and the contextual features of the target cohort or individual.

In an embodiment, the policy semantic comparator module180performs a semantic comparison between the text (policy) section inputs in each time interval. For example, returning toFIG.3A, the policy semantic comparator module180contextually compares the semantics of policy P1V1305with aligned policy P2V1325aligned in time interval351; contextually compares the semantics of policy P1V2306with aligned policy P3V1327aligned in time interval361; and contextually compares the semantics of policy P1V3307with aligned policy P2V2 aligned in time interval371. Further, as shown inFIG.3B, the policy semantic comparator module180contextually compares the semantics of same policy P1, i.e., policies P1V1305, version P1V2306, and third policy version P1V3307.

Finally, at710, assuming the compared policy versions in the current time interval are semantically matched, those matched sections are flagged if of interest for the target user/group for output with the associated time interval. Those flagged policy sections can be further processed to determine the semantic differences for each policy in the current time interval to be output. Such processing includes employing the NLP tool175to create rules for semantically comparing the texts of two or more aligned policies in an interval, i.e., dependent upon information in the policy text, rules are used to make similarity comparisons. For example, such rules can be applied to determine whether the matched policy texts are directed to same policy conditions, or whether conditions have different values, e.g., a policy is to close a school vs. reduce numbers of students attending school.

Continuing at713,FIG.7, or at any time during or at the end of the analysis and temporal policy change evolution process, a determination can be made as to whether the system has received any user feedback regarding the choice of policy selections based on the matched sections. If there is negative user feedback received at713, then the method proceeds to step716to determine from the feedback whether new policy/policy sections are needed for better or more accurate semantic alignment/matching. Then at720, based on the feedback, there can be performed a realignment of the policies within the time intervals of the time frame, and/or the identification of different policies to search and align and/or identification of different sections within the existing policies for better matching, and or an identification of a different time interval lengths or span for determining policy evolution, and the process returns to step702,FIG.7for repeating of the semantic matching steps in selected time intervals or all time intervals. The feedback received is also used to refine the ML models of the system and can be applied for both current comparison and for future comparisons.

Otherwise, if it is determined at step713that no user feedback has been received, then a determination is made at722as to whether the current time interval of the time span is the last time interval having polices for comparison. If the current time interval is the last time interval, then the next time interval of the time span is identified at725,FIG.7, and the process returns to repeat the semantic comparison steps704-713.

Otherwise, at722, if it is determined that the current time interval is the last time interval, then the process proceeds to step730, where the policy semantic comparator module180outputs a set250of the semantic differences/changes between the aligned policies' versions matched in each time interval is output.

The method steps ofFIG.7are repeated for each time interval of the specified or relevant time evolved span. Once all policies are semantically compared as determined at722in each of the relevant time intervals, at step730, the process provides output of the policy differences to the next processing module. In an embodiment, the policy semantic comparator module180further performs at730a ranking by importance of the set of differences/changes between the policy versions in relation to the target user/cohort. Thus, the presented outputs shows the differences/changes ranked by importance in relation to the target user/cohort.

In an embodiment, after performing the contextual comparisons of the semantics of policies (versions) within each respective time interval(s), the policy temporal differences composer and explainer module190receives as input, the output set250of the semantic differences/changes between the aligned policies' versions and runs one or more of predictive models, regression models, KGs, neural networks, etc. to infer the link and causality of the changes in policy(ies) in time (between identified) semantic changes based on the events that have occurred around the time of change. In an embodiment, such a predictive model or like machine-learned model can be trained based on historical events data and how they have effected similar policy changes in the past.

Further received as input data220to the policy temporal differences composer and explainer module190from the data mining/ingestion module165is events data searched and obtained from a knowledgebase or the like. The events data220or focus event represent events that have occurred at such a time can be determined by the system as having been potentially a factor or the sole factor in triggering a relevant policy change(s) or policy version iteration of the received set of policies210.

The policy temporal differences composer and explainer module190further receives mined events input data220. For example, the data mining module165mines historical events from external data sources (e.g. Wikipedia, events knowledge graphs, etc.), and extracts important features for example using NLP tools it may extract events, features and relations and form a knowledge graph (KG), based on the events' domain knowledge (e.g. ontologies, KGs, models, etc.) and/or manually identified types, relations, etc.

In an embodiment, to obtain events, the modules,180,190may implement event data mining techniques and algorithms as described in the book entitled “Event Mining: Algorithms and Applications” by Tao Li, CRC Press (2016) the contents and disclosure thereof being incorporated herein by reference. Mined event or event focus data includes a set of information including, but not limited to, one or more of: the relevant date/time, location, description, tags, number of participants, event duration, etc.

Optionally provided as inputs to the policy temporal differences composer and explainer module190are user-specified “focus events”260. If a focus event is not specified, the system will work with events mined by event mining module.

FIG.8shows an automated method800carrying out operations to compose and explain the reasons for policy change differences that have evolved over a temporal span at policy temporal differences composer and explainer module190. In an embodiment, the module receives as input a given mined event data or a user-specified “focus event” data used to determine whether specific policy changes are correlated (i.e., causal linked) to the occurrence of such events/focus event over time that may have impacted the changes. Thus, as indicated at802, for one or a combination of received event(s) or focus event(s), the following steps are applied.

At step804, there is received as input: the output set or passages250of the semantic differences/changes between the aligned policies' versions (e.g., doubles or triples) of one or more policies in each time interval. These semantic differences/changes between the aligned policies' versions are received as text (data) passages of the critical semantic content associated with the particular time interval. Additionally received at804is event domain knowledge relating to the current event/focus event. Event domain knowledge265obtained from ontologies, knowledgebases, terminologies, etc. and is used for event and policy changes feature extraction and linkage among them.

Continuing at807,FIG.7, there is made a determination whether a link or causality of the changes in policy(ies) in time (between identified time instances) can be inferred based on their relevance to the user/cohort and the events or focus events happening around the time of the change.

In an embodiment, the inferring whether there is a causal link between the changes (semantic differences) in policy(ies) in time (between identified time intervals) can include comparing the semantic changes in policy based on event/focus event features if provided in the input. At807, policy temporal differences composer and explainer module190can additionally receive historical events data that may have impacted policies as an input from the Event Mining Module165and the knowledge can be used to infer a possible link between the changes and those historical events. In one embodiment, to perform the inferencing and/or linking, the policy temporal differences composer and explainer module190semantically compares the event features and policy changes. Additionally, event domain knowledge such as e.g., ontologies, knowledge graphs, terminologies, etc. is used for determining causal linkage of events to policy changes. For example, a predefined set of domain knowledge rules can be applied that governs known specific types of events to specific policy changes. In such processing, a particular change can be caused by an event which may be determined as a background event, and however determined, for example, not applicable for a target user/cohort. These link inferences are used by the composer module190to set forth an explanation based on their relevance to the user/cohort, the explanation including, but not limited to: this policy change was caused by this event or focus event for a reason based on a particular rule or knowledge graph link, or because historical data shows a pattern of behavior or changes. Such types of predictive models are used to build up the explanations to identify events of the contextual data of the target user that has influenced the policy change.

Continuing to step810,FIG.8, for the current event and set of policy differences, a determination is made as to whether there is inferred any causal link between them that is relevant to the target user/group. If there is determined no causal link that is relevant, then the process proceeds to step818to determine whether any user feedback has been received that may impact the comparison/explanation results. Otherwise, if there is determined a causal link exists that is relevant, then the process proceeds to step815where module190performs collecting data of the linking and composing the explanation for policy change based thereon. The module then can present results on a timeline for interface display output before proceeding to step818.

In an embodiment, the policy temporal differences composer and explainer module190arranges the composed collected data and explanations results on the timeline and displays it in a form of easily consumable by the end-user. For example, as shown inFIG.2A, a set276of differences/changes and explanations of the causality engendering the policy version change in relation to the target user/cohort is presented as output270and can be visualized in timed relation along a timeline350. The explanation of changes over time is particularly causally linked (inferred) to either one or more of focus events380or other events happening around that time, or both, based on their relevance to the user/cohort. Evidences/explanations for each identified difference can be further augmented with relevant context data, e.g., an averaged cost of service (e.g., of a surgery as published by a state) or any other information that can be used to reflect/estimate and impact. For example, as shown inFIG.2A, for each time interval the comparison of aligned policy versions based on the found context, is presented as shown over time, to depict an evolution of changes. The evolution of targeted features in the policy over time is visualized. The explanation of each comparison is provided relative to the focus event.

As the output270data including a visualization of explanations describing the causality or linking of events that have occurred at such a time to have triggered the relevant policy changes is presented in timed relation along a timeline350for visualizing272by a user275, the user275can agree and accept that the comparison of the aligned policies (e.g., doubles or triplets) per time interval are correct or viable based on the policy features/context. The user275can also determine the correctness of the linking or causality found between the event and policy change for the particular policy or whether the mined event that was input was correct/relevant and provide this as feedback280into the system.

Thus, returning to step818,FIG.8, or at any time during or at the end of the analysis and temporal policy change evolution determining process, a determination can made as to whether the system has received any such user feedback. In an embodiment, feedback is provided by the user based on the analysis of the temporal evolution of policies, corresponding explanations and suggestions of the next actions. Feedback can be expressed in any form, such as an up-vote, down-vote, etc. If there is user feedback received at818, then the method proceeds to step822to apply feedback on the system models (e.g. predictive models, regression models, KGs, neural networks, etc.) for refining those models, and further to propagate the user feedback280to other modules. For example, such propagated user feedback280can be feedback to Policy Semantic Comparator180of the system so that the policy comparison may be changed based on other or different policy features extracted, or different target user/cohort.

For example, at822, based on the user feedback determining that a given mined event that was input was incorrect/irrelevant, there can be performed an identification of a different event(s)/focus event(s), and the process can return to step802,FIG.8for repeating of the causal inference determining and composing/explaining steps for the new or different focus event.

Otherwise, returning to step818, if it is determined that no user feedback has been received, then a determination is made at820as to whether any further focus events can be attempted for causal inferencing to specific policy changes. If there is no additional focus event for inference of linking, the process ends. Otherwise, if it is determined that there is additional focus events for inference linking, a next focus event is identified at825,FIG.8, and the process returns to repeat the causal linking and explaining steps804-818for the next identified event. The method steps ofFIG.8are thus repeated for each input event/focus event.

FIG.2Bdepicts a further embodiment of the analysis and explanatory tool of the temporal evolution of policies method200B implemented by computer system100. The system200B is similar to the system200A ofFIG.2Ain the respect of the functionality of policy alignment module170, policy semantic comparator module180and policy temporal differences composer and explainer module190. The system200B ofFIG.2Bfurther includes the suggestion module195that receives the target user/cohort data215, the contextual data219, the events data220, and the output visualization data and explanations270describing the causality or linking of events to the relevant policy changes output from the policy temporal differences composer and explainer module190and feedback280from the user and processes this data using a recommender model to output any suggested next changes290for the policy of interest considering the target user/group. The suggestion module195uses a recommender model to receive the detected changes in the policies including the detected explanations as to why the change has happened, and based on that information, determine and output a suggestion such as a next most likely change to one policy. For example, for a current policy, the system looks at and compares similar policies from the past and corresponding detected changes that had happened and when, and thus the recommender model can build a suggestion(s) of such change(s) for a current model given a similar event/context.

The suggestion module195invokes a recommender system running a machine-learned recommender model and the features for the recommender model are extracted from contextual data and event data, depending on what impact the system targets to minimize or maximize (e.g., in a pandemic scenario: the impact on the lockdown vs. high immunity strategy on the number of deaths, the transmission rates, was it adopted by the public, etc.). In addition the suggestion module195captures user feedback280and applies it on its recommender models. Such user feedback280returned to the suggestion module195may be whether a particular next policy change suggestion is good or not good so as to refine the policy and refine the AI prediction.

FIG.5shows an example application of the computer-implemented tool100for analysis and explanation of temporal change policies. InFIG.5, the application of the analysis and explanatory tool of temporal change policies is for an example application to analyze and explain changes to wage and unemployment subsidies policies in force between 2018 and 2020 in two countries: Country #1 (C1) vs. Country #2 (C2). The input set of policies to compare may be obtained at a governmental web-site in each country. As shown, country C1 policy P1 on wage and unemployment subsidies501include a first, second and third versions P1V1, P1V2, P1V3 issued in respective time intervals551,561,571defined between time deltas t0-t1, t1-t2and t2-t3. As shown inFIG.5, country C1 policy P1V3 is issued in time interval571and is in effect at least through a further fourth time interval between times t3-t4. Further, in the example, country C2 policy P2 on corresponding wage and unemployment subsidies502includes a first and second version P2V1, P2V2. Country C2 policy P2V1 is in effect at least through successive time intervals551and561, while country C2 policy P2V2 has issued in time interval571and is in effect through at least through the further fourth time interval581.

Further, in the example the input target user515is a population of the respective countries C1, C2. Data mining module165has further detected a global pandemic event and resulting lockdown and closing of businesses and the limiting of gatherings. Alternatively, or in addition, a “focus event” specifying this information, e.g., a global pandemic event and resulting lockdown and closing of businesses, is entered by a user as input into the system. The analysis system will be oriented to focus on the input focus events in addition to user/cohort contextual data when the policies are being compared/explained.

In the example, the modules of the tool receive the contextual data associated with the target users (populations of country C1 and of country C2) and further can receive a context, e.g., an economy metric, e.g., the Gross Domestic Product (GDP), and the population count (census) of each of countries C1, C2. For the cohort (population), other operational data for the cohort, e.g., other statistical data describing the population of a state or a country can be input. For example, for a single user, contextual data may be claims, operational data, user profile information, preferences, or other type of contextual data related to the user.

As shown inFIG.5, in the example, the policy alignment module170semantically compares and aligns versions of the policies as shown along timeline output550. For example, within the 1 year time interval551, text content such as the title, topic or indicated service or action of policies P1V1 and P2V1 are compared and are indicated as being aligned. Within the next year time interval561, text content as the title, topic or indicated service or action of policies of P1V2 and P2V1 are aligned for subsequent semantic comparison purposes. Within the third year time interval571, text content as the title, topic or indicated service or action of policies of P1V3 and P2V2 are compared and indicated as being aligned for subsequent semantic comparison purposes. Within the next year time interval581, text content of policies of P1V3 and P2V2 are compared and deemed aligned for subsequent semantic comparison purposes.

As shown inFIG.5, in the example, the policy semantic comparator module180receives the input focus events (either mined or entered by a user) and the module generates for visualization on time line550respective event focus time stamp indicator for country C1 and similarly for country C2. Further, the policy semantic comparator module180generates a list of semantic differences/changes between the aligned policies' versions in each corresponding time interval. From this depiction, an evolution of policy changes over the time period is created.

As shown inFIG.5, in the example, the policy semantic comparator module180receives the domain context data525, e.g., the GDP of each country C1, C2 and plots the combined GDP of each country C1, C2 along the Y-axis as a function530of the time (X-axis) spanning across all time intervals.

For the input example analyzing temporal changes in policies governing unemployment and wage subsidies over time, the detected semantic differences/changes between the aligned policies' versions P1V1, P1V1 in time interval551would reveal that subsidies were introduced first (as being closest to time t0) in country C2 followed by subsidies introduced in country C1, for a target cohort of individuals unemployed for a time greater than 6 months. The detected semantic differences/changes between the aligned policies' versions P1V2 in time interval561would reveal that subsidies introduced for the population in country C1 has changed to govern a target cohort of individuals having been unemployed for a reduced time greater than 3 months.

Further, the detected semantic differences/changes between the aligned policies' versions P1V3, P2V2 in time interval571would reveal the increase of wage subsidies in both countries C1, C2. Further semantic differences detected would reveal that the target cohort population subject to the new wage subsidy has changed to any unemployed individual, or individuals employed but in a furlough scheme.

Further, the detected semantic differences/changes between the aligned policies' versions P1V3, P2V2 in time interval581would reveal that the target cohort population subject to the new wage subsidy has changed to individuals unemployed for a time space greater than 12 months.

The policy temporal differences composer and explainer module190performs an analysis using a regression model to determine a causality that explains the reason for the unemployment subsidy changes indicated in the policy versions over time. For example, analysis at explainer module190shows unemployment benefits are relaxed when the GDP530dips (e.g., such as shown by the version changes found near time t0in time intervals551and version changes detected near time t2in time interval571), unless the wider and prolonged effects of/after the COVID-19 lockdown mean that the economy is not able to support as many people such as shown in the time interval581beginning at delta time (t3). This is shown as evidences and explanations output570via a user interface display.

When configured according to the embodiment of the system200B ofFIG.2B, for the example application depicted inFIG.5, the suggestion module195performs an analysis using a recommender system model to determine suggestions based on the comparisons over time. Given the example depicted inFIG.5, the system100may suggest that “based on the evolution of the unemployment policies in other countries that were put in place before, and considering the similar context (e.g. population distribution, size, health system, etc.) and impact results, the system can suggest to restrict coverage of the unemployment benefit by x %” as shown as a suggestion output580via a user interface display.

In embodiments, the tool100could be used as part of a portal, chatbox or FAQs to explore the evolution of policy across multiple programs, state policies, federal policy in response to an event or a crisis. For example, the present invention can be implemented as a post-crisis follow up and after-action review of the responses carried out during the crisis to understand lessons learned, and feedback into the preparation stage for future emergency responses. In addition, the system tool supports policy makers on the planning next steps to prevent and/or have an earlier response to reduce the impact to an event.

The system tool100can also provide suggestions based on understanding and explaining past policy changes and their impact for similar context, or by comparing policies from other states/countries that implemented before changes in policy in response to same or similar event (e.g., a previous pandemic, in preparation for a global recession, etc.) and based on receipt of the context associated with any policy changes effects. Further another knowledgebase or databases may be mined to determine the occurrence of an event(s) has taken place of such meaningful influence to have affected an area(s) or a target group(s) of people and that engendered a particular policy change.

In this manner, policy makers can better understand how similar policies or policies in other countries/other domains have changed over time and explain why these changes had occurred at the particular time. This information, can be used to suggest how policies they control can change, e.g., if similar types of events happen, or based on contextual data common to what prefaced prior policy changes.

FIG.9illustrates an example computing system in accordance with the present invention. It is to be understood that the computer system depicted is only one example of a suitable processing system and is not intended to suggest any limitation as to the scope of use or functionality of embodiments of the present invention. For example, the system shown may be operational with numerous other general-purpose or special-purpose computing system environments or configurations. Examples of well-known computing systems, environments, and/or configurations that may be suitable for use with the system shown inFIG.9may include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the above systems or devices, and the like.

In some embodiments, the computer system may be described in the general context of computer system executable instructions, embodied as program modules stored in memory16, being executed by the computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, and so on that perform particular tasks and/or implement particular input data and/or data types in accordance with the present invention (see e.g.,FIGS.6-8).

The components of the computer system may include, but are not limited to, one or more processors or processing units12, a memory16, and a bus14that operably couples various system components, including memory16to processor12. In some embodiments, the processor12may execute one or more modules11that are loaded from memory16, where the program module(s) embody software (program instructions) that cause the processor to perform one or more method embodiments of the present invention. In some embodiments, module11may be programmed into the integrated circuits of the processor12, loaded from memory16, storage device18, network24and/or combinations thereof.

The computer system may also communicate with one or more external devices26such as a keyboard, a pointing device, a display28, etc.; one or more devices that enable a user to interact with the computer system; and/or any devices (e.g., network card, modem, etc.) that enable the computer system to communicate with one or more other computing devices. Such communication can occur via Input/Output (I/O) interfaces20.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. The corresponding structures, materials, acts, and equivalents of all elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the present invention has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the invention in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the invention. The embodiment was chosen and described in order to best explain the principles of the invention and the practical application, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated.

Characteristics are as follows:

Service Models are as follows:

Deployment Models are as follows:

Workloads layer90provides examples of functionality for which the cloud computing environment may be utilized. Examples of workloads and functions which may be provided from this layer include: mapping and navigation91; software development and lifecycle management92; virtual classroom education delivery93; data analytics processing94; transaction processing95; and processing96to automatically analyze and explain temporal evolution of policies according to aspects of the present disclosure.