Search engine with privacy protection

A search engine system with privacy protection, including a data indexer configured to create an index of data, a search engine configured to search the index of the data in response to a query, and create a search result set including excerpts from the data, and a privacy protector configured to identify at least one data entity within at least one excerpt of the search result set that meets at least one predefined entity extraction criterion, redact the search result set by removing the data entity from the excerpt, and present the redacted search result set on a computer output device.

FIELD OF THE INVENTION

The invention relates to computer data access control and privacy in general, and more particularly in relation to search engines.

BACKGROUND OF THE INVENTION

Many companies strive to make data in one part of the company available to employees in other parts of the company to maximize the use and benefit of the data. While search engines are excellent tools for indexing company data and allowing the data to be searched, access to such data is often restricted in accordance with a predefined access control model, allowing employees to access data to which they are authorized, while preventing them from accessing data to which they are not authorized. As such, managers of company search engines must choose between excluding restricted information from the search engine index, potentially depriving employees from learning about useful company data which they may be authorized to access, or including restricted information in the search engine index, potentially exposing employees to company data, even in abbreviated form in a search result set, which they may not be authorized to access.

SUMMARY OF THE INVENTION

The invention in embodiments thereof discloses novel systems and methods for searching access-controlled documents that include restricted information and providing search results therefor.

In one aspect of the invention a search engine system with privacy protection is provided, the system including a data indexer configured to create an index of data, a search engine configured to search the index of the data in response to a query, and create a search result set including excerpts from the data, and a privacy protector configured to identify at least one data entity within at least one excerpt of the search result set that meets at least one predefined entity extraction criterion, redact the search result set by removing the data entity from the excerpt, and present the redacted search result set on a computer output device, where any of the data indexer, search engine, and privacy protector are implemented in either of computer hardware and computer software embodied in a computer-readable medium.

In another aspect of the invention the data indexer is configured to create a set of excerpts from the data independent from the processing of queries by the search engine, where the privacy protector is configured to identify at least one data entity within the set of excerpts that meets at least one predefined entity extraction criterion, and pre-redact the set of excerpts by removing from the set of excerpts the data entity identified in the set of excerpts, and where the search engine is configured to create the search result set from the pre-redacted set of excerpts in response to the query.

In another aspect of the invention the data indexer is configured to identify at least one data entity within the index that meets at least one predefined entity extraction criterion, and redact the index set by removing the data entity from the index.

In another aspect of the invention a method is provided for providing privacy protection in a search engine system, the method including searching a index of data in response to a query, creating a search result set including excerpts from the data as a result of the searching, identifying at least one data entity within at least one excerpt of the search result set, where the data entity meets at least one predefined entity extraction criterion, redacting the search result set by removing the data entity from the excerpt, and presenting the redacted search result set on a computer output device.

In another aspect of the invention the method further includes creating a set of excerpts from the data independent from the processing of queries by a search engine, identifying at least one data entity within the set of excerpts that meets at least one predefined entity extraction criterion, and pre-redacting the set of excerpts by removing from the set of excerpts the data entity identified in the set of excerpts, where the step of creating a search result set includes creating the search result set from the pre-redacted set of excerpts in response to the query.

In another aspect of the invention a system is provided for classifying data in accordance with data access reasons, the system including a predefined access reasons database having a plurality of predefined access reasons, and a data classifier configured to create a classification index of data by identifying an association between any of the data and at least one of the predefined access reasons, and storing the association in the classification index, where any of the predefined access reasons database and data classifier are implemented in either of computer hardware and computer software embodied in a computer-readable medium.

In another aspect of the invention the predefined access reasons database includes at least one predefined association between any of the predefined access reasons and at least one predefined data entity type, and where the predefined association indicates that access is permitted to any of the data that includes a data entity of the data entity type.

In another aspect of the invention the data classifier is configured to identify the association by identifying at least one data entity within any of the data, where the data entity is of any of the predefined data entity types, and associating any of the data that includes the identified data entity with the access reasons that is associated with the data entity type of the identified data entity.

In another aspect of the invention the data classifier is configured to identify the association by determining that the data matches a statistical profile of a sample data set associated with the access reason.

In another aspect of the invention the system further includes a data access request logger configured to require the selection of an access reason from the database in association with a query to be processed regarding the data, and a privacy protector configured to eliminate from a search result set resulting from the processing of the query any search result excerpt except where access to the data underlying the search result excerpt is permitted for the selected access reason as indicated by the association in the classification index between the underlying data and the selected access reason.

In another aspect of the invention the system further includes a data access request logger configured to require the selection of an access reason from the database in association with a query to be processed regarding the data, and a privacy protector configured to identify at least one data entity within at least one excerpt of a search result set resulting from the processing of the query, where the data entity meets at least one predefined entity extraction criterion, and redact the search result set by removing the data entity from the excerpt unless the data entity is of a data entity type that is associated with the selected access reason as indicated in the predefined access reasons database as permitting access to the data entity type.

In another aspect of the invention further includes a data access request logger configured to require the selection of an access reason from the predefined access reasons database if the underlying data of any excerpt in a search result set resulting from the processing of a query is associated with any access reason in classification index, and a privacy protector configured to eliminate from the search result set any search result excerpt except where access to the data underlying the search result excerpt is permitted for the selected access reason as indicated by the association in the classification index between the underlying data and the selected access reason.

In another aspect of the invention the data access request logger is configured to present for selection only those of the access reasons that are associated with the underlying data of the search result set as indicated in the classification index.

In another aspect of the invention the system further includes a privacy protector configured to identify at least one data entity within at least one excerpt of a search result set resulting from the processing of a query, where the data entity meets at least one predefined entity extraction criterion, and redact the search result set by removing the data entity from the excerpt, and a data access request logger configured to allow the selection of an access reason from the database in association with the query to be processed regarding the data, and restore the data entity that was previously removed from the excerpt of the search result set if the data entity is of a data entity type that is associated with the selected access reason as indicated in the predefined access reasons database as permitting access to the data entity type.

In another aspect of the invention a method is provided for classifying data in accordance with data access reasons, the method including providing a plurality of predefined access reasons, identifying an association between a unit of data and at least one of the predefined access reasons, and storing the association in the classification index.

In another aspect of the invention the providing step includes providing at least one predefined association between any of the predefined access reasons and at least one predefined data entity type, where the predefined association indicates that access is permitted to any of the data that includes a data entity of the data entity type.

In another aspect of the invention the identifying step includes identifying at least one data entity within any of the data, where the data entity is of any of the predefined data entity types, and associating any of the data that includes the identified data entity with the access reasons that is associated with the data entity type of the identified data entity.

In another aspect of the invention the identifying step includes determining that the data matches a statistical profile of a sample data set associated with the access reason.

In another aspect of the invention the method further includes requiring the selection of an access reason from the access reasons in association with a query to be processed regarding the data, and eliminating from a search result set resulting from the processing of the query any search result excerpt except where access to the data underlying the search result excerpt is permitted for the selected access reason as indicated by the association in the classification index between the underlying data and the selected access reason.

In another aspect of the invention the method further includes requiring the selection of an access reason from the access reasons in association with a query to be processed regarding the data, identifying at least one data entity within at least one excerpt of a search result set resulting from the processing of the query, where the data entity meets at least one predefined entity extraction criterion, and redacting the search result set by removing the data entity from the excerpt unless the data entity is of a data entity type that is associated with the selected access reason as indicated in the predefined access reasons as permitting access to the data entity type.

In another aspect of the invention the method further includes requiring the selection of an access reason from the predefined access reasons if the underlying data of any excerpt in a search result set resulting from the processing of a query is associated with any access reason in classification index, and eliminating from the search result set any search result excerpt except where access to the data underlying the search result excerpt is permitted for the selected access reason as indicated by the association in the classification index between the underlying data and the selected access reason.

In another aspect of the invention the method further includes identifying at least one data entity within at least one excerpt of a search result set resulting from the processing of a query, where the data entity meets at least one predefined entity extraction criterion, redacting the search result set by removing the data entity from the excerpt, allowing the selection of an access reason from the access reasons in association with the query to be processed regarding the data, and restore the data entity that was previously removed from the excerpt of the search result set if the data entity is of a data entity type that is associated with the selected access reason as indicated in the predefined access reasons as permitting access to the data entity type.

In another aspect of the invention a computer program product is provided for providing privacy protection in a search engine system, the computer program product including a computer readable medium, and computer program instructions operative to search a index of data in response to a query, create a search result set including excerpts from the data as a result of the searching, identify at least one data entity within at least one excerpt of the search result set, where the data entity meets at least one predefined entity extraction criterion, redact the search result set by removing the data entity from the excerpt, and present the redacted search result set on a computer output device, where the program instructions are stored on the computer readable medium.

In another aspect of the invention a computer program product is provided for classifying data in accordance with data access reasons, the computer program product including a computer readable medium, and computer program instructions operative to provide a plurality of predefined access reasons, identify an association between a unit of data and at least one of the predefined access reasons, store the association in the classification index, require the selection of an access reason from the access reasons in association with a query to be processed regarding the data, and eliminate from a search result set resulting from the processing of the query any search result excerpt except where access to the data underlying the search result excerpt is permitted for the selected access reason as indicated by the association in the classification index between the underlying data and the selected access reason, where the program instructions are stored on the computer readable medium.

DETAILED DESCRIPTION OF THE INVENTION

The invention is now described within the context of one or more embodiments, although the description is intended to be illustrative of the invention as a whole, and is not to be construed as limiting the invention to the embodiments shown. It is appreciated that various modifications may occur to those skilled in the art that, while not specifically shown herein, are nevertheless within the true spirit and scope of the invention.

Reference is now made toFIG. 1, which is a simplified conceptual illustration of a search engine system with privacy protection, constructed and operative in accordance with an embodiment of the invention. In the system ofFIG. 1, a data indexer100creates an index102of data104, where index102is created for use by a search engine106in accordance with conventional techniques except as otherwise described herein. Data104preferably includes any kind of data that are indexable into index102for use by search engine106, such as database records and word processing documents. Where access to data104is restricted, such as where access to data104is controlled by an access control system (not shown), data indexer100is preferably given unrestricted access to data104or is otherwise provided with data104, such as by a crawler (not shown) that has unrestricted access to data104.

When search engine106queries data104in response to a query, search engine106creates a search result set108which includes excerpts from data104in accordance with conventional techniques. A privacy protector110uses conventional techniques, such as those employed by System T™ and IBM Classification Module™ (ICM), both commercially available from International Business Machines, Armonk, N.Y., to identify data entities within the excerpts in search result set108that meet predefined entity extraction criteria112, where the criteria preferably indicate what types of information should be removed from the search result excerpts due to privacy considerations. Examples of such data entities include, but are not limited to, Social Security Numbers, patient names, and credit card numbers. Predefined entity extraction criteria112may include criteria that are applicable regardless of the persons or parties that initiated the query, and/or criteria that are specific to individual persons or parties. Privacy protector110then presents search result set108, such as on a computer display or other computer output device, in a redacted format, where access to those data entities of search result set108that meet predefined entity extraction criteria112is prevented, such as by displaying the excerpts of search result set108without those data entities. Privacy protector110preferably presents one or more access control indicators in place of removed data entities indicating that access to the data entities is prevented, such as by displaying in place of the removed data entity an icon or the words “Access Restricted” or the semantic type of the deleted data.

In accordance with an alternative embodiment of the invention, data indexer100creates a set of excerpts114from data104in accordance with conventional techniques independent from the processing of queries by search engine106. Privacy protector110then identifies data entities within excerpts114that meet predefined entity extraction criteria112, preferably where the criteria are applicable regardless of the persons or parties that may initiate queries, and pre-redacts excerpts114by removing the identified data entities from within excerpts114as described above. Search engine106then creates search result set108from pre-redacted excerpts114in response to a query and presents search result set108as is, or with additional redaction as described above by applying predefined entity extraction criteria112that are specific the person or party that initiated the query.

In accordance with an embodiment of the invention, predefined entity extraction criteria112that are specific to individual persons or parties are defined in accordance with an access control model, such as a regulation-driven policy model, of entities, roles, policies, rules, actions, and obligations based on regulations and business requirements. For example, where data104includes medical records, predefined entity extraction criteria112may be defined to identify any data entities within data104that are to be redacted as described above for any individuals identified as having a role other than that of a physician. Thus, access to a given type of data entity or to a data entity matching a given rule may be defined as permitted or forbidden to an individual having a certain role, such as that of Physician or Chief Financial Officer.

Once search result set108has been presented as described above, any item in search result set108may be selected by a requestor, whereupon access may be given to the complete version of the item in data104from which the selected excerpt was taken, subject to any applicable access and/or privacy controls, such as may be provided by an underlying enterprise information system in accordance with conventional techniques. In accordance with an embodiment of the invention, a data access request logger116is configured such that when an item in search result set is selected, data access request logger116allows or requires that the request be accompanied by a reason. The reason may be manually entered or selected from a database118of predefined access reasons, which may be predefined at any level of access control, such as for a specific user, role, data item, etc., and which may also indicate whether or not to grant access to requested data, subject to any applicable access controls. Data access request logger116is preferably configured to maintain a record of the identity of the requestor together with a description of the requested data item and the reason given for the request, if given, and may also include additional information such as the role of the requester and the time of the request. Such information may be later analyzed to determine whether the request was justified, whether disciplinary action should be taken, and/or whether any of the data access model entities, permissions, or reasons described above should be modified to further restrict access to data and/or relax existing restrictions.

In the embodiments above index102is constructed from data104without restriction and is itself allowed to contain data that are removed from search result excerpts as described hereinabove. Alternatively, index102is constructed such that it does not itself contain data that are defined, using any conventional technique, as private data. Thus, in this alternative embodiment, index102would not, for example, contain patient names or identification numbers.

Reference is now made toFIG. 2, which is a simplified flowchart illustration of an exemplary method of operation of the system ofFIG. 1, operative in accordance with an embodiment of the invention. In the method ofFIG. 2, an index of data is created for use by a search engine, preferably when full access to the data is provided. Optionally, a set of excerpts from data is created from which data entities are removed that meet predefined entity extraction criteria. When, in response to a query, a search engine queries the data using the index, a search result set is created of excerpts from the data and/or from the previously-created set of excerpts. Data entities within the search results set are identified that meet predefined entity extraction criteria. The search results are then presented in a redacted format, where access to those data entities of the search result set that meet the predefined entity extraction criteria is prevented by removing the data entities. Access control indicators may be presented in place of removed data entities indicating that access to the data entities is prevented. Any item in the search result set may be selected by a requestor, whereupon access may be given to the complete version of the item in data from which the selected excerpt was taken, subject to any applicable access controls. Optionally, when an item in the search result set is selected, the request may be or must be accompanied by a reason. The reason may be manually entered or selected from a database of predefined reasons. Optionally, a record of the identity of the requestor is maintained together with a description of the requested data item, the reason given for the request, if given, and/or additional information such as the role of the requester and the time of the request.

The system and method ofFIGS. 1 and 2may also be understood by way of example as shown inFIGS. 3A-3Bas follows. InFIG. 3Aa set300of search engine search results are shown as they would appear on a computer display without being redacted in accordance with privacy criteria as described hereinabove. Thus, in the example shown, company names, account numbers, financial data, and other sensitive information are presented, as indicated by outlined regions302. InFIG. 3Ba set300′ of the same search results are shown as they would appear on a computer display after being redacted in accordance with privacy criteria as described hereinabove. Thus, in the example shown, company names, account numbers, financial data, and other sensitive information are removed and replaced with access control indicators302′. A warning message304may also be displayed indicating that the search results have been redacted due to privacy considerations.

In addition to, or as an alternative to, removing data from search result excerpts due to privacy considerations as described hereinabove, documents may be classified based on the types of data that they contain and access reasons that permit or restrict access to such data. These classifications may then be used in conjunction with search engine queries as is now described hereinbelow.

Reference is now made toFIG. 4A, which is a simplified conceptual illustration of a system for classifying data in accordance with data access reasons, constructed and operative in accordance with an embodiment of the invention, and additionally toFIGS. 4B-4E, which are simplified flowchart illustrations of exemplary methods of operation of the system ofFIG. 4A, operative in accordance with embodiments of the invention. In the system ofFIG. 4A, a data classifier400creates a classification index402of data104. Data104preferably includes any kind of data that are indexable and searchable as described hereinabove with reference toFIGS. 1 and 2. Where access to data104is restricted, such as where access to data104is controlled by an access control system (not shown), data classifier400is preferably given unrestricted access to data104or is otherwise provided with data104, such as by a crawler (not shown) that has unrestricted access to data104.

Data classifier400preferably employs predefined access reasons database118(FIG. 1) and constructs classification index402by determining which predefined access reasons in database118are associated with which portions (e.g., documents, data sets) of data104and storing the associations in classification index402. In one embodiment, database118includes predefined associations between the access reasons in database118and predefined data entity types in a set408, where an association between an access reason and a data entity type indicates that access is permitted to data that includes data entities of that data entity type. For example, predefined access reasons for accessing patient records may include “treating the patient,” “reviewing the record for quality control,” and “authorized use of patient history for statistical research.” The “social security number” data entity type may be associated with “treating the patient” to allow access to data that includes social security numbers. Data classifier400performs data entity extraction using conventional techniques, such as those employed by System T™, to identify data entities within data104that match the predefined data entity types in database118, whereupon data classifier400associates portions of data104that include data entities identified in this manner with the access reasons that are associated with their data entity types.

In another embodiment, data classifier400employs text classification techniques, such as those employed by the IBM Classification Module™ (ICM), to build statistical profiles410of sample data sets412, where for each predefined access reason in database118one manually categorized sample data set412is provided as an example of data to which access is permitted for the access reason, and/or another manually categorized sample data set412is provided as an example of data to which access is not permitted for the access reason. Data classifier400then constructs classification index402by determining which portions (e.g., documents, data sets) of data104match the statistical profiles410of which predefined access reasons in database118and stores the associations in classification index402.

The system ofFIG. 4Ais used in conjunction with elements of the system ofFIG. 1as follows. In one embodiment, as shown inFIG. 4B, when a requestor wishes to perform a query on data104using search engine106, data access request logger116requires that the requestor select an access reason from database118that is to be associated with the query, where the list of access reasons may be tailored for the requestor based on predefined relationships between access reasons and user roles and other characteristics. Search engine106performs the query on data104to produce search result set108. Privacy protector110eliminates from search result set108any search result excerpt except where access to the data104underlying the search result is permitted for the selected access reason as indicated by an association in classification index402between the underlying data104and the selected access reason. The remaining excerpts in search result set108are then displayed.

In another embodiment, as shown inFIG. 4C, when a requestor wishes to perform a query on data104using search engine106, data access request logger116requires that the requestor select an access reason from database118that is to be associated with the query. Search engine106performs the query on data104to produce search result set108. Privacy protector110removes information from the excerpts of search result set108in accordance with privacy concerns as described hereinabove with reference toFIG. 1, with the exception of any data entities whose data entity type is associated with the selected access reason as indicated in database118, where the association indicates that access to the data entity type for the selected access reason is permitted. The redacted excerpts in search result set108are then displayed.

In yet another embodiment, as shown inFIG. 4D, when a requestor wishes to perform a query on data104using search engine106, data access request logger116does not require that the requestor select an access reason from database118that is to be associated with the query. Search engine106performs the query on data104to produce search result set108. Prior to displaying any of the results in search result set108, if the underlying data104of any excerpt in search result set108is associated with any access reason in classification index402, data access request logger116requires that the requestor select an access reason from database118, preferably where the list of access reasons presented to the requestor includes only those access reasons that are associated with the underlying data104of search result set108as indicated in classification index402. Privacy protector110eliminates from search result set108any search result excerpt except where access to the data104underlying the search result is permitted for the selected access reason as indicated in classification index402. The remaining excerpts in search result set108are then displayed.

In still another embodiment, as shown inFIG. 4E, when a requestor wishes to perform a query on data104using search engine106, data access request logger116does not require that the requestor select an access reason from database118that is to be associated with the query. Search engine106performs the query on data104to produce search result set108. Privacy protector110removes information from the excerpts of search result set108in accordance with privacy concerns as described hereinabove with reference toFIG. 1, and the redacted search results are displayed. Data access request logger116then allows or requires that the requestor select an access reason from database118, preferably where the list of access reasons presented to the requestor includes only those access reasons that are associated with the underlying data104of search result set108as indicated in classification index402. If an access reason is provided, privacy protector110restores any information previously removed from any excerpt of search result set108of any data entities whose data entity type is associated with the selected access reason as indicated in database118, where the association indicates that access to the data entity type for the selected access reason is permitted. The search results are then redisplayed with the restored information.

In any of the embodiments ofFIGS. 4B-4E, once search result set108has been presented as described above, any item in search result set108may be selected by a requestor, whereupon access may be given to the complete version of the item in data104from which the selected excerpt was taken, subject to any applicable access and/or privacy controls, such as may be provided by an underlying enterprise information system in accordance with conventional techniques.

Any of the elements shown inFIGS. 1,2, and4A-4E are preferably executed by or otherwise made accessible to computer120(FIG. 1), such as by implementing any of the elements shown inFIGS. 1,2, and4A-4E in computer hardware and/or in computer software embodied in a computer-readable medium in accordance with conventional techniques.

Referring now toFIG. 5, block diagram500illustrates an exemplary hardware implementation of a computing system in accordance with which one or more components/methodologies of the invention (e.g., components/methodologies described in the context ofFIGS. 1-4) may be implemented, according to an embodiment of the invention.

As shown, the techniques for controlling access to at least one resource may be implemented in accordance with a processor510, a memory512, I/O devices514, and a network interface516, coupled via a computer bus518or alternate connection arrangement.

While the methods and apparatus herein may or may not have been described with reference to specific computer hardware or software, it is appreciated that the methods and apparatus described herein may be readily implemented in computer hardware or software using conventional techniques.