Investigation apparatus, computer-readable recording medium, and investigation method

An investigation apparatus according to an embodiment includes a processor that executes a process including: analyzing uniqueness of a plurality of values included in an attribute for each attribute of a table; reducing a plurality of values included in an attribute of which the analyzed uniqueness is lower than a predetermined value by a predetermined ratio with respect to an investigation original table and outputting a feature of the investigation original table; reducing a plurality of values included in an attribute of which the analyzed uniqueness is higher than a predetermined value by a predetermined ratio with respect to an investigation target table and outputting a feature of the investigation target table; and investigating similarity of the investigation target table to the investigation original table by comparing the feature of the investigation original table with the feature of the investigation target table.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is based upon and claims the benefit of priority of the prior Japanese Patent Application No. 2015-204014, filed on Oct. 15, 2015, the entire contents of which are incorporated herein by reference.

FIELD

The embodiment discussed herein is related to an investigation apparatus, an investigation program, and an investigation method.

BACKGROUND

In the related art, in companies or the like, as one of measures to information leakage, posterior investigation for specifying an information leaked original is performed. In posterior investigation for the information leakage, together with a manipulation log at the time of file manipulation such as document browsing or document preserving, features (fingerprints (hereinafter, referred to as feature data)) of the manipulated document are recorded. Therefore, in the investigation of the information leakage, the information leaked original is specified by specifying the manipulation log of a document similar to the leaked information by referring to the recorded feature data.

However, in the related art described above, in the investigation of table data describing information of various attributes (columns) for each record (row), feature data become huge in comparison with typical documents such as texts, so that there is a problem in that the investigation time is long. For example, in the case of performing posterior investigation for leakage of table data, feature data of the table data recorded at the time of manipulation become huge, so that a certain time is needed for the investigation.

FIG. 20is an explanation diagram illustrating investigation of information leakage. As illustrated inFIG. 20, with respect to company confidential information501, a manipulation log of a document502on which a user511performs manipulation such as referring or updating or feature data of the document502are recorded in an intra-company browsing file log521. For example, together with a file name such as “kimitsu.doc” of the document502on which the user511performs manipulation, data representing features (lists of keywords) of “for the middle-term business plane . . . ” in the document502are extracted and recorded in the intra-company browsing file log521.

Herein, printing of the document502is assumed to be leakage to the outside of the company as a document503after editing and processing. In addition, the document503is assumed to have a file name of “maruhi.pdf” or the like, and thus, the file name is changed to a file name different from that of the document502. In addition, the contents of the document503are changed so that a portion of the contents is different from those of the document502, for example, like “for the middle-term plan of the business department . . . ”. Therefore, in the posterior investigation of the information leakage, the document502as the leaked original is specified by searching for features similar to the feature extracted from the document503among the features recorded in the intra-company browsing file log521.

With respect to the feature data representing the features of the document, the feature data is difficult to treat in a keyword state, and thus, data obtained by hashing the keywords are used. For example, the keywords are hashed, and a range-reduced hash value is obtained by performing modulo operation (mod) with a constant n (in addition, the hash value is set as a value obtained by performing mod operation with a constant n, and a hash value before performing the mod operation is set as an intermediate hash value). In this manner, since the keywords included in the document are represented by hash values, feature data where the features of the document are represented by an n×n effective graph are obtained.

In addition, in the case where the value of n is set as about 10000 and, after that, the keywords are hashed, in some cases, the same hash values may be obtained between different keywords. However, in the n×n effective graph, the features are represented by a combination of the keywords. Therefore, between the documents of which contents are different, a possibility that the contents are changed into the same hash value is low, and the effective graph has a property that the contents are difficult to become the same.

In the posterior investigation of the information leakage, a similar document (document502as the leaked original) is obtained by comparing the feature data recorded in the intra-company browsing file log521with the feature data extracted from the document503.

FIG. 21is an explanation diagram illustrating comparison of the feature data. As illustrated inFIG. 21, in the posterior investigation of the information leakage, by obtaining a comparison result504by comparing feature data502arecorded in the intra-company browsing file log521with feature data503aextracted from the document503, a document similar to the document503among the documents of which the features are recorded in the intra-company browsing file log521is specified as a leaked original.

More specifically, by representing the feature data502aand503aby an n×n effective graph, a combination of features of the document can be represented in an n×n space by setting a flag (being set to 1) of a location where a pair exist. Therefore, in the comparison of the feature data502aand the feature data503a, the comparison result504is obtained by applying “and(&)” to the n×n space. Next, with respect to the obtained comparison result504, similarity between the documents corresponding to the feature data502aand503ais determined based on the number of 1 (true value) in the “and” of the n×n space.

Herein, in the case of adapting the documents502and503or the like as the table data, the above-described keyword is replaced with one attribute value included in the table data. However, the table data are obtained by extracting from original data stored in database by using SQL statements or the like. Accordingly, replacing of attributes included in the table data and replacing, adding, removing, or the like of records can be simply changed. Therefore, in the case of extracting the features from the table data, in order to correspond to replacing of the attributes and replacing, adding, removing or the like of records, features as a combination of two features for each attribute are comprehensively produced.

FIG. 22is an explanation diagram illustrating extraction of the feature data from a table data505. As illustrated inFIG. 22, in the case of extracting the feature data from the table data505, combinations of “baseball” of keyword 1, “43294” of keyword 2, and the like are comprehensively produced. Therefore, the data amount (feature amount) of the feature data is combinations of attributes such as “ID”, “name”, “age”, or “hobby”×the number of rows (number of records), which becomes huge. Therefore, in the case of performing the posterior investigation of table data leakage, a time (investigation time) taken for the process according to the extraction of the feature data or the determination of the similarity is increased in comparison with a typical document.

SUMMARY

According to an aspect of an embodiment, an investigation apparatus includes a processor that executes a process including: analyzing uniqueness of a plurality of values included in an attribute for each attribute of a table; reducing a plurality of values included in an attribute of which the analyzed uniqueness is lower than a predetermined value by a predetermined ratio with respect to an investigation original table and outputting a feature of the investigation original table; reducing a plurality of values included in an attribute of which the analyzed uniqueness is higher than a predetermined value by a predetermined ratio with respect to an investigation target table and outputting a feature of the investigation target table; and investigating similarity of the investigation target table to the investigation original table by comparing the feature of the investigation original table with the feature of the investigation target table.

DESCRIPTION OF EMBODIMENTS

Preferred embodiments of the present invention will be explained with reference to accompanying drawings. In the embodiments, components having the same functions are denoted by the same reference numerals, and redundant description thereof is omitted. In addition, the investigation apparatus, the investigation program, and the investigation method described in the embodiments hereinafter are exemplary ones, but these do not limit the embodiments. In addition, the embodiments hereinafter may be appropriately combined to the extent that there is no inconsistence.

FIG. 1is a block diagram illustrating a functional configuration of an investigation apparatus1according to an embodiment. The investigation apparatus1illustrated inFIG. 1is, for example, an information processing apparatus such as a PC (personal computer). The investigation apparatus1receives table data D1according to an investigation original table and table data D2according to an investigation target table and investigates similarity of the investigation target table to the investigation original table.

Herein, the table data D1and the table data D2are data of tables describing information of various attributes (columns) for each record (row). The table data D1and the table data D2can be obtained, for example, by extracting SQL statements or the like from original data stored in database. Therefore, by investigating the similarity between the table data D1and the table data D2, it can be determined whether or not the table data D1and the table data D2are obtained from the same original data.

As an example, the investigation apparatus1is used for posterior investigation for specifying a table of an information leaked original. More specifically, the investigation apparatus1stores feature of the table data D1of a table which undergoes browsing, preserving, or the like inside the company (investigation original table) as a log and investigates the similarity by comparing the feature of the table data D1with the feature of the table data D2of a table leaked to the outside of the company (investigation target table). Next, the investigation apparatus1specifies the table data D1of the table having a high similarity, that is, the table of which a degree of similarity to the table leaked to the outside of the company is a predetermined value or more as an information leaked original.

As illustrated inFIG. 1, the investigation apparatus1is configured to include a recording unit10, a storage unit20, and an investigation unit30.

The recording unit10receives an input of the table data D1according to the investigation original table and records feature data21representing features of the investigation original table extracted from the table data D1in the storage unit20.

The feature data21are data obtained by hashing data (numerical values, texts, or the like) of various attributes (columns) for each record (row) included in the table data D1. As the hashing of the data performed by the recording unit10, well-known methods are used.

For example, an intermediate hash value is obtained from original data such as numerical values, texts, or the like included in the table data D1by performing a keyed cryptographic hash process using a fixed key. By the keyed cryptographic hash process, the original data is prevented from being estimated. Next, a range-reduced hash value is obtained by performing modulo operation (mod) with a constant n on the intermediate hash value. Therefore, the number of patterns of the hash value in the feature data21is allowed to be small, so that a data amount of the feature data21is reduced.

The storage unit20stores the feature data21for each table together with identification information (for example, document ID (Identification Data) or the like) for identifying each table. Therefore, the feature data21obtained from the table data D1of, for example, the table which undergoes browsing, preserving, or the like inside the company, that is, the investigation original table is managed as a log in the storage unit20.

The investigation unit30receives an input of the table data D2according to the investigation target table and compares the feature of the table extracted from the table data D2with the feature data21of the investigation original table stored in the storage unit20. By the comparison, the investigation unit30investigates the similarity of the investigation target table to the investigation original table. More specifically, the investigation unit30compares the hash values included in the respective features to calculate the number (number of times of hit) of times of coincidence between the hash values, so that a degree of similar relationship representing the similarity between the tables is obtained.

In addition, the investigation unit30performs comparison with the feature data21for each table managed as a log in the storage unit20, searches for the table having a high degree of similar relationship among the tables managed in the storage unit20, and outputs the search result. Therefore, a user can specify the table of the leaked original having a high similarity to the table leaked to the outside of the company among the tables managed as a log in the storage unit20based on the search result of the investigation unit30using the table data D2of the table leaked to the outside of the company.

Herein, details of the recording unit10and the investigation unit30are described. The recording unit10is configured to include an input unit11, a table data analyzing unit12, and a feature output unit13.

Herein, the phrase “uniqueness of a plurality of values included in an attribute for each attribute” denotes the number of unique values in the attribute. For example, the uniqueness of the attribute is a ratio of the number of unique values to a total number of values included in the attribute. The larger the number of unique values is, the higher the uniqueness is (H). On the contrary, the smaller the number of unique values is, the lower the uniqueness of the attribute is (L).

For example, with respect to the attribute such as “ID” in the table where all the values are unique values, the ratio of the number of unique values to the total number of values included in the attribute is “1”, and thus, the uniqueness is highest. In addition, with respect to the attribute such as “name” where the number of unique values is large, the ratio of the number of unique values to the total number of values included in the attribute becomes a value close to 1, and thus, the uniqueness is heightened. On the contrary, with respect to the attribute such as “age” or “hobby” where the number of unique values is small, the ratio of the number of unique values to the total number of values included in the attribute becomes a value close to 0, and thus, the uniqueness is lowered.

The feature output unit13hashes values of various attributes for each record included in the table data D1and outputs the feature data21. The output feature data21are stored in the storage unit20. At this time, the feature output unit13reduces (thins out) the number of values by reducing the values for each record in the attribute of which the uniqueness is lower than a preset predetermined value by a predetermined ratio based on the analysis result of the table data analyzing unit12. Next, the feature output unit13obtains the feature data21by hashing each after-thinning-out value with respect to the attribute of which the uniqueness is low. In addition, with respect to the attribute on which the thinning-out is not performed, each value included in the attribute is hashed as it is.

FIG. 2is a flowchart illustrating a process of recording the feature data. As illustrated inFIG. 2, if the process is started, the input unit11receives an input of the table data D1(S11).

FIG. 3is a diagram illustrating an example of the table data D1. As illustrated inFIG. 3, the table data D1include the attributes (columns) such as “ID”, “name”, “age”, and “hobby” for each record (row). Herein, with respect to the attributes “ID” and “name”, the uniqueness is assumed to be higher than a preset predetermined value. In addition, with respect to the attributes “age” and “hobby”, the uniqueness is assumed to be lower than a preset predetermined value.

Next, the table data analyzing unit12obtains the number of records included in the table and the number of unique values for each attribute by referring to the records (rows) included in the input table data D1and the values of the attributes (columns) (S12).

FIG. 4is a flowchart illustrating details of the process of S12. As illustrated inFIG. 4, initially, the table data analyzing unit12resets the number of records (for example, number of records=0) (S21). Next, the table data analyzing unit12determines whether or not all the records included in the input table data D1are processed (S22). In the case where the process on all the records is ended (S22: YES), the table data analyzing unit12ends the process of S12.

In the case where the process on all the records is not ended (S22: NO), the table data analyzing unit12extracts non-processed records from the records included in the input table data D1(S23). Next, the table data analyzing unit12counts up the number of records from the extracted records (S24) and acquires statistical information for obtaining the number of unique values for each attribute (S25). Next, the table data analyzing unit12determines whether or not all the records included in the table data D1are processed (S26). In the case where all the records are not processed (NO), the table data analyzing unit returns to the process of S23, and in the case where all the records are processed (YES), the table data analyzing unit ends the process of S12.

FIG. 5is a flowchart illustrating details of the process of S25. As illustrated inFIG. 5, the table data analyzing unit12extracts non-processed attributes from the records extracted in S23(S31). Next, the table data analyzing unit12determines whether or not each value of the extracted attribute is a value (previously-appeared value) which has already appeared in the attribute (S32).

In the case where the value is a previously-appeared value (S32: YES), the table data analyzing unit12counts up the counter corresponding to the appearing value (S34). In the case where the value is not a previously-appeared value (S32: NO), the table data analyzing unit12produces the counter corresponding to the appearing value (S33). The counter which is counted for each previously-appeared value is statistical information for obtaining the number of unique values for each attribute. For example, the case where the counter is “1” represents that the value is a unique value.

Next, the table data analyzing unit12determines whether or not all the attributes are processed (S35). In the case where the process on all the attributes is not performed (S35: NO), the table data analyzing unit returns to the process of S31. In addition, in the case where the process on all the attributes is performed (S35: YES), the table data analyzing unit12ends the process of S25.

Returning toFIG. 2, subsequently to S12, the table data analyzing unit12obtains a ratio of the unique values for each attribute from the ratio between the number of previously-appeared values and the number of unique values of which the counter for each previously-appeared value is “1” based on the statistical information for each attribute (S13). The ratio of the unique values for each attribute obtained in S13is a value representing the uniqueness for each attribute.

Next, the table data analyzing unit12determines a feature residual ratio for each attribute, that is, a ratio of values remaining as features from a plurality of values included in the attribute based on the ratio of the unique values for each attribute obtained in S13(S14).

More specifically, the table data analyzing unit12determines based on the ratio of the unique values for each attribute obtained in S13whether or not the uniqueness of the attribute (ratio of the unique values) is lower than a preset predetermined value. With respect to the attribute (H) of which the uniqueness is determined not to be lower, the feature residual ratio is set as “1”, in other words, all the values are set to remain as features. In addition, with respect to the attribute (L) of which the uniqueness is determined to be lower, the values obtained by thinning-out the feature residual ratio by a predetermined ratio, that is, a predetermined ratio are set to remain as features.

FIG. 6is a flowchart illustrating details of the process of S14. As illustrated inFIG. 6, the table data analyzing unit12extracts the non-processed attributes (S41) and obtains the feature residual ratio of the extracted attributes (S42).

More specifically, with respect to the attribute (H) of which the uniqueness is not low, the feature residual ratio is set to “1”, in other words, all the values are set to remain as features. In addition, with respect to the attribute (L) of which the uniqueness is low, the number of unique values/the number of records of the attribute is set as the feature residual ratio. In this manner, by setting the feature residual ratio corresponding to the ratio of the number of unique values/the number of records, the feature residual ratio in accordance with the uniqueness can be set.

In addition, the calculation of the feature residual ratio described above is exemplary one. For example, with respect to the attribute classified as L, the feature residual ratio may be set to be a constant ratio. In this manner, the feature residual ratio is set so that, with respect to the attribute classified as H, all the values are allowed to remain as features; and with respect to the attribute classified as L, the thinning-out is performed.

For example, in the example of the table data D1inFIG. 3, with respect to the attributes such as “ID” and “name” of which the uniqueness is H, the feature residual ratio is set as “all”(feature residual ratio=“1”). In addition, with respect to the attributes such as “age” and “hobby” of which the uniqueness is L, the feature residual ratio is set to “1/1000”.

Next, the table data analyzing unit12determines whether or not all the attributes are processed (S43). In the case where the process on all the attributes is not performed (S43: NO), the table data analyzing unit returns to the process of S41. In addition, in the case where the process on all the attributes is performed (S43: YES), the table data analyzing unit12ends the process of S14.

In this manner, in S14, the feature residual ratio is set so that, with respect to the attributes classified as H, all the values are set to remain as features, and with respect to the attributes classified as L, the thinning-out is performed.

Returning toFIG. 2, subsequently to S14, the feature output unit13outputs the features obtained by thinning-out the values of each attribute based on the feature residual ratio for each attribute and records the features as the feature data21in the storage unit20(S15).

FIG. 7is a flowchart illustrating details of the process of S15. As illustrated inFIG. 7, if the process is started the feature output unit13determines whether or not all the records included in the input table data D1are not processed (S51). In the case where the process on all the records is ended (S51: YES), the feature output unit13ends the process of S15.

In the case where the process on all the records is not ended (S51: NO), the feature output unit13extracts non-processed records from the records included in the input table data D1(S52). Next, the feature output unit13extracts non-processed attributes from the extracted records (S53).

Next, the feature output unit13performs thinning-out values based on the feature residual ratio of the attributes and outputs the features obtained by hashing the after-thinning-out values (S54). More specifically, with respect to the attributes classified as H, of which the feature residual ratio is “1”, the features obtained by hashing all the values are output. In addition, with respect to the attributes classified as L, of which the feature residual ratio is a predetermined ratio other than “1”, the features obtained by hashing the values after performing the thinning-out by the feature residual ratio are output.

Next, the feature output unit13determines whether or not all the attributes are processed (S55). In the case where the process on all the attributes is not performed (S55: NO), the feature output unit returns to the process of S53. In addition, in the case where the process on all the attributes is performed (S55: YES), the feature output unit13proceeds to the process S56.

In S56, the feature output unit13determines whether or not all the records included in the input table data D1are processed. In the case where the process on all the records is ended (S56: YES), the feature output unit13ends the process of S15. In addition, in the case where the process on all the records is not ended (S56: NO), the feature output unit13returns to the process of S52.

FIG. 8is an explanation diagram illustrating an example of the feature data21. As illustrated inFIG. 8, the feature data21include items of “feature”, “document ID”, “record number”, “attribute number”, and the like.

The “feature” is a value of the feature output by the feature output unit13. In the example illustrated, although before-hashing values are illustrated for the easier understanding, in actual cases, after-hashing hash values are stored. The “document ID” is identification information representing the table (document) of the input table data D1, and a unique numerical value or the like allocated at the time of storing data or the like is stored. The “record number” is a record number of an item according to the “feature”, and a record numbers of a record extracted in S52are stored. The “attribute number” is information representing an attribute of an item according to the “feature”, and a number (for example, a position of arrangement of a column) or the like representing an attribute extracted in S53is stored.

For example, with respect to the attributes classified as H, that is, attribute 1 and attribute 2 in “ID” and “name” ofFIG. 3, the values of the entire items are hashed based on the feature residual ratio to be recorded as features. In addition, with respect to the attributes classified as L, that is, attribute 3 . . . in “age” and “hobby” inFIG. 3, the values of the items which are reduced down to 1/1000 are hashed based on the feature residual ratio to be recorded as features. In this manner, by performing value reduction based on the feature residual ratio, it is possible to reduce the data amount of the feature data21.

Returning toFIG. 1, the investigation unit30is configured to include an input unit31, a table data analyzing unit32, a feature output unit33, a searching unit34, and output unit35.

The feature output unit33obtains features of the table according to the table data D2by hashing the values of various attributes for each record included in the table data D2and outputs the features to the searching unit34. At this time, the feature output unit33reduces (thins out) the number of values by reducing the values for each record in the attribute of which the uniqueness is higher than a preset predetermined value by a predetermined ratio based on the analysis result of the table data analyzing unit32. Next, the feature output unit33hashes each after-thinning-out value with respect to the attribute (H) of which the uniqueness is high. In addition, with respect to the attribute (L) on which the thinning-out is not performed, each value included in the attribute is hashed as it is.

The searching unit34investigates the similarity of the table by comparing the feature of the table according to the table data D2output from the feature output unit33with the feature of each table stored in the feature data21. More specifically, the searching unit34compares the hash values included in the respective features to calculate the number (number of times of hit) of times of coincidence between the hash values, so that a degree of similar relationship representing the similarity between the tables is obtained. Next, the searching unit34outputs the table having a high degree of similar relationship among the tables stored in the feature data21as a search result (for example, document ID or the like representing a table having a high degree of similar relationship) to the output unit35. In addition, evaluation of a height of a degree of similar relationship may be arbitrarily determined. For example, a table where the degree of similar relationship is a predetermined value or more may be used, or a table ranging from a side having a high degree of similar relationship down to a predetermined order may be used.

The output unit35performs outputting the search result based on the search result of the searching unit34by screen display in a display device such as an LCD (Liquid Crystal Display) or printing on a paper medium or the like in a printing device such as a printer.

FIG. 9is a fluorescent image an investigation process. As illustrated inFIG. 9, if the process is started the input unit31receives an input of the table data D2(S61). Next, the feature output unit33extracts the feature data from the input table data D2based on the analysis result of the table data analyzing unit32with respect to the table data D2(S62).

FIG. 10is a flowchart illustrating details of the process of S62. As illustratedFIG. 10, if the process is started, similarly to S12, the table data analyzing unit32obtains the number of records included in the table and the number of unique values for each attribute by referring to the input table data D2(S71).

Next, similarly to S13, the table data analyzing unit32obtains a ratio of the unique values for each attribute (S72). The ratio of the unique values for each attribute obtained in S72is a value representing the uniqueness for each attribute.

Next, the table data analyzing unit32determines a feature residual ratio for each attribute, that is, a ratio of values remaining as features from a plurality of values included in the attribute based on the ratio of the unique values for each attribute obtained in S72(S73).

More specifically, the table data analyzing unit32determines based on the ratio of the unique values for each attribute obtained in S72whether or not the uniqueness of the attribute (ratio of the unique values) is higher than a preset predetermined value. With respect to the attribute (L) of which the uniqueness is determined not to be higher, the feature residual ratio is set to “1”, in other words, all the values are set to remain as features. In addition, with respect to the attribute (H) of which the uniqueness is determined to be higher, the values obtained by thinning-out the feature residual ratio by a predetermined ratio, that is, a predetermined ratio are set to remain as features.

FIG. 11is a flowchart illustrating details of the process of S73. As illustrated inFIG. 11, the table data analyzing unit32extracts the non-processed attributes (S81) and obtains the feature residual ratio of the extracted attribute (S82).

More specifically, with respect to the attribute (L) of which the uniqueness is not higher, the feature residual ratio is set as “1”, in other words, all the values are set to remain as features. In addition, with respect to the attribute (H) of which the uniqueness is high, (1−(number of unique values/number of records of attribute)) is set as the feature residual ratio. In this manner, by setting the feature residual ratio corresponding to the ratio according to (1−(number of unique values of attribute/number of records)), the feature residual ratio in accordance with the uniqueness can be set.

In addition, the calculation of the feature residual ratio described above is exemplary one. For example, with respect to the attributes classified as H, the feature residual ratio may be set to be a constant ratio. In this manner, the feature residual ratio is set so that, with respect to the attributes classified as L, all the values are allowed to remain as features; and with respect to the attributes classified as H, the thinning-out is performed.

Next, the table data analyzing unit32determines whether or not the feature residual ratio is a preset predetermined value or less (S83). In the case where the feature residual ratio is the predetermined value or less (S83: YES), the feature residual ratio is set as a preset minimum residual ratio (S84). In addition, in the case where the feature residual ratio is not the predetermined value or less (S83: NO), the table data analyzing unit32skips the process of S84and proceeds to the process of S85. In this manner, by setting the minimum residual ratio, the feature is prevented from not remaining due to the lowering of the feature residual ratio as the uniqueness is increased, so that it is possible to secure a certain amount or more as the feature amount in the attribute.

Next, the table data analyzing unit32determines whether or not all the attributes are processed (S85). In the case where all the attributes are not processed (S85: NO), the table data analyzing unit returns to the process of S81. In addition, In the case where all the attributes are processed (S85: YES), the table data analyzing unit32ends the process of S73.

In this manner, in S73, the feature residual ratio is set so that, with respect to the attributes classified as L, all the values are set to remain as features, and with respect to the attributes classified as H, the thinning-out is performed.

Returning toFIG. 10, subsequently to S73, the feature output unit33outputs the features of the table data D2obtained by thinning-out the values of each attribute based on the feature residual ratio for each attribute to the searching unit34(S74).

FIG. 12is a flowchart illustrating details of the process of S74. As illustrated inFIG. 12, if the process is started, the feature output unit33extracts the non-processed attributes (S91).

Next, the feature output unit33performs thinning-out values based on the feature residual ratio of the attributes and adds the values of the attributes corresponding to the feature residual ratio to a list according to the table data D2(S92). More specifically, with respect to the attributes classified as L, of which the feature residual ratio is “1”, all the values are added to the list. In addition, with respect to the attributes classified as H, of which the feature residual ratio is a predetermined ratio other than “1”, the values after performing the thinning-out by the feature residual ratio are added.

Next, the feature output unit33determines whether or not all the attributes are processed (S93). In the case where all the attributes are not processed (S93: NO), the feature output unit returns to the process of S91. In addition, in the case where the process on all the attributes is processed (S93: YES), the feature output unit33proceeds to the process of S94. In S94, the feature output unit33allows the list to become unique (hashes each value of the list) and outputs the list to the searching unit34.

Returning toFIG. 9, the searching unit34performs searching for the features of which hash values are coincident with each other by comparing with the feature data21stored in the storage unit20using the feature data of the table data D2extracted in S62(S63). Next, the searching unit34obtains the number of times of hit of the features for each document by classifying the features obtained from the searching for each document represented by the document ID (S64).

FIG. 13is an explanation diagram illustrating searching.FIG. 13illustrates the case of searching for the feature data21by using the features extracted from the table data D2. In the table data D2, with respect to the attribute (L) such as “age” and “hobby”, of which the uniqueness is not high, the entire features are set to be extracted. In addition, with respect to the attribute (H) such as “ID” and “name”, of which the uniqueness is high, the features on which the thinning-out is performed by 1/10 are set to be extracted. In addition, with respect to each feature, in the example illustrated, although before-hashing values are illustrated for the easier understanding, in actual cases, after-hashing hash values are stored.

As illustrated inFIG. 13, the searching unit34performs searching for the features which are coincident with the features of the feature data21on the features extracted from the table data D2and obtains a search result D3by classifying the features searched from the feature data21for each document ID. The search result D3is summarized as the number of times of hit of the features for each document ID. Based on the search result D3obtained in this manner, the table (table corresponding to the document ID) of which the number of times of hit is large becomes a candidate for a table similar to the table of the table data D2.

FIG. 14is an explanation diagram illustrating correspondence between table data D1and D2at the time of searching. As illustrated inFIG. 14, all the values of “ID” and “name” of the table data D1according to the investigation original table are recorded as the features in the feature data21because the attributes are the attributes (H) of which the uniqueness is high. On the contrary, the values of “ID” and “name” of the table data D2according to the investigation target table are thinned out because the attributes are the attributes (H) of which the uniqueness is high. Therefore, at the time of searching, the number of times of comparison is reduced due to the thinning-out, so that the searching can be performed at a high speed. In addition, with respect to the investigation target table, although the values of the attributes of which the uniqueness is high are thinned out, in the case where the investigation target table is similar to the investigation original table, the corresponding features are hit at a high accuracy.

In addition, the values of “age” and “hobby” of the table data D1according to the investigation original table are recorded as the features in the feature data21because the attributes are the attributes (L) of which the uniqueness is low. On the contrary, all the values of “age” and “hobby” of the table data D2according to the investigation target table are extracted because the attributes are the attributes (L) of which the uniqueness is low. Therefore, at the time of searching, the number of times of comparison is reduced due to the thinning-out, so that the searching can be performed at a high speed. In addition, with respect to the attributes (L) of which the uniqueness is low in the table data D1, almost all the features which do not overlap each other even through the thinning-out is performed more or less times are recorded in the feature data21. Therefore, with respect to the investigation target table, all the features of the attributes of which the uniqueness is low are extracted, and the features are compared with the feature data21. In the case where the investigation target table is similar to the investigation original table, the corresponding features are hit at a high accuracy.

FIG. 15is an explanation diagram illustrating a search time and a search rate. More specifically,FIG. 15illustrates formulation of the search time and the search rate for each condition at the time of comparing the above-described table data D1according to the investigation original table and the above-described table data D2according to the investigation target table. InFIG. 15, H represents the attribute of which the uniqueness is high, and L represents the attribute of which the uniqueness is low. In addition, p represents the feature residual ratio of the attribute of L in the table data D1. In addition, n represents the number of rows of the table in the table data D2. As illustrated inFIG. 15, a condition (1) where H and L of the search input (table data D2) are entirely used, a condition (2) where L of the search input is entirely used, and H is thinned out by 1/C, and a condition (3) where both of L and H of the search input are thinned out by 1/C are formulated.

FIG. 16is an explanation diagram illustrating a specific example of the search time and the search rate. More specifically,FIG. 16illustrates insertion of numerical values into the formulas of the conditions (1) to (3) ofFIG. 15. InFIG. 16, p=1/1000, n=10, and C=2. As illustrated inFIG. 16, with respect to the search rate representing the ratio where the corresponding features are hit, almost all the conditions are maintained, and in contrast to the condition (1), the condition (2) corresponding to the embodiment provides more appropriate result than the condition (3). In addition, with respect to the search time, in contrast to the condition (1), in the conditions (2) and (3), the search time becomes ½, so that the speed of two times can be expected.

Returning toFIG. 9, subsequently to S64, the searching unit34extracts the result (information of hit classified for each document ID) of one document which has not been processed from the search results D3(S65). Next, the searching unit34quantifies the degree of similar relationship according to the extracted document (S66).

FIG. 17is a flowchart illustrating details of the process of S66. As illustrated inFIG. 17, if the process is started, the searching unit34obtains a correspondence relationship of attributes between the searched side (search original) and the searching side (search target) which are hit by referring the information of hit with respect to the extracted document (S101).

FIG. 18is an explanation diagram illustrating a correspondence relationship of attribute. More specifically,FIG. 18illustrates details of search results D31where document ID is “0001” in the search result D3ofFIG. 13. In the “hit search original attribute” at the right end of the details of search results D31, the numbers of times of hit of the attributes of the search target are arranged in the order of the attribute numbers (1, 2, 3, 4). For example, with respect to the attribute number “1” of the search original, in the attribute number “1” of the search target, the feature is hit “10” times. In addition, with respect to the attribute number “2” of the search original, in the attribute number “1” of the search target, the feature is hit “1” time, and in the attribute number “2” of the search target, the feature is hit “104” times.

In S101, by comparing the “number of times of hit” and the “hit search original attribute” for each attribute number, the correspondence relationship of attribute between the search original and the search target is obtained. For example, with respect to the attribute number “1” of the search original, the number of times of hit is “10”; and with respect to the attribute number “1” of the search target, the feature is hit “10” times. Therefore, in the attribute number “1”, the search original and the search target corresponds to each other. Similarly, the attribute number “2” of the search original→the attribute number “2” of the search target, the attribute number “3” of the search original→the attribute number “3” of the search target, and the attribute number “4” of the search original→the attribute number “4” of the search target are obtained.

Returning toFIG. 17, subsequently to S101, the searching unit34removes exceptional relationships such as overlapped attributes from the relationships obtained in S101(S102).

Next, the searching unit34allows hit information to remain by using the remaining correspondence relationships obtained by removing the exceptional relationships (S103) and arranges the features and the attributes which are hit by the same records (S104). More specifically, the features which are hit between the search original and the search target are arranged again in the order of the record number and the attribute number.

Next, the searching unit34counts the records in the same patterns as those of the records of the searching side (search target) (S105). More specifically, the features of the search original are set to be the record 1: the attribute number 1 (38432), the attribute number 2 (Ichiro), and the attribute number 3 (40), the record 2: the attribute number 1 (76138), the attribute number 2 (Juro), and the attribute number 4 (Basketball) . . . . In addition, the features of the records of the search target are set to be the attribute number 1 (38432), the attribute number 2 (Ichiro), and the attribute number 3 (40). In this case, the record 1 of the search original is counted as the record in the same pattern. In this manner, the larger the number of counts in S105is, the higher the degree of similarity is.

Next, the searching unit34applies a weighting factor to the number of times of hit in accordance with the number of counts and calculates the degree of similar relationship according to the extracted document (S106).

Returning toFIG. 9, subsequently to S66, the searching unit34determines whether or not the process for all the documents classified in S64is ended (S67). In the case where the process for all the documents is not ended (S67: NO), the searching unit34returns to the process of S65.

In the case where the process for all the documents is ended (S67: YES), the searching unit34sorts the documents in the order of the degree of similar relationship (S68). Next, the output unit35outputs the sort result of S68by screen display, printing, or the like (S69).

As described heretofore, the investigation apparatus1is configured to include the table data analyzing units12and32which analyze the uniqueness of a plurality of the values included in the attribute for each attribute of the tables. In addition, the investigation apparatus1is configured to include the feature output unit13which reduces a plurality of the values included in the attribute of which the analyzed uniqueness is lower than a predetermined value by a predetermined ratio with respect to the investigation original table of the table data D1and outputs the feature data21. In addition, the investigation apparatus1is configured to include the feature output unit33which reduces a plurality of the values included in the attribute of which the analyzed uniqueness is higher than a predetermined value by a predetermined ratio with respect to the investigation target table of the table data D2and outputs the feature. In addition, the investigation apparatus1is configured to include the searching unit34which investigates the similarity of the investigation target table to the investigation original table by comparing the feature of the investigation original table with the feature of the investigation target table. Therefore, as clarified from the comparison of the condition (1) and the condition (2) inFIGS. 15 and 16, the investigation apparatus1can prevent the time needed for the investigation from being increased.

In addition, the investigation apparatus1is configured to further include the storage unit20which stores the feature data21of the investigation original table, and the searching unit34compares the feature data21stored in the storage unit20with the features of the investigation target table. Therefore, the investigation apparatus1can store the feature data21of the table which undergoes browsing, preserving, or the like inside the company and can use the feature data21for posterior investigation for specifying the table of the information leaked original.

In addition, each component of each device illustrated is not necessarily configured as illustrated physically. In other words, a specific mode of distribution and integration of each device is not limited to the illustrated one. All or some of each device may be distributed or integrated functionally or physically in arbitrary units according to various loads, use environment, or the like to be configured.

In addition, with respect to various process functions performed by the investigation apparatus1, all or some thereof may be executed on a CPU (or a microcomputer such as an MPU or an MCU (Micro Controller Unit)). In addition, all or some of the various process functions may be allowed to be executed on a program which is analyzed and executed by a CPU (or a microcomputer such as an MPU or an MCU) or on hardware configured with wired logic.

The various processes described in the above-described embodiments may be implemented by allowing a computer to execute a program which is prepared in advance. Hereinafter, an example of the computer (hardware) executing a program having the same functions as those of the above-described embodiments will be described.FIG. 19is a block diagram illustrating an example of a hardware configuration of an investigation apparatus1according to an embodiment.

As illustrated inFIG. 19, the investigation apparatus1is configured to include a CPU101which executes various calculation processes, an input device102which receives data input, a monitor103, and a speaker104. In addition, the investigation apparatus1is configured to include a medium reading device105which reads a program and the like from a storage medium, an interface device106for connection to various devices, and a communication device107for communication connection to an external device in a wired or wireless manner. In addition, the investigation apparatus1is configured to include a RAM108which temporarily stores various types of information and a hard disk device109. In addition, the components (101to109) in the investigation apparatus1are connected to a bus110.

The hard disk device109stores a program111for executing various processes in the recording unit10, the storage unit20, and the investigation unit30described in the above embodiments. In addition, the hard disk device109stores various data112(feature data21and the like) which are referred to by the program111. The input device102receives, for example, an input of operation information from an operator of the investigation apparatus1. The monitor103displays, for example, various screens which are operated by the operator. The interface device106is connected to, for example, a printing device or the like. The communication device107is connected to a communication network such as LAN (Local Area Network) to exchange various types of information with an external device through the communication network.

The CPU101reads the program111stored in the hard disk device109and expands the program on the RAM108to execute the program, so that various processes are performed. In addition, the program111is not necessarily stored in the hard disk device109. For example, the program111stored in a storage medium which can be read by the investigation apparatus1may be allowed to be read and executed by the investigation apparatus1. The storage medium which can be read by the investigation apparatus1corresponds to, for example, a portable recording medium such as a CD-ROM, a DVD disk, or a USB (Universal Serial Bus) memory, a semiconductor memory such as a flash memory, a hard disk drive, or the like. In addition, the program may be stored in a device connected to a public network, the Internet, LAN, or the like, and the investigation apparatus1may be allowed to read the program from the device to execute the program.

According to one embodiment, it is possible to prevent a time taken for investigation from being increased.