Medical data publishing based on average distribution and clustering

CAAI Transactions on Intelligence TechnologyVolume 7, Issue 3 p. 381-394 ORIGINAL RESEARCHOpen Access Medical data publishing based on average distribution and clustering Tong Yi, Tong Yi orcid.org/0000-0003-0946-5237 School of Computer Science and Engineering, Guangxi Normal University, Guilin, China School of Computer Science, Communication University of China, Beijing, ChinaSearch for more papers by this authorMinyong Shi, Corresponding Author Minyong Shi [email protected] School of Computer Science, Communication University of China, Beijing, China Correspondence Minyong Shi, School of Computer Science, Communication University of China, Beijing 100024, China. Email: [email protected]Search for more papers by this authorHaibin Zhu, Haibin Zhu orcid.org/0000-0003-1922-1631 Department of Computer Science and Mathematics, Nipissing University, North Bay, Ontario, CanadaSearch for more papers by this author Tong Yi, Tong Yi orcid.org/0000-0003-0946-5237 School of Computer Science and Engineering, Guangxi Normal University, Guilin, China School of Computer Science, Communication University of China, Beijing, ChinaSearch for more papers by this authorMinyong Shi, Corresponding Author Minyong Shi [email protected] School of Computer Science, Communication University of China, Beijing, China Correspondence Minyong Shi, School of Computer Science, Communication University of China, Beijing 100024, China. Email: [email protected]Search for more papers by this authorHaibin Zhu, Haibin Zhu orcid.org/0000-0003-1922-1631 Department of Computer Science and Mathematics, Nipissing University, North Bay, Ontario, CanadaSearch for more papers by this author First published: 22 April 2022 https://doi.org/10.1049/cit2.12094Citations: 3AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Abstract Most of the data publishing methods have not considered sensitivity protection, and hence the adversary can disclose privacy by sensitivity attack. Faced with this problem, this paper presents a medical data publishing method based on sensitivity determination. To protect the sensitivity, the sensitivity of disease information is determined by semantics. To seek the trade-off between information utility and privacy security, the new method focusses on the protection of sensitive values with high sensitivity and assigns the highly sensitive disease information to groups as evenly as possible. The experiments are conducted on two real-world datasets, of which the records include various attributes of patients. To measure sensitivity protection, the authors define a metric, which can evaluate the degree of sensitivity disclosure. Besides, additional information loss and discernability metrics are used to measure the availability of released tables. The experimental results indicate that the new method can provide better privacy than the traditional one while the information utility is guaranteed. Besides value protection, the proposed method can provide sensitivity protection and available releasing for medical data. 1 INTRODUCTION Medical data has been used in various areas over recent years, such as health blockchain [1], disease diagnosis [2], medical data publishing [3] and so on. Medical data publishing can provide useful information for statistics and research. Researchers can analyse and discover factors related to diseases by released tables. For example, researchers can find that the Japanese have a very small probability of suffering from heart disease, and the related causes can be further explored. However, the medical data table also involves a great deal of sensitive data [4, 5] of individuals; it is necessary to anonymise the medical data table before releasing it. Data publishing methods [6-8] can be exploited in this scenario. Data publishing aims to provide useful information while guaranteeing the privacy security. There are three categories of attributes in data publishing: sensitive attributes (SAs), identifiers, and quasi-identifiers (QIs). The sensitive attribute [9] usually relates to the privacy of individuals, and may include, for example, Disease, Call log and Salary. The values of SA are the focus of privacy protection in data publishing. The identifier [9], which can uniquely identify individuals, is usually removed from the raw data table. The quasi identifier [9], which includes related information of individuals, such as Age and Zip code, may be combined to identify an individual. Therefore, the adversary may be able to disclose privacy by QIs, even the identifier has been removed [6]. Faced with this, various privacy-preserving data publishing methods were proposed. K-anonymity [6, 7] is a classical one of data publishing methods. It divides table into groups that have at least K records, then generalises QIs to make records have the same QI values in a group. The adversary cannot distinguish an individual from at least K-1 other ones. However, K-anonymity has not considered the diversity of sensitive values. In extremis, the sensitive values in a group are all the same. To solve this problem, L-diversity [8] ensures each group includes at least L different sensitive values. For example, Table 1 is a raw data table that includes the information of patients: Age and Zip code are QIs; Disease is an SA. Table 2 is the released version of Table 1. We can find that each group includes at least L (L = 2) different SA values, and hence Table 2 satisfies L-diversity. Table 2 exploits Anatomy [10] to anonymise data. Anatomy is a classical anonymity method, where the SAs are separated from QIs by randomly permuting SA values in each anonymous group. Almost all the follow-up studies [9-24] about data publishing are based on K-anonymity or L-diversity. They only consider the value protection but not sensitivity protection. Let us see this example: TABLE 1. Original data table ID Age Zip Disease 1 25 100,033 HIV 2 34 100,032 Cancer 3 35 100,024 Morbus cardiacus 4 21 230,091 Appendicitis 5 24 390,012 Mild pancreatitis 6 32 530,025 Leukaemia 7 31 220,016 Cold 8 35 240,011 Fever TABLE 2. Table satisfies 2-diversity ID Age Zip Disease Group 1 25 100,033 Cancer 1 2 34 100,032 HIV 3 35 100,024 Appendicitis 2 4 21 230,091 Morbus cardiacus 5 24 390,012 Mild pancreatitis 3 6 32 530,025 Leukaemia 7 31 220,016 Fever 4 8 35 240,011 Cold Example 1.Similar to the example in Ref. [27], the adversary uses the sensitivity to disclose privacy. As a neighbour of Alice, Bob knows Alice is 25 years old and her zip code is 100033, Bob can link Alice to record t1 in the released table (Table 2) according to her QI values. Although group 1 includes two different diseases, both HIV and cancer are incurable diseases. Therefore, Bob can know that Alice is seriously ill. Table 2 can satisfy 2-diversity, but it cannot protect the sensitivity of privacy. To avoid this problem, the work in Ref. [25, 26] partitions SA values according to their sensitivity and try to protect the sensitivity of privacy. However, the work lacks specific sensitivity determination methods. In previous work [27], we presented a medical data publishing method, called (C, L)-diversity, which determines the sensitivity of disease information, and guarantees the diversity of sensitivity in anonymous groups. However, we had ignored some key factors of disease sensitivity. Besides the disease information with low sensitivity does not need sensitivity protection, the stringent grouping conditions of (C, L)-diversity results in unnecessary information loss. Faced with these problems, this paper presents a new method called Spreading. Our contributions can be summarised as follows: (1) An improved sensitivity determination method based on the previous work [27] is proposed. (2) A new medical data publishing method, called Spreading, is proposed to achieve the balance of information utility and privacy security. The new method focusses on the sensitivity protection of highly sensitive disease information, and the other disease information only needs to meet the privacy requirement of L-diversity. (3) The experiments conducted on real-world datasets verify the effectiveness of the new method. The rest of this paper is organised as follows. We discuss related research in Section 2 and recap some basic concepts, which are related to our approach in Section 3. In Section 4, the improved sensitivity determination method is proposed and we present the new data publishing method formally in Section 5. We evaluate the performance of Spreading in Section 6, and this paper is concluded in Section 7. 2 RELATED WORK A constellation of data publishing methods based on L-diverisity or K-anonymity was presented in recent years. The data publishing methods focus on different areas, even though the goals of them are to achieve the balance of information utility and privacy security. Some studies aim to improve the utility of the released table, without compromising the privacy requirement. To guarantee privacy security, it is necessary to anonymise data before releasing it. The information loss caused by anonymity methods is inevitable, and hence the related studies exploit effective anonymity methods to improve information utility. The common anonymity methods can be classified into two categories: generalisation and anatomisation. Generalisation. The specific values of records are replaced by the generalised ones in this category of anonymity methods. There is a lot of work that focusses on this area. To achieve K-anonymity effectively, LeFevre et al. [11] presented a full-domain generalisation method, called Incognito, which refers to dynamic programming [12]. Khaled et al. [13] proposed a K-anonymity algorithm that is globally optimal and applicable for de-identifying health datasets. The experimental results demonstrate the proposed algorithm has better performance than Incognito. To avoid excessive information loss, Khaled et al. [14] made the actual risk close to the threshold risk. Three different modifications to K-anonymity were evaluated, and guidelines for using various versions of K-anonymity were concluded. To prevent over-generalisation, Lee et al. [3] proposed a utility-preserving health data publishing method that limits the generalisation degree to a threshold h. The counterfeit records are included by equivalent classes for achieving K-anonymity, and data recipients can remove these counterfeit ones by the corresponding catalogue. The experimental results also validate the information utility of the proposed method. Considering that full-domain generalisation (global recoding) may cause unnecessary information loss, some local recoding methods [15, 16] were presented. Yaseen et al. [16] presented three generalisation hierarchy methods, of which Cardinality-Based Generalisation Hierarchy (CBGH) is based on the concept of local recoding or multi-valued generalisation. In CBGH, different nodes from the same level can have different distortions. The generalisation hierarchy tree is constructed from bottom to top, and the nodes with low cardinalities are merged. The experimental results imply that CBGH can provide better information utility than other generalisation methods. Anatomisation. The goal of Anatomisation is to de-link the association among attributes without modifying their values [19]. Xiao et al. [10] presented the original version of Anatomisation, called Anatomy, to meet the privacy requirement of L-diversity. Correspondingly, an algorithm based on a greedy strategy was proposed to achieve Anatomy. To separate SAs from QIs, Anatomy permutes the SA values randomly in each anonymous group. For example, Anatomy was exploited to anonymise Table 1, and we can get Table 2. Both theoretical and experimental analyses indicate that the proposed method outperforms Generalisation in information utility. Many improved or variant versions [17-20] were presented after this work. Li et al. [17] proposed a data publishing method, called Slicing, which is based on Anatomy. The proposed method puts the highly correlated attributes in the same column. The attributes from different columns are considered uncorrelated, and hence the associations among columns are broken. Slicing can meet the privacy requirement of L-diversity and provide membership protection. The experiments confirm that the proposed method can improve information utility greatly. Similarly, the data publishing method proposed by Yao et al. [19] meets the privacy requirements by table and group division. It is noteworthy that the presented table division algorithm considers the QI weight besides correlations between QIs. Compared with Slicing, the time consumption of the proposed method is reduced significantly, even though its information utility decreases slightly. Wang et al. [20] proposed a high-dimensional data publishing that combines Anatomisation with Generalisation approach. The raw data are divided into different disjointed data subsets, where both proportions of QIs and correlations between QIs are considered. Each subset is generalised by a new heuristic method, which is based on local recoding. The proposed methods can provide guaranteed information utility in the context of classification analysis. The privacy requirement may be compromised in various scenarios, even though Generalisation and Anatomisation are exploited to sanitise the raw data table. Therefore, some studies [21-27] aim to improve security by adding other anonymity techniques, without the significant erosion of information utility. The work in Ref. [21, 22] guarantees privacy security by limiting the frequency of SA values. Wong et al. [21] ensured that the frequency of each SA value in any anonymous group does not exceed the predefined threshold α, while the privacy requirement of K-Anonymity is met. The proposed method can provide better privacy than K-anonymity and is easy to be extended to the scenario of multiple SAs. Li et al. [22] pointed out that K-anonymity and L-diversity are insufficient to preserve privacy and proposed a data publishing method called t-closeness, which requires that the distribution of SA values in any anonymous group is close to the one in the overall table. The difference between these two distributions are bounded by the predefined threshold t. Wang et al. [9] presented a new privacy attack, called a distinguishability-based attack, by which the privacy information of outliers can be disclosed. Faced with this, a privacy model called plain L-diversity was defined. Besides, an efficient algorithm was proposed to achieve plain L-diversity with the consideration of characteristics of outliers. Majeed et al. [24] argued that some types of QI information are highly identity vulnerable and cause identification more easily. Hence Random forest [28] was exploited to determine the identity vulnerability of QIs. Besides, the diversity and evenness of an anonymous group were calculated by the Simpson index [29]. According to the identity vulnerability and diversity, the corresponding generalisation level was selected. The studies in Ref. [25-27] focus on sensitivity protection in data publishing. Huang et al. [25] proposed a privacy-preserving method that can limit the frequency of SA values with high sensitivity in any anonymous group, and a sensitivity protection framework was presented in follow-up work [26]. However, the lack of a specific sensitivity determination method blocks the development and application of these methods. In previous work [27], we presented a sensitivity protecting data publishing method, which is applicable for medical data. The proposed method determines the sensitivity of disease information semantically. However, it is still faced with two problems: (1) some key factors of disease sensitivity are ignored; (2) the stringent privacy requirement for low-sensitivity value may cause unnecessary information loss. 3 PRELIMINARIES In this section, we recap the common information utility measures, privacy attacks, and related cluster methods. For the convenience of reference, the major symbols used in this paper are summarised in Table 3. Assume the original data table T contains n records with n ≥ 2, T = {t1, t2, … , tn}. Each record t has the form t = {Q1, Q2, … ,Qd, S}, where {Q1, Q2, …, Qd} are d quasi-identifiers, and S is the sensitive attribute. This paper focusses on medical data protection, and hence S actually refers to the Disease attribute. Besides, n u m G ( H S D ) , S ( G ) and N will be mainly used by the description of the proposed method (Section 5). TABLE 3. Meanings of symbols Symbol Description t A record T An original medical data table, T = {t1, t2, …, tn} |T| The number of records in T Q A quasi-identifier attribute S A sensitive attribute A A quasi-identifier or sensitive attribute t.A The value of t in attribute A G A group |G| The number of records in G n u m G ( H S D ) The number of highly sensitive diseases in G S ( G ) The set of diseases included by G N The number of highly sensitive diseases in ungrouped records Information utility measures and privacy attacks Definition 3.1.(Additional information loss). This method is exploited to measure the quality of released tables. Let the released table T ∼ contain m groups, then additional information loss (AIL) [30] of T ∼ is: A I L T ∼ = ∑ G ∈ T ∼ ( | G | − L ) / m L (1) Definition 3.2.(Discernability metric). This method measures the quality of released tables by the size of anonymous groups. Assume there are m groups in released table T ∼ , then the discernability metric (DM) [8] of T ∼ is: D M T ∼ = ∑ G ∈ T ∼ | G | 2 (2)where larger DM means larger anonymous groups and less information utility. Definition 3.3.(Link attack). Link attack [6] would occur if the adversary can re-identify the targeted individual by linking the QI values to some other datasets.The adversary uses QI values in combination to disclose privacy in Link attack, and hence almost all the anonymous methods based on K-anonymity can thwart this attack. However, K-anonymity focusses on QI values without the consideration of SA values and is vulnerable to Homogeneity Attack. Definition 3.4.(Homogeneity attack). A homogeneity attack [8] would occur if any anonymous group lacks diversity on the SA values.Most of the records may have the same SA values in an anonymous group, if there is a lack of diversity. The adversary can disclose the SA value with a high probability, even though he/she cannot identify the record of the targeted individual. L-diversity and its variants can defend against this attack. However, almost all the previous work focus on value protection, without the consideration of sensitivity protection. Hence, Huang et al. [25] proposed a new type of privacy attack: Definition 3.5.(Sensitivity attack). Sensitivity Attack [25] would occur if any anonymous group lacks limits on the SA value sensitivity. There may be a lot of highly sensitive SA values included by an anonymous group, if there is a lack of limits on SA value sensitivity. As illustrated in Example 1, the adversary may disclose privacy as long as he/she can locate the targeted individual to a specific anonymous group. Unfortunately, most of the previous methods cannot protect privacy from this attack well. Clustering Similarities in data can be found and similar data is put into groups in the clustering process [31, 32]. K-means [33-36] is one of the most popular clustering methods, a dataset can be partitioned into K disjoint clusters by it. The number of iterations is usually less than that of points in K-means, which converges quickly [37]. Some optimisation methods [37-39] for K-means were also presented recently. Bifulco et al. [37, 38] executed the K-means algorithm by multiple random starting points, in order to find different approaches to the same problem. Chabih et al. [39] proposed a method to estimate the optimal number of clusters. The presented method detects and observes the reference clusters during the multiple clustering processes and finds the optimal number of clusters. 4 ANALYSIS OF DISEASE CHARACTERISTICS To protect the sensitivity of disease information, it is necessary to divide information according to sensitivity. Our previous work in Ref. [27] analyses the characteristics of diseases from the privacy protection and clusters the disease information. We review this method [27] and supplement it in this section. The disclosure of sensitive information will harm individuals, and hence privacy should be protected in data publishing. If the disease information of an individual is disclosed, the physical condition of the patient will be known. In addition, for some special diseases, disclosure may bring extra harm. Therefore, from the perspective of privacy protection, two characteristics can be summarised for diseases. That is, we analyse the sensitivity of diseases from two aspects: the effect on health and other effect. Both two characteristics determine the sensitivity of diseases. The effect on health Diseases undermine the health of bodies, even can pose a severe threat to the life of patients. A more serious effect of a disease on health means more serious invasion of privacy, if the disease information of an individual is disclosed. Example 2.If Alice suffers from HIV unfortunately, the disclosure of her disease information will be very harmful to her privacy. On the contrary, if Alice only catches a cold, the disclosure is much less of an intrusion on privacy. With the above analysis, the disease has higher sensitivity with a more serious effect on health. Therefore, diseases can be divided into four levels with the consideration of effects on health: Level 1: mild symptoms, patients usually do not have to be hospitalised for these diseases, for example, cold; Level 2: patients need to be hospitalised for recovery; Level 3: serious illnesses, it is necessary for recovery to take a surgery, or the disease is not curable but will not pose a threat to life; Level 4: fatal diseases, they cannot be cured and will finally rob lives of patients. Compared with the previous work [27], we add a type of disease to Level 3. Other effects The harm brought by diseases is mainly reflected in undermining health. The disclosure of disease information can make the health condition of patients known, which will bring harm to the privacy of patients. However, for some special diseases, the disclosure may bring extra negative influence. Example 3.If Alice suffers from HIV and her privacy is disclosed unfortunately, other people know that the illness she suffers from cannot be cured. Furthermore, Alice may receive moral condemnation by society, especially in a conservative country. Therefore, compared with other incurable patients, the disclosure will put even more pressure on Alice. The aspect B of the previous work [27] only considers the moral pressure, but the disclosure of some disease information may bring other inconveniences. Let's see the following example. Example 4.Assume that Bob is a patient with sexual dysfunction. In general, this disease can be cured by drug treatment or psychotherapy treatment. If this information is disclosed, Bob may receive funny looks from other people. Compared with other treatable conditions, Bob is under extra pressure from public opinions if the disease information is disclosed. Therefore, we present an improved version of aspect B of Ref. [27]: Level 1: The disclosure of disease information only will make the health condition of patients known; Level 2: The disclosure of disease information can make the health condition of patients known. Furthermore, patients may receive moral condemnation or will be under pressure from public opinions. Sensitivity determination After the previous analysis, two characteristics of diseases can be obtained: the effect on health and other effects. These two characteristics are represented with variable A and variable B, respectively. Assign corresponding order and weight to A and B according to the method in Ref. [27, 40], and the results can be seen in Table 4. In applications, the proper weight can be assigned to diseases after consulting the medical expert. Table 5 shows the weights of diseases in Table 1. Assume u i , u j are two diseases, and u i w ( A ) and u j w ( A ) represent the weight of u i and u j in A, respectively. u i w ( A ) > u j w ( A ) means u i has higher sensitivity than u j in A. Thus, we use these weights to cluster disease, and the diseases that have similar sensitivity can be put into the same cluster. Similar to the previous work [27], K-means is used to cluster. However, we had not determined the number of clusters in the previous work [27] and the choice of the cluster number may significantly affect the clustering result [39]. To solve this, we utilise the approach in Ref. [39] to determine the reasonable number of clusters. TABLE 4. Description of disease sensitivity Level (A. the effect on health) Order Weight Level (B other effect) Order Weight 1 1 0 1 1 0 2 2 1/3 ≈ 0.33 2 2 1 3 3 2/3 ≈ 0.66 \ \ \ 4 4 1 \ \ \ TABLE 5. Weight of diseases Disease Weight (A) Weight (B) HIV 1 1 Cancer 1 0 Fever 0 0 Appendicitis 0.66 0 Mild pancreatitis 0.33 0 Leukaemia 1 0 Cold 0 0 Morbus cardiacus 0.66 0 The proposed method needs to distinguish the highly sensitive value from disease information, and hence we define the sensitivity as follows. Definition 4.1.(Sensitivity of diseases). The sensitivity of disease u i is defined as: u i s = u i w ( A ) + u i w ( B ) (3) Disease u i is more sensitive than disease u j , if u i s > u j s . The disclosure of highly sensitive value will be much more of an intrusion on privacy. Definition 4.2.(Sensitivity of clusters). Let β denote the disease set included by the cluster ch. The sensitivity of ch is defined as follows: c h s = ∑ u i ∈ β u i s / | β | (4)where | β | represents the number of diseases in β . For example, uses K-means (k = 2) to cluster diseases according to Table 5, get two clusters, c1 = {HIV, cancer, and leukaemia} and c2 = {cold, fever, morbus cardiacus, appendicitis, and mild pancreatitis}. According to the above equations, it is easy to know c 1 s > c 2 s , which means the diseases in c1 are more sensitive than the ones in c2. 5 PROPOSED METHOD To ensure the diversity of disease sensitivity in groups, (C, L)-diversity requires that each group includes at least C values from different clusters. Assume Table 1 is the raw medical data table, and we have H = 1, C = 2, and L = 2. According to Table 4, diseases can be divided into two clusters: c1 = {HIV, cancer, and leukaemia} and c2 = {cold, fever, morbus cardiacus, appendicitis, and mild pancreatitis}, and Table 6 is a released table for Table 1. For each group of Table 6, it includes at least two different diseases, in addition, there are at least two diseases that come from different clusters. Hence Table 6 meets the requirements of (C, L)-diversity. However, for some clusters, which includes low-sensitivity diseases, the grouping conditions of (C, L)-diversity are too stringent to guarantee information utility. For example, t7 and t8 make up group 4 in Table 2. Although group 4 cannot satisfy (2, 2)-diversity, it is difficult to pose a threat to privacy because of the low sensitivity of the disease information in group 4. In order to meet the requirement of (2,2)-diversity, the remaining records t7 and t8 have to be added to group 1 and group 2, respectively in Table 6. Hence the larger size of anonymous groups in Table 6 may bring more information loss, for example, Additional information loss [30] of these related records increases. As the extended version of (C, L)-diversity, Spreading focusses on protecting the diseases information with high sensitivity, and it can achieve the balance between information utility and privacy security. TABLE 6. Released table of (C, L)-diversity ID Age Zip Disease Group 1 25 100,033 Cold 1 3 35 100,024 HIV 7 31 220,016 Morbus cardiacus 2 34 100,032 Fever 2 4 21 230,091 Appendicitis 8 35 240,011 Cancer 5 24 390,012 Leukaemia 3 6 32 530,025 Mild pancreatitis Spreading mode To guarantee the information utility, Spreading provides sensitivity protection for highly sensitive diseases (HSDs). We say the disease is highly sensitive if it is included by a highly sensitive cluster. Definition 5.1.(Highly sensitive clusters). Users can choose some clusters have high sensitivity as highly sensitive clusters (HSCs). Like the previous methods, Spreading controls the level of privacy protection by parameters. The related parameters can be set by users, and we define them as follows. H. H clusters that have the highest sensitivity are HSCs. Common sense suggests that