Project: YugabyteDB (App Definition and Development / Database)
File: 4-generalization.md

The main idea of generalization is to \"blur\" the original data. For example, instead of saying \"Mister X was born on July 25, 1989\", we can say \"Mister X was born is the 80\'s\". The information is still true, but it is less precise and it can\'t be used to reidentify the subject. Paul hired dozens of employees over the years. He kept a record of their hair color, size and medical condition. Paul wants to extract weird stats from these details. He provides generalized views to Pierre. In this section, we will learn: The difference between masking and generalization The concept of \"K-anonymity\" ``` { .run-postgres parse_query=False } DROP TABLE IF EXISTS employee CASCADE; CREATE TABLE employee ( id INT PRIMARY KEY, full_name TEXT, firstday DATE, lastday DATE, height INT, hair TEXT, eyes TEXT, size TEXT, asthma BOOLEAN, CHECK(hair = ANY(ARRAY['bald','blond','dark','red'])), CHECK(eyes = ANY(ARRAY['blue','green','brown'])) , CHECK(size = ANY(ARRAY['S','M','L','XL','XXL'])) ); ``` ::: caution This is awkward and illegal. ::: Loading the data: ``` bash curl -Ls https://dali.bo/employee -o /tmp/employee.tsv head -n3 /tmp/employee.tsv ``` ``` run-postgres COPY employee FROM '/tmp/employee.tsv' ``` ``` run-postgres SELECT count(*) FROM employee; ``` ``` run-postgres SELECT fullname,firstday, hair, size, asthma FROM employee LIMIT 3; ``` Paul wants to find if there\'s a correlation between asthma and the eyes color. He provides the following view to Pierre. ``` run-postgres DROP MATERIALIZED VIEW IF EXISTS vasthmaeyes; CREATE MATERIALIZED VIEW vasthmaeyes AS SELECT eyes, asthma FROM employee; ``` ``` run-postgres SELECT * FROM vasthmaeyes LIMIT 3; ``` Pierre can now write queries over this view. ``` run-postgres SELECT eyes, 100*COUNT(1) FILTER (WHERE asthma) / COUNT(1) AS asthma_rate FROM vasthmaeyes GROUP BY eyes; ``` Pierre just proved that asthma is caused by green eyes. The \'asthma\' and \'eyes\' are considered as indirect identifiers. ``` run-postgres SECURITY LABEL FOR anon ON COLUMN vasthmaeyes.eyes IS 'INDIRECT IDENTIFIER'; SECURITY LABEL FOR anon ON COLUMN

IS 'INDIRECT IDENTIFIER'; ``` ``` run-postgres SELECT anon.kanonymity('vasthma_eyes'); ``` The vasthmaeyes has \'2-anonymity\'. This means that each quasi-identifier combination (the \'eyes-asthma\' tuples) occurs in at least 2 records for a dataset. In other words, it means that each individual in the view cannot be distinguished from at least 1 (k-1) other individual. ``` run-postgres DROP MATERIALIZED VIEW IF EXISTS vstaffper_month; CREATE MATERIALIZED VIEW vstaffper_month AS SELECT anon.generalizedaterange(firstday,'month') AS first_day, anon.generalizedaterange(lastday,'month') AS last_day FROM employee; ``` ``` run-postgres SELECT * FROM vstaffper_month LIMIT 3; ``` Pierre can write a query to find how many employees were hired in november 2021. ``` run-postgres SELECT COUNT(1) FILTER ( WHERE make_date(2019,11,1) BETWEEN lower(first_day) AND COALESCE(upper(last_day),now()) ) FROM vstaffper_month; ``` Now let\'s check the k-anonymity of this view by declaring which columns are indirect identifiers. ``` run-postgres SECURITY LABEL FOR anon ON COLUMN vstaffpermonth.firstday IS 'INDIRECT IDENTIFIER'; SECURITY LABEL FOR anon ON COLUMN vstaffpermonth.lastday IS 'INDIRECT IDENTIFIER'; SELECT anon.kanonymity('vstaffpermonth'); ``` In this case, the k factor is 1 which means that at least one unique individual can be identified directly by his/her first and last dates. Generalizing dates per month is not enough. Write another view called \'vstaffper_year\' that will generalize dates per year. Also simplify the view by using a range of int to store the years instead of a date range. How many people worked for Paul for each year between 2018 and 2021? What is the k-anonymity of 'vstaffpermonthyears'? ``` run-postgres DROP MATERIALIZED VIEW IF EXISTS vstaffper_year; CREATE MATERIALIZED VIEW vstaffper_year AS SELECT int4range( extract(year from first_day)::INT, extract(year from last_day)::INT, '[]' ) AS period FROM employee; ``` :::tip '[]' will include the upper bound ::: ``` run-postgres SELECT * FROM vstaffper_year LIMIT 3; ``` ``` run-postgres SELECT year, COUNT(1) FILTER ( WHERE year <@ period ) FROM generate_series(2018,2021) year, vstaffper_year GROUP BY year ORDER BY year ASC; ``` ``` run-postgres SECURITY LABEL FOR anon ON COLUMN vstaffper_year.period IS 'INDIRECT IDENTIFIER'; SELECT anon.kanonymity('vstaffperyear'); ```