Business Glossary support in Google Data Catalog
By: Ricardo Mendes
Tags: Google Cloud Platform, Business Glossary, Egeria, Data Catalog, Data Governance

An Egeria-based custom model

There are multiple Business Glossary providers, each one with their proprietary entity models. Here Egeria comes into the scene: it is an Open Metadata and Governance project which promotes metadata exchange between tools and platforms.

Egeria’s metadata types are open, this is why I’m going to use them as a reference to explain the custom model in Google Data Catalog. But, anyway, the model is expected to be adaptable/extensible to fit real-life requirements even when the glossary metadata come from other sources — hopefully, my reasoning will be good enough to let it clear for you :).

The Open Metadata Types are organized in 7 areas, being Glossary and Semantics covered in Area 3.

More than providing the Open Metadata Types, Egeria actually works as an enterprise metadata “broker”. Although it is possible to connect Egeria and Data Catalog through the Open Connector Framework (OCF), this is out of this article’s scope. The focus here is on designing a custom model for Business Glossary support in Data Catalog, simply leveraging Egeria types as a reference.

Mapping Entities and Relationships into Entries, Templates, and Tags

Egeria has an extensive set of classes to fully represent a Business Glossary no matter what tool it comes from. For the sake of clarity, I will keep the sample model concise yet practical to describe how introductory Open Metadata Entities and Relationships are mapped into Data Catalog Custom Entries, Templates, and Tags. Mapping entities from one end to another is usually the most difficult part of the job and extending the model is a matter of adding new classes to the below set.

Let me map the Glossary, Glossary Term, and Semantic Assignment from Egeria to Data Catalog, starting from briefly explaining them for readers who are new to Egeria:

A Glossary is a collection of related semantic definitions, represented as the Glossary entity in the Open Metadata Type System;

entity in the Open Metadata Type System; “The vocabulary for the Glossary is documented using Terms. Each Term represents a concept of short phrase in the vocabulary.” A Term, represented as the GlossaryTerm entity, is owned by a Glossary;

entity, is owned by a Glossary; The Semantic Assignment, represented as the SemanticAssignment relationship, is used to assign a Term to a given asset (e.g. BigQuery table or column), which means the Term describes the meaning of that asset.

Please take a look at the below diagrams, they bring a visual presentation of the proposed mappings. Bear in mind the grayed classes’ stereotypes are actual Data Catalog types, while the class names are clues to the Open Metadata Types their instances refer to.

Glossary and Glossary Term

The Glossary itself is mapped as a Custom Entry with userSpecifiedType = business_glossary . Nothing special here…

Class diagram 1: Egeria Glossary and GlossaryTem mapped as Google Data Catalog entities

Glossary Terms are mapped as Custom Entries with userSpecifiedType = glossary_term . The standard fields of a Custom Entry are not enough to persist all Glossary Term information and need to be extended at some point. To fulfill this requirement, we can leverage Data Catalog’s flexibility and use Tags created from a particular Tag Template called Glossary Term Specification — their fields increase the Custom Entry metadata storage capabilities. Each Glossary Term Entry should have a Tag created from this Template to store metadata gathered from its underlying Glossary Term in Egeria.

This design also enables us to settle additional features. Linking Terms to their parent Glossary, for instance: the glossaryName field of a Glossary Term Specification Tag can be used to find all Terms that belong to a given Glossary. Another example: the guid field, which uniquely identifies a Glossary Term no matter where it is stored, can be used to keep the Terms synchronized between Egeria and Data Catalog.

Semantic Assignment

Other predefined Tag Templates derive from each Glossary Term to enable their Semantic Assignments — 1:1, which means if there are two Glossary Terms, Customer name and Street name, there will be two Glossary Term Semantic Assignment Tag Templates identified by something like semantic_customer_name and semantic_street_name . They are used to create Tags that represent the Semantic Assignments, linking Glossary Terms to their related assets.

Class diagram 2: Egeria SemanticAssignment mapped as Google Data Catalog entities

Please notice the Glossary Term Semantic Assignment Tags, created from those particular Templates, are used to describe the meaning of ordinary Data Catalog Entries or Schema Columns.

The rationale behind this design is that at the time this document has been written, November 2020, this is a feasible way to set more than one Semantic Assignment to an asset. Data Catalog currently supports attaching only a single Tag per Template on a given asset.

Supporting more features

Now you’ve got the foundation to increment the model by adding more features to the Data Catalog Glossary according to your needs. Would you like to try bringing the Glossary Category entity and its related Term Categorization relationship to Data Catalog as an exercise? Or maybe something related to Synonyms…