# Metadata Quality for Biobanks
## 6 Metadata Quality Characteristics and Metrics
### Metadata Reliability
#### 6.8.2 Calculating Metadata Reliability Metrics  
Following [10], we define an indirect metadata quality metric as the metric based not on the measurement of the metadata value itself, but on the quality of an element (or elements) of its context. The quality of the metadata value has to be calculated based on that context quality. In this paper, we only deal with _indirect metadata reliability metrics_ based on the reliability metrics defined for metadata sources (following [PERSON] [55]), collection methods, or related standards. As the context of the metadata value includes _the metadata source_, we further limit ourselves to the metrics based on the _reliability of metadata sources_. For example, the data item completeness metadata for the BMI data item attribute for a collection will be more reliable if it comes from the biobank itself, and not from the non-trusted external source. Here we will not deal with calculating the reliability of the metadata sources in detail; we just assume that these _source reliability values_ are available. A detailed treatment of metadata source reliability metrics will be the topic of the follow-up paper. After taking hold of the metadata source reliability values, the next step is to connect them to the biobank metadata elements. It can be done in one of the following ways:  
1. The description of the biobank collection includes the declarations of the metadata sources. The scope of these sources is limited to all metadata attributes connected to the data item attributes declared for this collection. They are instantiated for some or all the values of the metadata attributes describing the values of all data item attributes instantiated by the samples belonging to this collection. 2. The description of the data item attribute declared for a biobank collection includes the declaration of the metadata sources. The scope of such sources is limited to the metadata attributes connected to this data item attribute. They are instantiated for the values of all metadata attributes describing the values of this data item attribute instantiated by the samples belonging to this collection. 3. The description of the metadata attribute declared for a biobank collection includes the declaration of the metadata sources. The scope of such sources is limited to this metadata attribute. They are instantiated for the values of all data item attributes instantiated by the samples belonging to this collection. A sample-level aggregate metadata reliability metric can be calculated as _the average degree of source reliability for a specific sample_. It is calculated by aggregating the source reliability values connected to all metadata attribute values belonging to a sample for which the source information is available:  
\[MRel_{s}^{avgM}=\frac{\sum\limits_{a^{M}\in A_{C(s)}^{M}}Rel(r_{a^{M}}(s))}{n(A _{C(s)}^{M})}, \tag{22}\]  
where \(A_{C(s)}^{M}\)--a set of all metadata attributes declared for the collection \(C(s)\) containing the sample \(s\), for which the metadata source information is available, \(Rel(r_{a^{M}}(s))\) is a reliability value of the metadata source \(r_{a^{M}}(s)\) which provides the value for the metadata attribute \(a^{M}\) belonging to a sample \(s\). To simplify calculations, here and below, we assume that all values of a specific metadata attribute describing data item attributes for a specific sample are produced by the same metadata source (with the same reliability), and that this source can be different for different samples. For example, all data item consistency values describing data item attributes (patient age, disease code, etc.) for a given sample are produced by the same source; still, their source can be different for other samples. Suppose the sample \(s_{1}\) instantiates data item attribute values described by the data item completeness metadata attribute with the source reliability of 0.7, data item provenance metadata attribute with the source reliability of 0.8, and the data item precision metadata attribute with the source reliability of 0.9.