SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[22] Alexandre Lacoste, Evan David Sherwin, Hannah Kerner, Hamed Alemohammad, Björn Lütjens, Jeremy Irvin, David Dao, Alex Chang, Mehmet Gunturkun, Alexandre Drouin, Pau Rodríguez, and David Vázquez. Toward Foundation Models for Earth Monitoring: Proposal for a Climate Change Benchmark. In NeurIPS Workshops, 2021.
[23] Xiao Xiang Zhu, Devis Tuia, Lichao Mou, Gui-Song Xia, Liangpei Zhang, Feng Xu, and Friedrich Fraudorfer. Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources. IEEE Geoscience and Remote Sensing Magazine, 2017.
[24] Kumar Ayush, Burak Uzkent, Chenlin Meng, Kumar Tanmay, Marshall Burke, David Lobell, and Stefano Ermon. Geography-Aware Self-Supervised Learning. In ICCV, 2021.
[25] Oscar Manas, Alexandre Lacoste, Xavier Giró-i Nieto, David Vazquez, and Pau Rodríguez. Seasonal Contrast: Unsupervised Pre-Training From Uncurated Remote Sensing Data. In ICCV, 2021.
[26] Yezhen Cong, Samar Khanna, Chenlin Meng, Patrick Liu, Erik Rozi, Yutong He, Marshall Burke, David B. Lobell, and Stefano Ermon. SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery. In NeurIPS, 2022.
[27] Colorado J. Reed, Ritwik Gupta, Shufan Li, Sarah Brockman, Christopher Funk, Brian Clipp, Kurt Keutzer, Salvatore Candido, Matt Uytendaele, and Trevor Darrell. Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning. CoRR, abs/2212.14532, 2023.
[28] Aäron van den Oord, Yazhe Li, and Oriol Vinyals. Representation Learning with Contrastive Predictive Coding. CoRR, abs/1807.03748, 2018.
[29] Philip Bachman, R Devon Hjelm, and William Buchwalter. Learning Representations by Maximizing Mutual Information Across Views. In NeurIPS, 2019.
[30] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A Simple Framework for Contrastive Learning of Visual Representations. In ICML, 2020.
[31] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dólár, and Ross Girshick. Masked Autoencoders Are Scalable Vision Learners. In CVPR, 2022.
[32] Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. BEiT: BERT Pre-Training of Image Transformers. In ICLR, 2022.
[33] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL, 2019.
[34] Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive Multiview Coding. In ECCV, 2020.
[35] Michael Tschannen, Josip Djolonga, Marvin Ritter, Aravindh Mahendran, Neil Houlsby, Sylvain Gelly, and Mario Lucic. Self-Supervised Learning of Video-Induced Visual Invariances. In CVPR, 2020.
[36] Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, and Phillip Isola. What Makes for Good Views for Contrastive Learning? In NeurIPS, 2020.
[37] Tete Xiao, Xiaolong Wang, Alexei A Efros, and Trevor Darrell. What Should Not Be Contrastive in Contrastive Learning. In ICLR, 2021.
[38] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum Contrast for Unsupervised Visual Representation Learning. In CVPR, 2020.
[39] Xinlei Chen and Kaiming He. Exploring Simple Siamese Representation Learning. In CVPR, 2021.
[126] E Raymond Hunt, Jr. Relationship between woody biomass and PAR conversion efficiency for estimating net primary production from NDVI. International Journal of Remote Sensing, 1994.
[127] Bo-Cai Gao. NDWI—A normalized difference water index for remote sensing of vegetation liquid water from space. Remote sensing of environment, 1996.
[128] Zhao-Liang Li, Pei Leng, Chenghu Zhou, Kun-Shan Chen, Fang-Cheng Zhou, and Guo-Fei Shang. Soil moisture retrieval from remote sensing measurements: Current knowledge and directions for the future. Earth-Science Reviews, 2021.
[129] Compton J Tucker. Remote sensing of leaf water content in the near infrared. Remote sensing of Environment, 1980.
[130] Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilic, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Galle, et al. BLOOM: A 176B-Parameter Open-Access Multilingual Language Model. CoRR, abs/2211.05100, 2022.
[131] Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katherine Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Oriol Vinyals, Jack William Rae, and Laurent Sifre. Training Compute-Optimal Large Language Models. In NeurIPS, 2022.
[132] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. LLaMA: Open and Efficient Foundation Language Models. CoRR, abs/2302.13971, 2023.
[133] Kan Wu, Houwen Peng, Minghao Chen, Jianlong Fu, and Hongyang Chao. Rethinking and Improving Relative Position Encoding for Vision Transformer. In ICCV, 2021.
[134] David Arthur and Sergei Vassilvitskii. K-Means++: The Advantages of Careful Seeding. In Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, 2007.
[135] H. W. Kuhn. The Hungarian method for the assignment problem. Naval Research Logistics Quarterly, 1955.
Code and pretrained models: https://github.com/antofuller/CROMA
A.1 Other Pretraining Experiments
In this section, we experiment with additional CROMA settings. We use the same experimental conditions as §4.1 of our paper; i.e., we linear probe representations on BigEarthNet [76] (reporting mAP on the combined validation and test sets) and patch encodings on DW-Expert-120 [88] (reporting mIoU on the validation set). We use the linear probing hyper-parameters listed in §A.4.2 of this Appendix.
Table 8: Linear probing results on radar-only ("R"), optical-only ("O"), and joint radar-optical ("RO") inputs. Across all experiments, we use 2D-ALiBi with X-ALiBi,  $75\%$  shared masking, ViT-B backbones, and 100 pretraining epochs.
Cross-Modal	Cross-Modal	Decoder	Obj. Weights	HN Mixing	Classification (mAP)	Segmentation (mIoU)
Image Obj.	Patch Obj.	Depth, Dim	\( {\lambda }_{\text{Con }} \), \( {\lambda }_{\text{MAE }} \)	(1024, 0, n)	Cost	R	O	RO	R	O	RO
InfoNCE	MSE	1, 512	1, 1	✘	1×	77.4	83.9	84.3	40.5	56.0	56.7
InfoNCE	MSE	1, 768	1, 1	✘	1×	77.4	83.9	84.3	40.7	56.1	56.6
InfoNCE	MSE	3, 512	1, 1	✘	1.2×	77.5	83.9	84.4	40.5	56.3	57.1
InfoNCE	MSE	3, 768	1, 1	✘	1.3×	77.5	83.9	84.4	40.7	56.2	56.6
InfoNCE	MSE	6, 512	1, 1	✘	1.4×	77.5	83.9	84.4	40.3	56.0	56.7
InfoNCE	MSE	6, 768	1, 1	✘	1.6×	77.6	83.8	84.5	40.6	56.2	56.7
InfoNCE	✘	1, 512	1, 1	✘	1×	77.4	84.0	84.5	40.8	56.1	56.4
InfoNCE	✘	1, 768	1, 1	✘	1×	77.5	84.2	84.5	40.8	56.1	56.2
InfoNCE	✘	3, 512	1, 1	✘	1.2×	77.6	84.1	84.5	40.8	56.2	56.7
InfoNCE	✘	3, 768	1, 1	✘	1.3×	77.0	83.9	84.5	40.6	56.1	56.5
InfoNCE	✘	6, 512	1, 1	✘	1.4×	77.3	84.1	84.5	40.8	56.1	56.5
InfoNCE	✘	6, 768	1, 1	✘	1.6×	77.5	84.1	84.6	40.6	56.5	56.8
InfoNCE	InfoNCE	3, 512	1, 1	✘	2.2×	72.8	80.9	82.4	39.0	55.1	55.2
InfoNCE	✘	1, 512	1, 2	✘	1×	77.5	84.3	84.2	40.7	55.9	56.2
InfoNCE	✘	1, 512	1, 4	✘	1×	77.5	84.3	84.1	40.6	55.4	56.0
InfoNCE	✘	1, 512	2, 1	✘	1×	77.5	84.1	84.5	40.4	55.9	56.3
InfoNCE	✘	1, 512	4, 1	✘	1×	77.6	83.9	84.5	40.7	55.8	56.8
InfoNCE	✘	1, 512	1, 1	128	1×	73.6	81.6	83.0	38.0	53.2	55.0
InfoNCE	✘	1, 512	1, 1	256	1×	73.0	81.0	82.8	37.8	52.9	54.7
InfoNCE	✘	1, 512	1, 1	512	1×	72.5	80.2	82.4	37.6	52.6	54.4
VICReg	MSE	1, 768	1, 1	✘	1.1×	70.7	78.7	83.3	40.0	55.5	55.1
Self-supervised Objectives. Inspired by the local objective of VICRegL [93], we experiment with a mean squared error (MSE) objective between cross-modal patch encodings, i.e.,  $\mathcal{L}_{local} = \mathrm{MSE}(\mathcal{E}_R,\mathcal{E}_0)$ . This attracts patch encodings if they match locations, i.e., if they represent the same  $80\mathrm{m}\times 80\mathrm{m}$  square on the ground. We find this does not improve representations. Next, we experiment with the VICReg [92] objective (calculating VICReg statistics based on a batch size of 800) between cross-modal image representations, i.e.,  $\mathcal{R}_{\mathrm{R}}$  and  $\mathcal{R}_0$ ; we find it underperforms InfoNCE [28]. Finally, we experiment with the InfoNCE objective between cross-modal patch encodings; positive pairs are encodings that match locations across modalities, and negative pairs are all other encodings from the matched sample and encodings from all other samples in the batch. This does not improve representations and slows pretraining by  $2.2\times$  (Table 8).
Objective Weights. We find that weighting the contrastive loss term or MAE [31] loss term does not uniformly improve representations; hence, we select equal weights.
Hard Negatives. We find that hard-negative mixing [95] ( $N = 1024$ ,  $s = 0$ ,  $s' = n$ ,  $\beta = 0.5$ , with  $n$  of 128, 256, or 512) degrades performance when used in our framework.
Decoder Sizes. At least in these experiments, CROMA is not sensitive to the decoder size; a tiny decoder with a 1-layer, 512-d transformer performs similarly to a much larger 6-layer, 768-d transformer.
Position Encoding with Shared Masking. We find that using 2D-sinusoidal embeddings or PEG [90] with shared masking performs poorly. These two methods of position encoding store positional information in the internal representations, which can help solve the contrastive objective if both modalities share masks; 2D-ALiBi instead stores positional information in the attention matrix, which may pre
vent this from occurring. In our paper (Table 6), we show that 2D-sinusoidal or PEG can perform well in our framework if modalities are masked independently, although 2D-ALiBi still outperforms these approaches.
Table 9: Linear probing results with shared  $75\%$  masking, ViT-B, 100 epochs.
Method	Classification
mAP	Segmentation
mIoU
R	O	RO	R	O	RO
PEG [90]	67.9	75.9	79.0	32.6	49.8	51.0
2D-Sin.	69.4	75.6	79.8	29.0	44.1	50.7
Lower Masked Tuning. FLIP [68] performs contrastive learning using the representations of masked-out samples; after this masked pretraining, it leverages unmasked tuning to increase accuracy by  $1.3\%$  on zero-shot ImageNet-1K. Unmasked tuning continues FLIP pretraining by performing contrastive learning using the representations of unmasked samples to reduce the distribution gap between pretraining and inference [68]. We cannot perform fully unmasked
tuning because we must mask patches for our reconstruction objective. However, we can lower our mask ratio and perform lower masked tuning. Following FLIP, initializing parameters with our pretrained CROMA-L model, we train for 5 additional epochs using a base learning rate of 8e-8, warmup over the first epoch, and cooldown for 4 epochs using a cosine decay schedule. We explore mask ratios  $\{10\%, 25\%, 50\% \}$  and find that lower masked tuning does not improve linear probing accuracy for CROMA (Table 10).
Table 10: Lower masked tuning for 5 epochs after pretraining CROMA-L.
Mask Ratio	Classification
mAP	Segmentation
mIoU
R	O	RO	R	O	RO
10%	80.8	84.7	84.7	43.8	56.8	56.6
25%	80.8	84.7	84.8	43.9	56.8	56.6
50%	80.8	84.8	85.0	43.9	56.8	56.6
A.2 Two Reasons for 2D-ALiBi's Performance
Our primary reason for introducing 2D-ALiBi is to enable the test-time extrapolation demonstrated in §4.2. But even when no extrapolation occurs—training and testing on the same image size—2D-ALiBi outperforms both the most commonly used position encoding method (2D-sinusoidal) and a SoTA relative position encoding method (PEG [90]). We believe 2D-ALiBi outperforms these two methods for two reasons.
Table 11: Cosine similarity between the representations of images and the representations of transformed versions of the same images. Higher similarity means the representations are less influenced by the image transformation.
Method	Transformation (cosine similarity)
H-Flip	V-Flip	rotate(90°)	rotate(180°)	rotate(270°)
2D-ALiBi + X-ALiBi	0.992	0.992	0.992	0.992	0.992
2D-ALiBi	0.992	0.992	0.992	0.992	0.992
PEG [90]	0.992	0.992	0.988	0.992	0.988
2D-sinusoidal	0.641	0.629	0.523	0.559	0.523
12D-ALiBi learns image representations invariant to rotating and flipping. Learning representations that are invariant to certain transformations is often desirable. In CROMA, the contrastive objective between optical and radar data encourages sensor-invariant representations. Invariances to flipping and rotating are desirable properties of the representations of satellite imagery. However, CROMA's pretraining objectives do not explicitly encourage these invariances. To investigate if CROMA's optical representations (i.e.,  $\mathcal{R}_0$ ) are invariant to flipping and rotating, we produce representations of 5,000 optical images from DW-Expert-120 for four position encoding strategies—2D-ALiBi, 2D-ALiBi with X-ALiBi, PEG, and 2D-sinusoidal. Then, for each model, we produce representations of the same samples but transformed. We measure the cosine similarity between the original and transformed images' representations. When used in our CROMA framework, 2D-ALiBi learns representations invariant to flipping and rotating (average cosine similarity of 0.992, Table 11). Notably, 2D-sinusoidal learns representations that are not invariant to flipping and rotating
(average cosine similarity of 0.575, Table 11); we believe this contributes to the poor performance of 2D-sinusoidal embeddings on image classification. PEG performs similarly to 2D-ALiBi on image classification and also learns representations invariant to flipping and rotating.
Table 12: (Middle column) Cosine similarity between patch encodings at different locations within an image, averaged across 5,000 images. (Right column) Cross entropy loss of an MLP probe trained to predict patch locations given patch encodings.
Method	Patch Encoding Similarity (cosine similarity)	Patch Encoding Position Probing (cross entropy loss)
2D-ALiBi + X-ALiBi	0.546	4.43
2D-ALiBi	0.582	4.41
PEG [90]	0.701	4.13
2D-sinusoidal	0.493	0.00
2D-ALiBi learns patch representations that retain local information. [47] show that models trained with contrastive objectives can lose significant local information at deeper ViT layers that harm performance on dense prediction tasks. They show that the cosine similarity between the representations of patches at different locations within an image becomes high, indicating a patchwise representational collapse. At every ViT layer, 2D-ALiBi injects a local bias in each attention head, for each patch location. To investigate if 2D-ALiBi can successfully prevent this collapse from occurring, we measure the cosine similarity between different patch encodings of the same sample and take the average across 5,000 optical images. We find that 2D-ALiBi learns to represent patches with greater spatial diversity than PEG, and X-ALiBi further improves diversity (Table 12). Interestingly, 2D-sinusoidal learns the most diverse patch representations. We also train MLP probes on patch encodings to classify the patch location; this measures the amount of positional information represented in patch encodings. We find that the patch encodings of models that use 2D-sinusoidal embeddings fully specify the location of the patch within the image (Table 12); this is an undesirable property of patch encodings, which should represent the content of the patch, rather than its location.
A.3 Pretraining Details
A.3.1 Data
We use the SSL4EO dataset [70], which consists of Sentinel-1 & 2 imagery acquired at  $250\mathrm{K}$  locations around the world; each location (a  $2.64\mathrm{km} \times 2.64\mathrm{km}$  square) is imaged four times, spread out over a year. We use these 1 million samples of  $264 \times 264$  pixels for pretraining. Please see the SSL4EO paper [70] for more details.
A.3.2 Implementation
We use an NVIDIA DGX server (8×A100-80GB), the maximum batch size that can fit into 640 GB of VRAM (7,200 for our default ViT-B), bfloat16 precision, a base learning rate of 4e-6, warmup for 5% of the total epochs, and cooldown via a cosine decay schedule. We use the same normalization procedure as SatMAE [26]. For data augmentation, we randomly crop 60-180 pixel squares from the original  $264 \times 264$  pixels and resize the crops to  $120 \times 120$  pixels (our default image size). We also perform vertical and horizontal flipping, 90-degree rotations, and mixup=0.3. Crucially, we apply these transformations identically to both modalities; if we applied them to each modality independently, our spatial alignment would break. We use the AdamW optimizer with  $\beta_{1} = 0.9$  and  $\beta_{2} = 0.999$ , and a weight decay of 0.01.
A.4 Evaluation Details
The evaluation of foundation models for Earth Observation is less mature than in other fields. We do our best to re-use the experimental conditions of the SoTA, i.e., SatMAE [26], and improve upon them where possible. One such condition is to report results from a held-out validation set; precisely, the best validation performance measured after each finetuning epoch is reported. No test sets are used. To enable fair comparisons with prior work, we copy this approach. In trying to improve the evaluation of foundation models for Earth Observation, we detail our approach in this Appendix, share code and preprocessed datasets, re-evaluate all near-SOta models under identical
conditions, and evaluate models in more ways than prior work (i.e., linear and nonlinear probing, kNN classification, and  $K$ -means clustering).
We initialize parameters from publicly shared pretrained weights, evaluating all models ourselves under identical conditions. Although this process is laborious, we believe it significantly improves the value of our paper; several prior studies have often evaluated their models in different ways, using different data splits that cannot be directly compared. When downloading pretrained weights, we use the latest weights that are publicly available. For instance, SatMAE [26] released improved versions of their multispectral ViT-B and ViT-L models, pretrained for 200 epochs, after their manuscript was accepted for publication (edited on  $\text{arxiv}$  on January  $15^{th}$ , 2023). We exclusively evaluate these improved models throughout our paper, ensuring we compare CROMA to the best models available. For multispectral benchmarks with 13 channels (with cirrus included), we simply drop the cirrus band for models pretrained without it.
A.4.1 Data
BigEarthNet. [76] We use the same splits for training (10% of the complete training set) and evaluating (the entire validation set) as SatMAE [26] and SeCo [25]. However, we use the combined validation and test sets (236,130 samples) in our ablation studies to increase the reliability of our findings with minimal added cost. Images are  $120 \times 120$  pixels.
fMoW-Sentinel. [26] Inspired by how the BigEarthNet benchmark is used (i.e., training on  $10\%$  of the complete training set of 354,200 samples), we create a  $10\%$  split of the complete fMoW-Sentinel training set of 712,874 samples. We share the IDs of the  $10\%$  of fMoW-Sentinel training samples that we randomly selected. We believe this smaller training set should be used in future work to reduce the costs of hyper-parameter searches—a single finetuning run of SatMAE on the complete training set requires 192 hours on a V100 GPU [26]. Following SatMAE, we use the full validation set for evaluation. Images vary in size; the mean height is 45 pixels, and the mean width is 60 pixels.
In our paper, we benchmark this new split. However, we report results obtained by our CROMA models on the complete training set in Table 13. Due to the costs of finetuning on the complete training set (712,874 samples), we decide to allocate our resources elsewhere and not perform any hyper-parameter tuning. Instead, we select hyper-parameters we believe to be reasonable and finetune CROMA-B and CROMA-L once. For finetuning, we use a base learning rate of 1e-5 and all other hyper-parameters from §A.4.2.
Table 13: fMoW-Sentinel results (top 1 accuracy) using the complete training set. * denotes results reported in SatMAE (updated on arxiv on January  $15^{th}$ , 2023).
Method	Backbone	Finetuning	Linear Probing
SatMAE	ViT-B	62.65*	37.40
CROMA	ViT-B	61.00	40.94
SatMAE	ViT-L	63.84*	39.19
CROMA	ViT-L	63.59	41.96
EuroSAT. [77] We use the same training and validation sets as SatMAE. Images are  $64 \times 64$  pixels.
Canadian Cropland. [78] We are the first to benchmark this dataset of Canadian agricultural croplands, consisting of 10 classes (barley, canola, corn, mixedwood, oats, orchard, pasture, potato, soybean, and spring wheat). We select this dataset because it is a large dataset that evaluates different capabilities from the other benchmarks that typically consider croplands as a single class. Following EuroSAT [77], the authors selected an image size of  $64 \times 64$  pixels [78]; therefore, models evaluated on EuroSAT can be evaluated on Canadian Cropland with minimal modifications. We use the training set and combine their validation and test sets to form a single held-out set for evaluation. We share these complete training and validation sets. The performance (Table 1) and representation visualizations (Fig. 6 and 7 in this Appendix) indicate that the 10 classes present in this dataset are challenging to separate.
DFC2020. [87] This dataset is used for evaluation in diverse ways—both the choice of data split and image size. The original dataset comprises 6,114 samples of  $256 \times 256$  pixels. These samples are typically split into two: a so-called “validation set” of 986 samples and a so-called “test set” of 5,128 samples. Some studies use the “validation set” for training and the “test set” for validation; others use the “test set” for training and the “validation set” for validation. Some studies use the full  $256 \times 256$  pixels as inputs to their models, while others use smaller inputs. We select the split of 5,128 samples for training, which we divide into 46,152 images of  $96 \times 96$  pixels—leaving us with the split of 986 samples for validation, which we divide into 8,874 images of  $96 \times 96$  pixels. We select this final
resolution because it is the default image size of SatMAE, enabling a fair comparison to the SoTA. We share these complete training and validation sets.
DW-Expert. [88] The data collected by Dynamic World [88] is a new high-quality dataset annotated by experts with the help of auxiliary information. Thus, it should be used in the future when benchmarking models. Our work uses the expertly annotated data from Dynamic World, which we split into 20,422 train samples and 51,022 validation samples. All images are  $96 \times 96$  pixels to enable a fair comparison with SatMAE. We share these complete training and validation sets. We also create a version of this dataset that consists of  $120 \times 120$  pixel images (i.e., DW-Expert-120) that we only use for ablations because it is the default image size of CROMA.
MARIDA. [89] We use the training set and combine the validation and test sets to form a single held-out set for evaluation. Following our approach for DFC2020 and DW-Expert, we divide the original images into images of  $96 \times 96$  pixels. Because it is a sparsely labeled dataset (i.e., only a fraction of pixels per image are labeled), we include images with at least one labeled pixel. We select this dataset because it evaluates different capabilities from the other semantic segmentation benchmarks. It consists of the following classes: marine debris, dense Sargassum, sparse Sargassum, natural organic material, ship, clouds, marine water, sediment-laden water, foam, turbid water, shallow water, waves, cloud shadows, wakes, and mixed water. We share these complete training and validation sets.
A.4.2 Implementation