# Measuring the Effect of Circular Public Procurement on Government's Environmental Impact
## 2 Materials and Methods  
The practical implementation of CPP and its impacts were explored using the method developed by [PERSON] et al. (2018) [23]. The method is characterized by starting from a macro-level viewpoint, followed by an assessment at the micro level for a sample of selected tenders and then finalized by extrapolation to obtain results on the macro level again. The eight steps within this approach are summarized in Figure 1 and further described below. Step 1 was scope definition. This approach was applied to public procurement in the years 2017-2018. Public procurement was defined as the procurement of goods and services by all governmental bodies in the Netherlands both national and subnational. This included, for example, water boards, schools and universities. Seven product groups  
Figure 1: Stepwise approach to plan and assess the application and effects of circular public procurement for a nation or region (CPP, based on [PERSON] et al. (2020) [28]). Effects at individual procurements are extrapolated to insights at the macro level for the whole nation or region. were selected for analysis: road construction, furniture, cars, ICT hardware, solar panels, buildings and work wear. This selection was based on expert judgment of a combination of criteria: (1) material intensity, (2) diversity between product groups, (3) purchasing power of the government and (4) (direct or indirect) relation to the energy transition, as judged by expert pre-scanning. The expert judgement was performed by 10 experts from 7 different institutions of policy and research as well as consultants. Step 2 involved an overview. Public authorities in the Netherlands are obliged to publish their national and European tenders on the e-procurement platform TenderNed [29]. This platform's database was used to make longlists of tenders in the years 2017-2018. Based on Common Procurement Vocabulary (CPV) codes and screening criteria (i.e., to avoid doubles or void items), a longlist of operationalized procurements per product group was derived. Details on the tenders were collated, such as the contracting authority, description of the required product or service and the expected duration of the contract. Information on the expected expenditure per tender was corrected manually to uniform budget units. In cases of missing data, the gaps were filled with estimated median expenditure values per product group. The longlist for the selection of product groups contained 998 tenders in total. Step 3 was stratified sampling. Samples were taken randomly but in the same national-to-subnational ratio as the longlist. This was performed because tenders from the national government are most often larger in expenditure than tenders from decentralized governmental bodies, and according to [PERSON] (2021) [30], they are more likely to include CPP criteria than the smaller ones, although other research shows that some circularity criteria can only be met in a small amount, such as refurbished equipment [31]. A total of 72 tenders was sampled with 10 per product group, except for the buildings product group, for which 12 samples were taken because of its large heterogeneity in CPP measures. Step 4 was text analysis. The tender texts were scored on hints towards circularity strategies using a double screening process, with reconciliation made by two experts. The experts followed a review protocol, proceeding manually through the key procurement documents and being supported by a search engine to highlight potentially relevant texts. The list of words used for the text mining can be found in Supplementary Materials S1, Table S1. Ideally, this step would be fully performed using text mining, allowing us to analyze all 998 tenders in the longlist. However, we found too many false positives and false negatives to use it for this purpose. The results of the text analyses were structured using the framework of the nine circularity strategies (the so-called R-strategies) as defined by [PERSON] et al. (2016) [32] (Supplementary Materials S1, Table S2).