# Measuring the Effect of Circular Public Procurement on Government's Environmental Impact
## 4 Discussion and Conclusions
This conclusion on the necessity of whole organization implementation aligns with studies from different geographical regions in both developed and developing countries. These studies point toward the importance of top management support [18; 45] in the context of the United States and China, leadership in sustainability [19] in the context of the United Kingdom and environmental commitment \"urging environmental awareness in every area of the business\" in the context of Germany [17]. Various authors showed that the role of change agents seems to be crucial to realize this holistic implementation, such as [PERSON] (2015) [46] in the context of the Netherlands. The results of this study also show that the percentage of tenders with CPP- or SPP-related criteria is not necessarily a good indicator for the national implementation of SPP and CPP; that is, successful implementation depends not only on the use of CPP or SPP criteria but also on their ambition and their implementation. The indicators proposed in this study provide a more detailed estimation of implementation success and provide insights that can help to design measures for improvement in CPP practices. However, for the latter part especially, the actual estimation of effect was shown to be challenging due to the lack of data on what had been supplied or lack of insight in the current reference situation. Hence, the approach points to important data requirements needed for effect-based policy making. In situations that lacked data for quantification of the effect, the qualitative assessment (if the effect was expected or not) proved valuable as well. Although the results do not answer the question of to which extent the policy instrument CPP contributes to reaching policy goals, it does provide insight on the level of implementation success and insights into the reasons for (a lack of) effectiveness. The applied approach proved to be versatile (i.e., applicable in the different contexts of the different product groups). It was developed for the situation in the Netherlands, where there is an overview on tenders thanks to e-procurement, but no central administration exists for contracts and what products have actually been supplied. E-procurement covers only the procurement via tenders. Smaller purchases are thus a blind spot in e-procurement-based approaches, such as those applied in this study. On the other hand, [PERSON] (2021) [30] showed that the application of CPP is more likely to occur in larger than in smaller tenders. The sample-taking phase is essential in situations such as the one in the Netherlands, were there is no central administration on what has been contracted and supplied. The sample size was determined by the budget of the research. More statistically sound samples can be taken when there is more information on the variability in CPP application and its potential effects. To make this possible, automatic search engines (that also cover the tender annexes) are being explored, but until now, too many false negatives and positives appeared for this strategy to be useful for tender selection. The product groups were chosen based on, among other criteria, material intensity and thus their expected relevance for closing material loops. The selection of product groups, and hence the results of this study, are not meant to be representative for all procurement of the Dutch government. One of the criteria for selection was relation to the energy transition. This is of specific importance because the energy transition requires technology that can have trade-offs between GHG emission reductions and increased use of materials such as, among others, rare earth metals [47]. Here, the energy and circularity transition should go hand in hand, and thus co-designed CPP criteria should be applied in the procurement processes of these technologies. The results of the CPP effect assessment depend largely on the definition of the market standard. Market standards change over time (Figure 2).