composites combining processes, outputs and outcomes which make it hard to assess the effectiveness of government reforms. Fourth, many indicators which do not rely on expert judgement, instead track the existence of specific regulatory management tools and thus not adequate for assessing how each tool is used in practice and how they interact with their contexts such as changing economic circumstances.

While such measurement has been useful, it clearly has limitations in terms of accuracy and validity, calling for new metrics. The availability of large volumes of administrative data on laws and regulations open up new avenues for better gauging the determinants, extent, and effects of volatile laws and regulations. Such legislative predictability indicators should further contribute to a more encompassing understanding of legislative quality.

# **3.1 Legislative and Regulatory Data**

More and more data has been made publicly available in recent years by political institutions through dedicated websites and open-data portals. Yet, the scope and quality of legislative data that are made available still display a strong cross-country variation in terms of temporal scope and depth. Nonetheless, most countries tend to disclose the title and text of legislation, its date of publication and the ID number of previous versions of legislation. The data collection work underpinning the present analysis follows the data collection methods, database structure, and quality standards established by the research team (Fazekas et al, 2024). The wider data collection effort and process is described in Appendix B. Here, we outline the specific data collection approach in Jordan.

The data collection process consists of four stages: source identification, source annotation, web scraping and parsing, as well as data evaluation. First, source identification was conducted in consultation with Jordan’s Legislation and Opinion Bureau (LOB) which hosts the website and managed legislative data. Second, a detailed source annotation was created that precisely marks each variable in the dataset (table 1) on the LOB website. Such annotations are highly technical, offering guidance for the subsequent programming work. Third, the data collection process includes writing algorithms for scraping the LOB website and then parsing the scraped semi-structured data into standard data tables following our data standard.<sup>2</sup> Fourth, data quality, that is scope and accuracy, is evaluated both algorithmically and manually; and if errors are found then data collection and structuring are updated. Data scope is validated by comparing data record counts against record counts on the LOB website. Data accuracy is algorithmically checked by reviewing missing rates and looking at breaks in trends and distributions. Manual validation is done by comparing a small, random sample of dataset records against the original source data on the web.

# **3.2 Indicators**

Both governmental bodies and academics are beginning to develop methodologies which can measure and compare regulatory predictability across countries based on novel data sources. These studies and

> 2 See: <u>https://docs.google.com/spreadsheets/d/1cCuGZnIyAWJHD0ktk239AHTuWZLp566IItYyJFjDCE/edit?gid=0#gid=0</u>

6