dataesr/scientific-paragraphs-categorization huggingface dataset:
Acknowledgements We thank the  Department of Livestock Development (DLD), Bangkok, Thailand  and the  National Veterinary Research Institute (NaVRI)  for their support in this research, and the panel of experts for their input. This work was carried out within the framework of the GREASE platform in partnership for research and training in Southeast Asia (http://www. grease-network. org/). We are grateful to  Grace Delobel  for reviewing the English language of the manuscript. This research was supported by a grant from the " IDEX-nouveaux entrants" program  from the  Université Fédérale de Toulouse Midi-Pyrénées.
Initiated in 2004, the RICardo project focuses on bilateral trade data of all the world's countries from the early nineteenth century to the eve of the Second World War. The project includes the construction of a database and the creation of a website. It is a pioneering work in the field of historical trade statistics that aims at providing easy-to-access research material to the scientific community and at attracting a wider public to the history of trade relationships. The paper emphasizes the originality of our project and sums up its various aspects by reviewing previous trade databases, clarifying the main features of historical trade statistics, and describing the construction and use of the RICardo website.
RICardo (RIC for Research on International Commerce) is a database that documents bilateral trade flows the world over covering the period 1800/30 to 1938. When the idea emerged, in 2004, it was limited to the collection of data every ten years for the needs of our research team, and when the project was officially launched in 2007, it was intended to be completed within four years. More than ten years after its birth, the project has taken an enlarged dimension by providing annual series of historical trade statistics on an open-access website. 1 The aim of this paper is to present the different stages of this undertaking from the collection of data to the web visualizations, by insisting upon the difficulties we have experienced, the choices made, and the solutions that have been developed.
The original concept for the RICardo project started from the realization that all existing historical trade databases suffer from various limitations. Either they provide values for countries' total trade only, or cover the period after 1870, or are limited to a selection of countries or regions (mainly Europe and North America). Three main hurdles may have deterred scholars from building a large historical trade database: the relatively apparent low benefits of work, the unreliability of trade data, and the unavailability of sources. The first point is far from negligible as already noted by  (Lewis 1981, 33)   2 and confirmed by our experience. Secondly, the warning of  (Morgenstern 1963, 180)  concerning the reliability of trade data discourages further from embarking on such an undertaking: "Writers on all phases of foreign trade will have to assume the burden of proof that the figures on commodity movements are good enough to warrant the manipulation and the reasoning to which they are customarily subject".
This argument will be addressed more thoroughly in a later section. The last obstacle pertains to the availability of sources. It becomes more difficult to unearth bilateral trade statistics as we go back earlier in time, especially before 1870. The starting point of the RICardo project was the discovery of a significant and neglected compilation of bilateral foreign trade data, the French Extraits d'Avis Divers published between 1829 and 1839. Its exhumation prompted a search for all extant publications of commercial statistics the world over that suggested a great part of archival material had not yet been incorporated into available compilations of world trade statistics. This motivated our request for financing from the French Agence Nationale de la Recherche to build a large bilateral trade database that would allow a renewing of research on trade globalization by covering the entire period from the early trade globalization era to the eve of the Second World War.
After data collection, our work focused on the construction of a relational database structure to aggregate original data into a standardized information system. In fact, data transcribed from a great variety of sources are very heterogeneous. For two reasons: the international scope of the database includes many different currencies that have to be converted into a common value reference to allow comparison; the numerous names used to describe the trade partners necessitate some normalization work. Finally, we have created an Exploratory Data Analysis web application to help the research community use data for quantitative analysis. The paper is organized as follows. The first section compiles an inventory of existing historical trade databases to point out the innovative character of the RICardo project.
The innovative nature of the RICardo project Trade statistics of a country can be viewed in different ways, more or less disaggregated: total trade (sum of exports and/or imports), bilateral trade (distribution of total trade by partner countries), trade by products (distribution of total trade by products), bilateral trade by products (distribution of trade with each partner country by products). The RICardo database combines two sets of data: total trade data and bilateral trade data. The project started with the era of database management -with software such as Access or Filemaker. As years went by, we considered the new technological opportunities of web applications and decided to extend the project to the creation of a datascape  (Latour et al. 2012). The team was reorganized to include developers and designers, and the database was no longer seen as an end product but as an evolutionary base that will be open to new data.
After the First World War, the League of Nations centralised the compilation and publication of international statistics in its Memoranda on the balance of payments and the balance of trade. Since 1948, detailed bilateral and total trade databases are available from the International Monetary Fund's Direction of Trade Statistics and International Financial Statistics. Projects dealing with the compilation of foreign trade data of all countries of the world have thus began to mobilize the attention of statisticians about two centuries ago. It is not until after WWII that scholars attempted the task of reconstituting long-term series of foreign trade for all the world's countries. (Maddison 1962)  Maddison's bulky article deserves notice as the first attempt since World War Two to estimate the value (at current and constant prices) of the world's and the main traders' total imports and exports from 1870 to 1960.
Databases on total trade His global results are relatively close to Maddison's, but his survey is more comprehensive as the country annual series he reconstructed are included in the database, i. e. the total exports at current and constant prices for 53 countries (20 in Europe, 15 in America, 10 in Asia, 6 in Africa, 2 in Australasia) and sources are systematically indicated. For the period 1880-1913 he consulted the aforementioned US Statistical Abstract of Foreign Countries, with additions from the British Statistical Abstracts, those of the League of Nations as well as Mitchell's European Historical Statistics. The author acknowledges the scarcity of information before 1880 and mentions the secondary sources he used such as the Statesman's Yearbook as well as McCulloch's Dictionary. He explains at length the deflation procedures he adopted and the extrapolation method for bridging the gaps in the original series, as well as discusses individual country cases.
His results are compared with those published in Mulhall's Dictionary and League of Nations' Industrialisation and Foreign Trade. Figures are expressed in US dollars but the exchange rates used are not included. (Mitchell 2007)  Prominent among the data series are the chapters on foreign trade in the Historical Statistics published by B. R. Mitchell, the first edition of which goes back to 1975 and which have been updated and revised regularly. They present the most complete coverage with data on total exports and imports for a large number of countries: 125 for the pre-1939 period (24 in Europe, 31 in the Americas, 39 in Africa, 24 in Asia and 7 in Australasia). Some of the series go back to the end of the eighteenth century and include each country's bilateral trade with its major partners (5 in general).
In the introduction, the author lists the sources he used, consisting essentially in the national statistical abstracts. The main drawback of the Mitchell series 'for ready use' is the fact that they are expressed in national currencies and that the author does not provide any conversion tables. Databases on bilateral trade  (Bairoch 1973 (Bairoch , 1974) )  For the purpose of observing the changing structure of European trade with the rest of the world since the onset of industrialisation, Bairoch reconstructed series of European total and bilateral trade flows in current values over the 1800/30-1970 period. As his focus was on the geographical dispersion of European exports and imports, he aggregated country data at ten year intervals to build series by continent: Europe (Western and Continental), North America, South America, Asia, Africa, Oceania.
It is beyond doubt that Bairoch deployed a considerable effort for his investigation and it is a pity that only a fraction of statistical compilation found way in the final version of his article; it could have proved useful to other scholars. 6 (Barbieri, Keshk, Pollins 1996 -2012)  The experiment closest to the present RICardo project was undertaken in the 1990s by Barbieri. Originally it aimed at collecting and aggregating all foreign trade data from 1816, but upon the realisation that "trade data for the pre-1870 period are too scarce to make any meaningful analysis possible"  (Barbieri, Keshk, and Pollins 2008, 16-17) , she reduced the scope of her enquiry considerably. Originally, therefore, Barbieri's ambition was very close to ours. Upon closer inspection, however, it seems possible to improve substantially on the range and quality of the results presented therein.
On the crucial period 1870-1913, the information provided seems limited to the quotations from the Statesman's Yearbook; on the interwar period, to the data published by the League of Nations; and after 1945 to those of the IMF. Furthermore, data are collected for sovereign states only, excluding trade of colonies. The country nomenclature has been adapted throughout the period to match that of the Correlates of War Project. 7  The scope of the Barbieri database can be best described with some summary figures. On the period 1870-2009, the bilateral database (dyadic_trade_3. 0) includes 791 491 entries, but 90% of all observations are for the 1940-2009 period. For the period 1870-1939, of the 78 626 recorded observations, only 20 186 entries are fully documented, the rest representing reconstructed flows.
RICardo vs recent works Recent years have seen a tendency among researchers to build their own historical trade dataset for the needs of personal research. In those works, the collection of data primarily serves the purpose of answering a specific research question. (Mitchener and Weidenmier 2008)  assess the effects of empire on trade from 1870 to 1913;  (Jacks, Meissner, and Novy 2011)  gauge the importance of bilateral trade costs in determining international trade flows by selecting six years from 1870 to 2000;  (Pascali 2014 ) builds a dataset covering the period 1850-1900 to investigate the role of the adoption of the steamship in spurring trade after 1870;  (Gowa and Hicks 2013, 2015)  analyze the impact of war, institutions and politics on trade in the immediate pre-WWI and interwar period;  (Hugot 2015)  explores the chronology and geographical pattern of the two globalizations from 1827 to 2012.
The authors refer mainly to the British Statistical Abstracts, the Barbieri database, Mitchell's volumes, national statistical yearbooks, and the League of Nations yearbooks. Gowa and Hicks, Pascali, and Hugot have also relied on primary trade sources. The database of Hugot, the largest one, has 160 000 observations over  1827-1947. It should be noticed that it includes data from an earlier version of RICardo  (Hugot 2015, 42). Gowa and Hicks have gathered about 50 000 data over the 1900-1938 period. The other databases contain between 20 000 and 30 000 data. By quoting these works and previous databases, we would like to emphasize the original character of the RICardo project and the fact that it is not easily comparable with these scholar works. First, the primary aim of RICardo is to provide the researcher with a ready-to-use and comprehensive compilation of historical bilateral trade statistics so that he does not spent part of his time redoing part of the job made by others.
Second, the size of the RICardo database is not fixed. Thanks to a versioning process presented in the next section, new data will be regularly added. Third, RICardo data are accessible through a website to a wide audience. Furthermore, RICardo will be open for collaboration with other trade database projects so that it can become a reference in the field of historical bilateral trade statistics. A first example of such cooperation will be experimented with the trade dataset of  (Federico-Tena 2016). They have estimated the most exhaustive world exports and imports series since Lewis's attempt. The database they have built for the period 1800-1938 includes a total of 241 polities  (Federico-Tena 2016, 8-9)  8 and provides four types of data for each total import and export flow: at current prices and current borders; at current prices and constant borders (1913); at constant prices and current borders; at constant prices and constant borders.
Shortcomings of historical trade statistics Trade statistics are government statistics that originate from customs declarations about the quantity and/or the value or unit price of the merchandises that enter and exit the customs territory (the customs territory does not necessarily correspond to the 'national' or political territory of the entity). The customs declarations are sent to a central statistical office whose function is to evaluate, classify and publish national trade statistics  (Charles and Daudin 2015). The data contained in these publications are RICardo database's raw material. It is customary to warn prospective users of the pitfalls involved in the handling of foreign trade statistics, even more so when they go back in time, and even more if they include bilateral flows.
Hjerppe and L. -F. Andersson for Finnish and Swedish trade statistics. At the end of 2015 the total number of observations in the database amounted to 267 000 (one observation = one bilateral trade flow of exports or imports) of which about 20 000 were missing flows. An overview of the database then brought to light the most obvious flaw of our strategy. There was a significant decrease in the number of total observations over the interwar period mainly due to the use of the League of Nations sources. It proved that primary and secondary sources were not of similar quality. In fact, national trade tables generally offer a greater wealth of detail, especially with regard to bilateral flows. We therefore decided to return to our core ambition by progressively including more primary sources, either through our own efforts or through integration of external contributions.
The Annales du commerce extérieur, Commerce et navigation des principaux pays étrangers  is the last series. It adopted the format of the British statistical abstracts and offered tabulated statistical summaries relative to the various countries of the world. The data therein have been obtained from the official customs records of these countries and expressed in French francsconversion rates being given in each volume. It has been observed that these series offer a less comprehensive survey than the Statistical Abstracts which include a greater number of reporting and partner countries. British Statistical Abstracts These well-known series was the usual basis for the historical reconstruction of world trade accounts in the nineteenth century. Each volume includes total exports and imports by country for a relevant period as well as the geographical distribution of each country's foreign trade.
We have extracted from this source all available data of bilateral flows except for those countries for which we had the official publication supplying this type of information. In certain instances, however, the Abstracts have proved useful to check and, when necessary, supplement a country's total trade figures. The Statistical Abstract for the several Colonial and other Possessions of the United Kingdom in each year from… 10 contain 70 volumes published from 1865 to 1950 and covering a period from 1850 to 1947, in which 92 entities are identified. Each volume presents a varying number of summary tables of countries' imports and exports, separating movements of goods and of precious metals. Other tables present a geographical breakdown of each country's foreign trade. The Statistical Abstract for the Principal and Other Foreign Countries in each year from… contain 40 volumes published from 1874 to 1924 and covering the period 1860-1918, in which 101 state entities are identified.
As mentioned above, this was the source we tapped to document total as well as bilateral trade flows for the period spanning 1924 to 1938. However, League of Nations' trade data are incomplete. Not all the world's countries are represented (only 64 to 71 depending on the year) and the list of partner countries for each reporting country is also simplified. For example, the 1929 Memorandum of the League of Nations provides figures for 43 trading partners of the United Kingdom (89 % of total trade) whereas the Annual Statement of Trade records 165 partner entities. As a result, we decided to perfect the database by collecting new data from national trade statistics. This is a work in progress.
Why bilateral flows differ One of the main richnesses of the RICardo database is that it offers four bilateral flows for every country pair. A trade flow is recorded twice, by the two partner countries in two different sources. It is therefore possible to check on the quality of the data by comparing so-called 'mirror flows'. The term-to-term comparison of mirror flows (exports from A to B as declared by A and imports from B to A as declared by B) can reveal more or less important differences which are the result either of varying procedures for recording flows by country of origin or destination, or methods for estimating trade flows (c. i. f. vs. f. o. b. estimates, trade types), not forgetting the choice of the exchange rate. The difference between mirror trade flows is such an issue that several studies have been dedicated to its extent and ways to allay its impact.
They illustrate perfectly the different cases the scholar might be confronted with  (Allen and Ely 1953; Petruzelli 1946). The recording method is a major source of discrepancy. During the period covered by the database, the geography of bilateral trade flows may have been recorded following one of three conventions then in use. In the first, assessing the direct origin or destination of the goods consists in recording the country from which or into which these are carried directly. The second considers the country of consignment, that is, imports are customarily credited to the country from which last directly consigned and the exports to the country to which first directly consigned. Under the third method, or "production-consumption" method, imports are credited to the country where the good was produced or manufactured in its present form at the time of its entry (country of ultimate origin) and exports to the country of final destination  (Durand 1953, 117-126; Don 1968, 89; League of Nations 1928, 25, 28-29).
(Giffen 1882, 189) emphasizes the distortion introduced by the faulty registration of partner countries by customs authorities. He points out that Switzerland was not being registered as such in British trade statistics although trade relations have existed between the two countries for a long time. In our database, which reproduces the information from the British Annual Statement of Trade, Switzerland appears as a partner country only from 1906 onwards. But the Swiss Federal Statistical Office started to enter estimates of bilateral trade with Britain as early as 1885. Durand points to the case of landlocked countries such as Switzerland, Czechoslovakia, Bolivia or Rhodesia, whose external trade was carried in large part over neighbouring countries. Exporters ignorant of or indifferent to the final destination of a particular shipment may have put it on the account of an intermediate country.
The direct origin or destination method was in force in Austria-Hungary until 1891 and until 1904 in the UK. Subsequently Austria-Hungary opted for the production-consumption method while, in 1905-8, the UK adopted the practice of recording the country of consignment, which best suited the type of trade it carried out (the largest re-export trade in the world)  (Durand 1953, 121, 126). Only in 1936 did it adopt the production-consumption method. Many other cases could worsen the picture. But while these criticisms should be kept in mind, they do not undermine the validity or usefulness of a historical database on bilateral trade flows. Despite them, both the IMF and the Barbieri team have adopted the mirror flows strategy with a view not just to discriminate between sources, but in order to supplement missing information.
12 Because of its larger size, the RICardo database provides some ways of mitigating trade data's shortcomings. The approach adopted for the RICardo database has consisted in making no selection, no extrapolation, and no estimation of missing flows. First, the possibility of viewing bilateral trade exchanges over a large time-span facilitates the detection of major inconsistencies. Second, the comprehensive character of a base which offers four bilateral flows for every country pair provides the user with the opportunity to compare data and ultimately select the more reliable figures. The prospective user can then take the option of selecting the associated mirror flow when it is quoted in the base.
Cleaning of entities and currencies The RICardo database documents a period spanning the beginning of the nineteenth century to the eve of the Second World War and assembles hundreds of thousands of data collected from a variety of sources. In order to be usable, the dataset has been subjected to a standardization process regarding the appellation of the entities and currencies. 'Entities' are the names of the territorial entities recorded under different appellations in the original sources. 'Reporting entities' consist of those entities which collected foreign trade statistics while 'partner entities' are the entities mentioned as the latters' trading partners. The two lists differ substantially because reporting entities are states with customs administration which collect and publish foreign trade figures whereas partner entities may be different kinds of entities: countries, but also harbours, group of countries, or regional areas.