DOI: 10.5061/dryad.ffbg79cvp
title: Repository Analytics and Metrics Portal (RAMP) 2018 data

description: The Repository Analytics and Metrics Portal (RAMP) is a web service that
 aggregates use and performance use data of institutional repositories. The
 data are a subset of data from RAMP, the Repository Analytics and Metrics
 Portal (http://rampanalytics.org), consisting of data from all
 participating repositories for the calendar year 2018. For a description
 of the data collection, processing, and output methods, please see the
 "methods" section below. Note that the RAMP data model changed
 in August, 2018 and two sets of documentation are provided to describe
 data collection and processing before and after the change.
description: <strong>RAMP Data
 Documentation – January 1, 2017 through August 18,
 2018</strong> <strong>Data
 Collection</strong> RAMP data were
 downloaded for participating IR from Google Search Console (GSC) via the
 Search Console API. The data consist of aggregated information about IR
 pages which appeared in search result pages (SERP) within Google
 properties (including web search and Google Scholar). Data from January
 1, 2017 through August 18, 2018 were downloaded in one dataset per
 participating IR. The following fields were downloaded for each URL, with
 one row per URL: <b>url</b>: This is returned as a 'page' by the GSC API, and is the URL of the page which was included in an SERP for a Google property. <b>impressions</b>: The number of times the URL appears within the SERP. <b>clicks</b>: The number of clicks on a URL which took users to a page outside of the SERP. <b>clickThrough</b>: Calculated as the number of clicks divided by the number of impressions. <b>position</b>: The position of the URL within the SERP. <b>country</b>: The country from which the corresponding search originated. <b>device</b>: The device used for the search. <b>date</b>: The date of the search. Following data processing describe below, on ingest into RAMP an additional field, <b>citableContent</b>, is added to the page level data. Note that no personally identifiable information is downloaded by RAMP. Google does not make such information available. More information about click-through rates, impressions, and position is available from Google's Search Console API documentation: https://developers.google.com/webmaster-tools/search-console-api-original/v3/searchanalytics/query and https://support.google.com/webmasters/answer/7042828?hl=en <strong>Data Processing</strong> Upon download from GSC, data are processed to identify URLs that point to citable content. <i>Citable content</i> is defined within RAMP as any URL which points to any type of non-HTML content file (PDF, CSV, etc.). As part of the daily download of statistics from Google Search Console (GSC), URLs are analyzed to determine whether they point to HTML pages or actual content files. URLs that point to content files are flagged as "citable content." In addition to the fields downloaded from GSC described above, following this brief analysis one more field, <b>citableContent</b>, is added to the data which records whether each URL in the GSC data points to citable content. Possible values for the <b>citableContent</b> field are "Yes" and "No." Processed data are then saved in a series of Elasticsearch indices. From January 1, 2017, through August 18, 2018, RAMP stored data in one index per participating IR. <strong>About Citable Content Downloads</strong> Data visualizations and aggregations in RAMP dashboards present information about citable content downloads, or CCD. As a measure of use of institutional repository content, CCD represent click activity on IR content that <i>may</i> correspond to research use. CCD information is summary data calculated on the fly within the RAMP web application. As noted above, data provided by GSC include whether and how many times a URL was clicked by users. Within RAMP, a "click" is counted as a potential download, so a CCD is calculated as the sum of clicks on pages/URLs that are determined to point to citable content (as defined above). For any specified date range, the steps to calculate CCD are: Filter data to only include rows where "citableContent" is set to "Yes." Sum the value of the "clicks" field on these rows. <strong>Output to CSV</strong> Published RAMP data are exported from the production Elasticsearch instance and converted to CSV format. The CSV data consist of one "row" for each page or URL from a specific IR which appeared in search result pages (SERP) within Google properties as described above. The data in these CSV files include the following fields: <b>url</b>: This is returned as a 'page' by the GSC API, and is the URL of the page which was included in an SERP for a Google property. <b>impressions</b>: The number of times the URL appears within the SERP. <b>clicks</b>: The number of clicks on a URL which took users to a page outside of the SERP. <b>clickThrough</b>: Calculated as the number of clicks divided by the number of impressions. <b>position</b>: The position of the URL within the SERP. <b>country</b>: The country from which the corresponding search originated. <b>device</b>: The device used for the search. <b>date</b>: The date of the search. <b>citableContent</b>: Whether or not the URL points to a content file (ending with pdf, csv, etc.) rather than HTML wrapper pages. Possible values are Yes or No. <b>index</b>: The Elasticsearch index corresponding to page click data for a single IR. <b>repository_id: </b>This is a human readable alias for the index and identifies the participating repository corresponding to each row. As RAMP has undergone platform and version migrations over time, index names as defined for the <b>index</b> field have not remained consistent. That is, a single participating repository may have multiple corresponding Elasticsearch index names over time. The <b>repository_id</b> is a canonical identifier that has been added to the data to provide an identifier that can be used to reference a single participating repository across all datasets. Filtering and aggregation for individual repositories or groups of repositories should be done using this field. Filenames for files containing these data follow the format <i>2018-01_RAMP_all.csv</i>. Using this example, the file <i>2018-01_RAMP_all.csv </i>contains all data for all RAMP participating IR for the month of January, 2018. <strong>Data Collection from August 19, 2018 Onward</strong> RAMP data are downloaded for participating IR from Google Search Console (GSC) via the Search Console API. The data consist of aggregated information about IR pages which appeared in search result pages (SERP) within Google properties (including web search and Google Scholar). Data are downloaded in two sets per participating IR. The first set includes page level statistics about URLs pointing to IR pages and content files. The following fields are downloaded for each URL, with one row per URL: <b>url</b>: This is returned as a 'page' by the GSC API, and is the URL of the page which was included in an SERP for a Google property. <b>impressions</b>: The number of times the URL appears within the SERP. <b>clicks</b>: The number of clicks on a URL which took users to a page outside of the SERP. <b>clickThrough</b>: Calculated as the number of clicks divided by the number of impressions. <b>position</b>: The position of the URL within the SERP. <b>date</b>: The date of the search. Following data processing describe below, on ingest into RAMP a additional field, <b>citableContent</b>, is added to the page level data. The second set includes similar information, but instead of being aggregated at the page level, the data are grouped based on the country from which the user submitted the corresponding search, and the type of device used. The following fields are downloaded for combination of country and device, with one row per country/device combination: <b>country</b>: The country from which the corresponding search originated. <b>device</b>: The device used for the search. <b>impressions</b>: The number of times the URL appears within the SERP. <b>clicks</b>: The number of clicks on a URL which took users to a page outside of the SERP. <b>clickThrough</b>: Calculated as the number of clicks divided by the number of impressions. <b>position</b>: The position of the URL within the SERP. <b>date</b>: The date of the search. Note that no personally identifiable information is downloaded by RAMP. Google does not make such information available. More information about click-through rates, impressions, and position is available from Google's Search Console API documentation: https://developers.google.com/webmaster-tools/search-console-api-original/v3/searchanalytics/query and https://support.google.com/webmasters/answer/7042828?hl=en <strong>Data Processing</strong> Upon download from GSC, the page level data described above are processed to identify URLs that point to citable content. <i>Citable content</i> is defined within RAMP as any URL which points to any type of non-HTML content file (PDF, CSV, etc.). As part of the daily download of page level statistics from Google Search Console (GSC), URLs are analyzed to determine whether they point to HTML pages or actual content files. URLs that point to content files are flagged as "citable content." In addition to the fields downloaded from GSC described above, following this brief analysis one more field, <b>citableContent</b>, is added to the page level data which records whether each page/URL in the GSC data points to citable content. Possible values for the <b>citableContent</b> field are "Yes" and "No." The data aggregated by the search country of origin and device type do not include URLs. No additional processing is done on these data. Harvested data are passed directly into Elasticsearch. Processed data are then saved in a series of Elasticsearch indices. Currently, RAMP stores data in two indices per participating IR. One index includes the page level data, the second index includes the country of origin and device type data. <strong>About Citable Content Downloads</strong> Data visualizations and aggregations in RAMP dashboards present information about citable content downloads, or CCD. As a measure of use of institutional repository content, CCD represent click activity on IR content that <i>may</i> correspond to research use. CCD information is summary data calculated on the fly within the RAMP web application. As noted above, data provided by GSC include whether and how many times a URL was clicked by users. Within RAMP, a "click" is counted as a potential download, so a CCD is calculated as the sum of clicks on pages/URLs that are determined to point to citable content (as defined above). For any specified date range, the steps to calculate CCD are: Filter data to only include rows where "citableContent" is set to "Yes." Sum the value of the "clicks" field on these rows. <strong>Output to CSV</strong> Published RAMP data are exported from the production Elasticsearch instance and converted to CSV format. The CSV data consist of one "row" for each page or URL from a specific IR which appeared in search result pages (SERP) within Google properties as described above. Also as noted above, daily data are downloaded for each IR in two sets which cannot be combined. One dataset includes the URLs of items that appear in SERP. The second dataset is aggregated by combination of the country from which a search was conducted and the device used. As a result, two CSV datasets are provided for each month of published data: <b>page-clicks</b>: The data in these CSV files correspond to the page-level data, and include the following fields: <b>url</b>: This is returned as a 'page' by the GSC API, and is the URL of the page which was included in an SERP for a Google property. <b>impressions</b>: The number of times the URL appears within the SERP. <b>clicks</b>: The number of clicks on a URL which took users to a page outside of the SERP. <b>clickThrough</b>: Calculated as the number of clicks divided by the number of impressions. <b>position</b>: The position of the URL within the SERP. <b>date</b>: The date of the search. <b>citableContent</b>: Whether or not the URL points to a content file (ending with pdf, csv, etc.) rather than HTML wrapper pages. Possible values are Yes or No. <b>index</b>: The Elasticsearch index corresponding to page click data for a single IR. <b>repository_id: </b>This is a human readable alias for the index and identifies the participating repository corresponding to each row. As RAMP has undergone platform and version migrations over time, index names as defined for the previous field have not remained consistent. That is, a single participating repository may have multiple corresponding Elasticsearch index names over time. The <b>repository_id</b> is a canonical identifier that has been added to the data to provide an identifier that can be used to reference a single participating repository across all datasets. Filtering and aggregation for individual repositories or groups of repositories should be done using this field. Filenames for files containing these data end with “page-clicks”. For example, the file named <i>2018-09_RAMP_all_page-clicks.csv</i> contains page level click data for all RAMP participating IR for the month of September, 2018. <b>country-device-info</b>: The data in these CSV files correspond to the data aggregated by country from which a search was conducted and the device used. These include the following fields: <b>country</b>: The country from which the corresponding search originated. <b>device</b>: The device used for the search. <b>impressions</b>: The number of times the URL appears within the SERP. <b>clicks</b>: The number of clicks on a URL which took users to a page outside of the SERP. <b>clickThrough</b>: Calculated as the number of clicks divided by the number of impressions. <b>position</b>: The position of the URL within the SERP. <b>date</b>: The date of the search. <b>index</b>: The Elasticsearch index corresponding to country and device access information data for a single IR. <b>repository_id: </b>This is a human readable alias for the index and identifies the participating repository corresponding to each row. As RAMP has undergone platform and version migrations over time, index names as defined for the previous field have not remained consistent. That is, a single participating repository may have multiple corresponding Elasticsearch index names over time. The <b>repository_id</b> is a canonical identifier that has been added to the data to provide an identifier that can be used to reference a single participating repository across all datasets. Filtering and aggregation for individual repositories or groups of repositories should be done using this field. Filenames for files containing these data end with “country-device-info”. For example, the file named <i>2019-09_RAMP_all_country-device-info.csv</i> contains country and device data for all participating IR for the month of September, 2018. <strong>References</strong> Google, Inc. (2021). Search Console APIs. Retrieved from https://developers.google.com/webmaster-tools/search-console-api-original. 
description: Data have been
 compressed into zip files to reduce the filesize. Each zip file contains
 one CSV file. The RAMP team values and appreciates the
 collaboration and contributions of RAMP participating IR. For a complete
 list of participating IR, please see the RAMP website at
 https://rampanalytics.org/ramp-repositories/. We also acknowledge Google,
 whose APIs are used to collect data. RAMP data are
 audited for completeness and accuracy prior to publication. However,
 access to GSC data for any single repository may be impacted by repository
 migrations, configuration changes, rate limits, or other causes. Such
 issues may be temporary or occur over longer time periods, and the
 completeness and accuracy of the published data for individual
 repositories can therefore be variable. When such issues are known, every
 reasonable attempt is made to correct or retrieve missing or dropped
 data. The variability that may exist within the dataset
 with regard to its completeness for individual repositories should be kep
 in mind when comparing data from two or more repositories. In particular, RAMP administrators strongly
 discourage qualitative assessments of any and all repositories,
 institutions, providers, and platforms represented within the
 dataset.  An additional file is included
 with the dataset, "RAMP_repository_info.csv". This file includes
 information about the participating repositories whose data are included
 in the dataset, and may be helpful in filtering or subsetting the data.
 The file includes information about repository platforms and the country
 where each repository is located that is not included in the RAMP data and
 may be also be useful for subsetting data by platform or country. The
 fields in this CSV file are: 
 <strong>repository_id: </strong>This is the same as the
 <strong>repository_id </strong>field described in the RAMP
 data documentation above. This shared ID can be used to merge RAMP data
 with the platform and country information available in this
 file. <strong>country: </strong>The two
 letter ISO code for the country in which the repository is
 located. <strong>ir_platform:</strong>
 The software platform used by the repository. Note that some RAMP
 repositories have migrated platforms over time and may as a result be
 represented multiple times in this file. All of the major IR platforms are
 represented, as well as custom built applications. 
 <strong>name: </strong>The name of the repository. Typically,
 this includes the name of the host organization and the name of the
 repository Please note that
 the "RAMP_repository_info.csv" file is generated from current
 RAMP configuration data, and may include information about repositories
 whose data are not in the dataset if they joined RAMP after
 2018.