cometadata/datacite-titles-descriptions-related-identifiers huggingface dataset:
DOI: 10.15468/dl.am4tnk
title: Occurrence Download

description: A dataset containing 4302 species occurrences available in GBIF matching the query:
{
 "TaxonKey" : [
 "is Leucojum aestivum L."
 ]
}

 The dataset includes 4302 records from 145 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0002564-151016162008034/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.g7sjuu
title: Occurrence Download

description: A dataset containing 263 species occurrences available in GBIF matching the query:
{
 "TaxonKey" : [
 "is Babiana bainesii Baker"
 ]
}

 The dataset includes 263 records from 15 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0033503-240229165702484/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.wsufee
title: Occurrence Download

description: A dataset containing 34 species occurrences available in GBIF matching the query:
{
 "TaxonKey" : [
 "is Habenaria lankesteri Ames"
 ]
}

 The dataset includes 34 records from 6 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0036408-190918142434337/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.khuj6c
title: Occurrence Download

description: A dataset containing 4765 species occurrences available in GBIF matching the query:
{
 "or" : [
 "TaxonKey is Anthophilax LeConte, 1850",
 "TaxonKey is Anthophila Haworth, 1811"
 ]
}

 The dataset includes 4765 records from 78 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0002564-160430102343922/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.srk3a5
title: Occurrence Download

description: A dataset containing 6 species occurrences available in GBIF matching the query:
{
 "TaxonKey" : [
 "is Microichthys coccoi Rüppell, 1852"
 ]
}

 The dataset includes 6 records from 4 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0050590-231002084531237/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.g4y7ra
title: Occurrence Download

description: A dataset containing 208195 species occurrences available in GBIF matching the query:
{
 "and" : [
 "HasCoordinate is true",
 "HasGeospatialIssue is false",
 "TaxonKey is Peromyscus maniculatus (Wagner, 1845)"
 ]
}

 The dataset includes 208195 records from 125 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0334223-200613084148143/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.gydrsy
title: Occurrence Download

description: A dataset containing 38244 species occurrences available in GBIF matching the query:
{
 "and" : [
 "Country is one of (Ireland, United Kingdom of Great Britain and Northern Ireland, Portugal, Spain, France, Germany, Belgium, Netherlands, Italy, Greece, Switzerland, Austria, Türkiye, Russian Federation, Belarus, Ukraine, Norway, Sweden, Finland, Estonia, Latvia, Lithuania, Romania, Bulgaria, North Macedonia, Albania, Slovenia, Croatia, Bosnia and Herzegovina, Hungary, Slovakia, Czechia, Poland)",
 "HasCoordinate is true",
 "HasGeospatialIssue is false",
 "OccurrenceStatus is Present",
 "TaxonKey is Scilla bifolia L."
 ]
}

 The dataset includes 38244 records from 431 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0192321-240321170329656/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.yzrnxs
title: Occurrence Download

description: A dataset containing 175 species occurrences available in GBIF matching the query:
{
 "TaxonKey" : [
 "is Wallago attu (Bloch & Schneider, 1801)"
 ]
}

 The dataset includes 175 records from 29 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0002564-160509122628363/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.ts6k4s
title: Occurrence Download

description: A dataset containing 60043 species occurrences available in GBIF matching the query:
{
 "and" : [
 "OccurrenceStatus is Present",
 "TaxonKey is Zelkova Spach"
 ]
}

 The dataset includes 60043 records from 327 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0036601-231120084113126/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.25549/pcra-c14-57015
title: Redemption tidings, vol. 38, nos. 1-52, 5 Jan. - 28 Dec. 1962

description: [description english/roman)] Issued weekly. "A full gospel magazine" -- subtitle.
DOI: 10.15468/dl.34x8sc
title: Occurrence Download

description: A dataset containing 5662 species occurrences available in GBIF matching the query:
{
 "TaxonKey" : [
 "is Myiodynastes bairdii (Gambel, 1847)"
 ]
}

 The dataset includes 5662 records from 33 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0380210-210914110416597/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.qxakfq
title: Occurrence Download

description: A dataset containing 190 species occurrences available in GBIF matching the query:
{
 "TaxonKey" : [
 "is Aparatrigona impunctata (Ducke, 1916)"
 ]
}

 The dataset includes 190 records from 4 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0024050-180508205500799/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.v6pfsj
title: Occurrence Download

description: A dataset containing 1719 species occurrences available in GBIF matching the query:
{
 "and" : [
 "TaxonKey is one of (Abelmoschus moschatus (L.) Medik.)",
 "HasCoordinate is true",
 "HasGeospatialIssue is false"
 ]
}

 The dataset includes 1719 records from 133 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0080068-230530130749713/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.25549/pcra-c14-57061
title: Redemption tidings, vol. 38, nos. 1-52, 5 Jan. - 28 Dec. 1962

description: [description english/roman)] Issued weekly. "A full gospel magazine" -- subtitle.
DOI: 10.15468/dl.3886fd
title: Occurrence Download

description: A dataset containing 170608 species occurrences available in GBIF matching the query:
{
 "and" : [
 "TaxonKey is one of (Vinca minor L., Viola minor (Hayward, 1933), Viola andina W.Becker, Viola canina L.)",
 "Geometry POLYGON((-9.36035 44.71551,-10.59082 42.81152,-11.82129 36.80928,-7.69043 36.17336,-3.38379 35.88905,3.12012 38.13456,9.18457 38.20366,11.7334 37.16032,12.78809 34.74161,17.09473 33.79741, 23.15918 34.23451,30.10254 32.62087,33.2666 32.99024,35.72754 35.03,33.00293 35.53223,28.69629 35.17381,28.34473 36.31513,25.7959 37.50973,24.91699 39.90974,28.78418 40.6473,30.71777 41.17865,36.87012 45.213,38.18848 46.98025,38.71582 47.7541,39.94629 47.93107,40.20996 49.61071,37.0459 50.34546,35.72754 50.40152,33.79395 52.26816,31.68457 51.94426,31.4209 53.27835,32.56348 53.43572,30.98145 54.77535,30.89355 55.52863,28.43262 56.02295,27.20215 57.65716,27.37793 58.30949,27.90527 59.17593,29.66309 59.88894,27.6416 60.58697,31.50879 62.75473,28.78418 69.83962,30.89355 70.28912,29.22363 71.55274,21.75293 71.30079,14.28223 70.14036,11.38184 67.47492,4.87793 63.66576,-5.53711 71.52491,-10.63477 71.69129,-21.5332 67.74276,-28.56445 65.36684,-24.3457 61.35461,-9.36035 44.71551))",
 "HasCoordinate is true",
 "HasGeospatialIssue is false",
 "Year is greater than or equal to 1980",
 "Year is less than or equal to 2021"
 ]
}

 The dataset includes 170608 records from 516 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0359957-210914110416597/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.fxhyrk
title: Occurrence Download

description: A dataset containing 201 species occurrences available in GBIF matching the query:
{
 "and" : [
 "Country is Argentina",
 "TaxonKey is Myracrodruon balansae (Engl.) Santin"
 ]
}

 The dataset includes 201 records from 34 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0040508-230530130749713/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.r70t4o
title: Occurrence Download

description: A dataset listing the 1930 species recorded in GBIF matching the query:
{
 "or" : [
 "TaxonKey is Elephas maximus Linnaeus, 1758",
 "TaxonKey is Loxodonta cyclotis (Matschie, 1900)"
 ]
}

 The dataset's 1930 records were derived from 73 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0002127-190415153152247/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.25549/pcra-c14-57012
title: Redemption tidings, vol. 38, nos. 1-52, 5 Jan. - 28 Dec. 1962

description: [description english/roman)] Issued weekly. "A full gospel magazine" -- subtitle.
DOI: 10.15468/dl.ubxkyz
title: Occurrence Download

description: A dataset containing 0 species occurrences available in GBIF matching the query:
{
 "TaxonKey" : [
 "is Pilosella samokovensis (T.Georgiev & Zahn) S.Bräut. & Greuter"
 ]
}

 The dataset includes 0 records from 0 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0080110-230530130749713/datasets/export for details.
DOI: 10.15468/dl.buwkfr
title: Occurrence Download

description: A dataset containing 5637 species occurrences available in GBIF matching the query:
{
 "and" : [
 "Continent is South America",
 "Country is Ecuador",
 "TaxonKey is Lepidoptera"
 ]
}

 The dataset includes 5637 records from 25 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0380211-210914110416597/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.jpefhp
title: Occurrence Download

description: A dataset listing the 4591 species recorded in GBIF matching the query:
{
 "DatasetKey" : [
 "is WII Herbarium Dataset"
 ]
}

 The dataset's 4591 records were derived from 1 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0002127-190621201848488/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.25549/pcra-c14-56833
title: Redemption tidings, vol. 37, nos. 1-52, 1 Jan. - 22 Dec. 1961

description: [description english/roman)] Issued weekly. "A full gospel magazine" -- subtitle.
DOI: 10.5281/zenodo.10775259
title: A Comprehensive Dataset for Webpage Classification (Part 1: Adult & Malicious)

description: This dataset, split across three parts due to Zenodo's size constraints, serves as a fundamental resource for enhancing webpage classification techniques. It encompasses 1,069,715 URLs, each annotated with labels to signify their categorization into Malicious, Benign, or Adult content, and further into 20 detailed sublabels for granular analysis. The dataset is designed to facilitate the evaluation and benchmarking of machine learning models, notably Stochastic Gradient Descent (SGD) and Support Vector Classifier (SVC), across a variety of tokenization methods and input types, including URLs, raw HTML, and parsed HTML content.

The primary objective of assembling this dataset is to support research into effective webpage classification, thereby improving content prioritization and filtering in web crawling applications. It has been meticulously curated to provide a robust framework for studying the impact of different feature representation techniques on classification accuracy.

The dataset is structured as JSON lines (jsonl) files, with each entry detailing a URL's label, sublabel, source, status code, and HTML content. This comprehensive dataset is divided into three parts due to size constraints on Zenodo, each targeting specific content categories to ensure ease of use and accessibility for researchers:



Part 1: Adult & Malicious encompasses URLs classified under Adult and Malicious categories, offering insights into content that requires stringent filtering.

Part 2: Benign 1 and Part 3: Benign 2 cover benign URLs, facilitating the study of safe web content and its classification nuances.


We also created a .csv file without the HTML content so it is easier to work with URLs only, this .csv file contains the next columns `['uid', 'url', 'label', 'sublabel']`

By providing this dataset, we aim to contribute significantly to the field of webpage classification, offering a valuable asset for researchers and practitioners looking to advance the state of web crawling technology and its applications.

JSON line format for each line:

{"url": "<URL>", "label": "<Label>", "sublabel": "<Sublabel>", "source": "<Source of this URL>", "status_code": <Status Code>, "html": "<HTML Textual Content>"}Other parts of this dataset:



A Comprehensive Dataset for Webpage Classification (Part 2: Benign 1)

A Comprehensive Dataset for Webpage Classification (Part 3: Benign 2)


 

Citation

if you use this dataset, please cite us:

Al-Maamari, M., Istaiti, M., Zerhoudi, S., Dinzinger, M., Granitzer, M. and Mitrovic, J., A COMPREHENSIVE DATASET FOR WEBPAGE CLASSIFICATION.

https://ca-roll.github.io/downloads/A_Comprehensive_Dataset_for_Webpage_Classification.pdf

 

Granitzer, M., Voigt, S., Fathima, N.A., Golasowski, M., Guetl, C., Hecking, T., Hendriksen, G., Hiemstra, D., Martinovič, J., Mitrović, J. and Mlakar, I., 2023. Impact and development of an Open Web Index for open web search. Journal of the Association for Information Science and Technology.

https://doi.org/10.1002/asi.24818
DOI: 10.5281/zenodo.10775260
title: A Comprehensive Dataset for Webpage Classification (Part 1: Adult & Malicious)

description: This dataset, split across three parts due to Zenodo's size constraints, serves as a fundamental resource for enhancing webpage classification techniques. It encompasses 1,069,715 URLs, each annotated with labels to signify their categorization into Malicious, Benign, or Adult content, and further into 20 detailed sublabels for granular analysis. The dataset is designed to facilitate the evaluation and benchmarking of machine learning models, notably Stochastic Gradient Descent (SGD) and Support Vector Classifier (SVC), across a variety of tokenization methods and input types, including URLs, raw HTML, and parsed HTML content.

The primary objective of assembling this dataset is to support research into effective webpage classification, thereby improving content prioritization and filtering in web crawling applications. It has been meticulously curated to provide a robust framework for studying the impact of different feature representation techniques on classification accuracy.

The dataset is structured as JSON lines (jsonl) files, with each entry detailing a URL's label, sublabel, source, status code, and HTML content. This comprehensive dataset is divided into three parts due to size constraints on Zenodo, each targeting specific content categories to ensure ease of use and accessibility for researchers:



Part 1: Adult & Malicious encompasses URLs classified under Adult and Malicious categories, offering insights into content that requires stringent filtering.

Part 2: Benign 1 and Part 3: Benign 2 cover benign URLs, facilitating the study of safe web content and its classification nuances.


We also created a .csv file without the HTML content so it is easier to work with URLs only, this .csv file contains the next columns `['uid', 'url', 'label', 'sublabel']`

By providing this dataset, we aim to contribute significantly to the field of webpage classification, offering a valuable asset for researchers and practitioners looking to advance the state of web crawling technology and its applications.

JSON line format for each line:

{"url": "<URL>", "label": "<Label>", "sublabel": "<Sublabel>", "source": "<Source of this URL>", "status_code": <Status Code>, "html": "<HTML Textual Content>"}Other parts of this dataset:



A Comprehensive Dataset for Webpage Classification (Part 2: Benign 1)

A Comprehensive Dataset for Webpage Classification (Part 3: Benign 2)


 

Citation

if you use this dataset, please cite us:

Al-Maamari, M., Istaiti, M., Zerhoudi, S., Dinzinger, M., Granitzer, M. and Mitrovic, J., A COMPREHENSIVE DATASET FOR WEBPAGE CLASSIFICATION.

https://ca-roll.github.io/downloads/A_Comprehensive_Dataset_for_Webpage_Classification.pdf

 

Granitzer, M., Voigt, S., Fathima, N.A., Golasowski, M., Guetl, C., Hecking, T., Hendriksen, G., Hiemstra, D., Martinovič, J., Mitrović, J. and Mlakar, I., 2023. Impact and development of an Open Web Index for open web search. Journal of the Association for Information Science and Technology.

https://doi.org/10.1002/asi.24818
DOI: 10.15468/dl.2djhz4
title: Occurrence Download

description: A dataset containing 103 species occurrences available in GBIF matching the query:
{
 "and" : [
 {
 "or" : [
 "Country is Norway",
 "Country is Finland",
 "Country is Sweden"
 ]
 },
 "TaxonKey is Helophilus lapponicus Wahlberg, 1844"
 ]
}

 The dataset includes 103 records from 9 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0051021-200221144449610/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.3cruex
title: Occurrence Download

description: A dataset containing 76197 species occurrences available in GBIF matching the query:
{
 "and" : [
 "OccurrenceStatus is Present",
 "TaxonKey is Strangomera bentincki (Norman, 1936)"
 ]
}

 The dataset includes 76197 records from 14 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0097179-230530130749713/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.25549/pcra-c14-56837
title: Redemption tidings, vol. 37, nos. 1-52, 1 Jan. - 22 Dec. 1961

description: [description english/roman)] Issued weekly. "A full gospel magazine" -- subtitle.
DOI: 10.15468/dl.n76tj6
title: Occurrence Download

description: A dataset containing 72 species occurrences available in GBIF matching the query:
{
 "and" : [
 {
 "or" : [
 "BasisOfRecord is Observation",
 "BasisOfRecord is Machine Observation",
 "BasisOfRecord is Human Observation",
 "BasisOfRecord is Occurrence evidence"
 ]
 },
 "Country is China",
 "HasCoordinate is true",
 "HasGeospatialIssue is false",
 "OccurrenceStatus is Present",
 "TaxonKey is Emberiza hortulana Linnaeus, 1758",
 "Year 1972-2022"
 ]
}

 The dataset includes 72 records from 3 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0278804-210914110416597/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.thf89n
title: Occurrence Download

description: A dataset containing 0 species occurrences available in GBIF matching the query:
{
 "TaxonKey" : [
 "is Psoralea exile"
 ]
}

 The dataset includes 0 records from 0 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0097184-230530130749713/datasets/export for details.
DOI: 10.15468/dl.35f7tp
title: Occurrence Download

description: A dataset containing 260 species occurrences available in GBIF matching the query:
{
 "TaxonKey" : [
 "is Anolis petersii Bocourt, 1873"
 ]
}

 The dataset includes 260 records from 41 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0070715-200221144449610/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.vq42fe
title: Occurrence Download

description: A dataset containing 287 species occurrences available in GBIF matching the query:
{
 "TaxonKey" : [
 "is Graphium fulleri (Grose-Smith, 1883)"
 ]
}

 The dataset includes 287 records from 8 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0359973-210914110416597/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.25549/pcra-c14-57036
title: Redemption tidings, vol. 38, nos. 1-52, 5 Jan. - 28 Dec. 1962

description: [description english/roman)] Issued weekly. "A full gospel magazine" -- subtitle.
DOI: 10.15468/dl.y7q427
title: Occurrence Download

description: A dataset containing 9769 species occurrences available in GBIF matching the query:
{
 "and" : [
 "VerbatimScientificName is Phylloscopus maculipennis",
 "BasisOfRecord is one of (Machine Observation, Human Observation)",
 "Year 2011-2021",
 "HasCoordinate is true",
 "OccurrenceStatus is Present",
 {
 "not" : {
 "BasisOfRecord" : [
 "is one of (Fossil, Living Specimen, Specimen, Material citation, Material sample)"
 ]
 }
 }
 ]
}

 The dataset includes 9769 records from 6 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0080035-230530130749713/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.15468/dl.eqxc6s
title: Occurrence Download

description: A dataset containing 823 species occurrences available in GBIF matching the query:
{
 "and" : [
 "TaxonKey is Tracheophyta",
 "BasisOfRecord is one of (Specimen, Human Observation, Observation, Machine Observation)",
 "Geometry POLYGON ((144.70941 13.23500, 144.70245 13.23500, 144.69302 13.23889, 144.68054 13.24694, 144.67026 13.25694, 144.65054 13.28444, 144.64499 13.29222, 144.63416 13.35222, 144.64499 13.39889, 144.65524 13.42778, 144.70245 13.46833, 144.71912 13.47889, 144.73273 13.48555, 144.74301 13.48889, 144.76221 13.49694, 144.77859 13.50750, 144.79581 13.52417, 144.80942 13.54416, 144.81693 13.56361, 144.82858 13.61222, 144.83469 13.62611, 144.84024 13.63416, 144.85052 13.64417, 144.85886 13.64944, 144.86911 13.65278, 144.87524 13.65361, 144.87994 13.65139, 144.94775 13.59889, 144.95135 13.59555, 144.95331 13.59083, 144.95331 13.58416, 144.94662 13.57083, 144.92804 13.54222, 144.91080 13.51889, 144.90469 13.51167, 144.87051 13.47833, 144.84998 13.46500, 144.83609 13.45833, 144.80664 13.44028, 144.79248 13.42778, 144.78610 13.42028, 144.78082 13.41222, 144.77441 13.39833, 144.76831 13.37778, 144.76361 13.34222, 144.76276 13.32972, 144.76276 13.30972, 144.76080 13.29167, 144.75610 13.27639, 144.75192 13.26694, 144.74356 13.25500, 144.73691 13.24833, 144.72940 13.24222, 144.72052 13.23750, 144.70941 13.23500))",
 "HasCoordinate is true",
 "HasGeospatialIssue is false",
 "Year is greater than or equal to 1900"
 ]
}

 The dataset includes 823 records from 21 constituent datasets; see https://api.gbif.org/v1/occurrence/download/0042823-200221144449610/datasets/export for details.
 
Data from some individual datasets included in this download may be licensed under less restrictive terms.
DOI: 10.25549/pcra-c14-56323
title: Redemption tidings, vol. 37, nos. 1-52, 1 Jan. - 22 Dec. 1961

description: [description english/roman)] Issued weekly. "A full gospel magazine" -- subtitle.