Package: websiteclassificationapi
Website Classification APIPython3 client library forURL Classification.Website classification API s a python library that allows to classify websites based on IAB and E-commerce taxonomies.Installationpip install websiteclassificationapiRequirementsOnly Python 3 is supported. You need an API key which you can obtain at .
Python library requires only requests package.DocumentationMore detailed API documentation on URL Classification isavailable here.Examplesfrom websiteclassificationapi import websiteclassificationapi

api_key = 'h2XurA' # you can get API key from www.websitecategorizationapi.com
url = 'www.alpha-quantum.com' # can be set to any valid URL
classifier_type = 'iab1' # should be set to either iab1 (Tier 1 categorization) or iab2 (Tier 2 categorization) for general websites or ecommerce1, ecommerce2 and ecommerce3 for E-commerce or product websites

# calling the API
print(websiteclassificationapi.get_categorization(url,api_key,classifier_type))How to select classifiers of different taxonomiesClassifier_type should be set to either iab1 (Tier 1 categorization) or iab2 (Tier 2 categorization) for general websites or ecommerce1, ecommerce2 and ecommerce3 for E-commerce or product websites.IAB Tier 1 categorization returns probabilities of text being classified as one of 29 possible categories.IAB Tier 2 categorization returns probabilities of text being classified as one of 447 possible categories.Ecommerce Tier 1 categorization returns probabilities of text being classified as one of 21 possible categories.Ecommerce Tier 2 website categorization returns probabilities of text being classified as one of 182 possible categories.Ecommerce Tier 3 website categorization returns probabilities of text being classified as one of 1113 possible categories.TaxonomiesThe list of categories available by classifier is also known as Taxonomy. There are many taxonomies available, some are standard are well known, e.g. IAB taxonomy is well suited for ads and advertising in general, whereas Facebook product categories taxonomy
is appropriate for ecommerce field.Taxonomy also differ in how many tiers, levels, or depths do they support. E.g. taxonomy may only support 1 set of main categories, or it can further subcategories.The categorization in the form of Tier 1/Tier 2/Tier 3/.... is also known as taxonomy path.The classifiers can be either built in a way that they predict single Tier categories or they can return full taxonomy paths. It really depends on the use case what is most appropriate.You can find more information about IAB taxonomy at this page:https://www.iab.com/guidelines/content-taxonomy/.Taxonomy should be chosen in a way that it suits your use case. E.g. let us say you have an online store and currently you just list your products without any categorizations.Then it may be very valuable if you could provide some kind of menus that categorize products in different verticals.Why? Because your users may more easily find your products, you will have more subpages that can be indexed by search engines and thus provide you with more traffic and visits.Having verticals set up may also mean better filtering and lead to higher conversions and thus lower cost of acquisition. There are a multitude of opportunities in adding categorization to an online store.AI explainabilityOne of the unique features of classifiers is that they provide machine learning interpretability or artificial intelligence explainability (XAI) in the form of words that most contribute to resulting classification.Example 1 of explainability:Example 2 of explainability:Why the need of AI explainability?AI models are increasingly being used in ways that affect humans. E.g. you may apply for a loan at the bank and get rejected, but even though a human may have sent or explained you this, the decision may have actually been made by a machine learning model.Machine learning models making decisions is increasingly part of every day and because often these decisions are made by what could be termed black boxes, there is increasing desire for having ML decisions made in a way that are explainable.There are also many regulations that demand this, e.g. GDPR.Support for languagesClassification service supports classifications of websites in 30+ major languages, including English, French, German, Italian, Spanish, Chinese and others.Offline database of categorized domainsWe offer offlineURL databaseof millions of categorized domains. It can be used web content filtering, AdTech marketing, cybersecurity, brand safety, contextual targeting.It is ideal for those use cases where you require very low latency of requests, which can be achieved with pre-classified websites stored in database.Handling websites with no textsWhen encountering websites that have no text and just images, our classifier relies ononline optical character recognition API serviceto extract text (if any available) from images on the website. And then classify it.To deal with potential duplicates we use thereverse IP lookup of domainsto find similar domains that are hosted on the same IP.Application of website categorization to technologies usageWe have collected usage of technologies by millions of websites, by combining this with categorization, one can find interesting results.Here is for example usage of Intercom across industry verticals:Based on 50 millions of usage points we built an AI recommender which can predict which technologies for company using a set of technologies.Here are e.g. recommendations for company using Mouse Flow:TechnologyAI Recommendation ScoreWebsiteAppNexus0.15http://appnexus.comMicrosoft Clarity0.14https://clarity.microsoft.comOsano0.14https://www.osano.com/Jetpack0.14https://jetpack.comRaphael0.14https://dmitrybaranovskiy.github.io/raphael/Svelte0.13https://svelte.devAWS Certificate Manager0.12https://aws.amazon.com/certificate-manager/Extendify0.11https://extendify.comKendo UI0.11https://www.telerik.com/kendo-uiFlywheel0.11https://getflywheel.comExample classificationsExample classification for websitewww.github.com:{
  "classification": [
    {
      "category": "Technology & Computing",
      "value": 0.7621352908406164
    },
    {
      "category": "Business and Finance",
      "value": 0.0785701408756428
    },
    {
      "category": "Video Gaming",
      "value": 0.06626958968249749
    },
    {
      "category": "Fine Art",
      "value": 0.017105357862223433
    },
    {
      "category": "Hobbies & Interests",
      "value": 0.016812511656388394
    },
    {
      "category": "Sports",
      "value": 0.011396157737341801
    },
    {
      "category": "Home & Garden",
      "value": 0.009099685741207822
    },
    {
      "category": "Personal Finance",
      "value": 0.0076400890345109055
    },
    {
      "category": "News and Politics",
      "value": 0.006692288300928684
    },
    {
      "category": "Careers",
      "value": 0.0039930258544077606
    },
    {
      "category": "Automotive",
      "value": 0.0029276292555247764
    },
    {
      "category": "Events and Attractions",
      "value": 0.0026449624402393084
    },
    {
      "category": "Shopping",
      "value": 0.0023606962223306537
    },
    {
      "category": "Family and Relationships",
      "value": 0.0023174171750800186
    },
    {
      "category": "Music and Audio",
      "value": 0.0020517145262615513
    },
    {
      "category": "Movies",
      "value": 0.0018936850100483473
    },
    {
      "category": "Travel",
      "value": 0.0009448942095545797
    },
    {
      "category": "Science",
      "value": 0.0008432696857311802
    },
    {
      "category": "Pets",
      "value": 0.0006956402098649299
    },
    {
      "category": "Television",
      "value": 0.0005261918310662409
    },
    {
      "category": "Real Estate",
      "value": 0.0005058920662560916
    },
    {
      "category": "Religion & Spirituality",
      "value": 0.000492253420442475
    },
    {
      "category": "Healthy Living",
      "value": 0.0004690261931844088
    },
    {
      "category": "Medical Health",
      "value": 0.0004467617749304944
    },
    {
      "category": "Education",
      "value": 0.00036333686743226124
    },
    {
      "category": "Food & Drink",
      "value": 0.0003463620639422737
    },
    {
      "category": "Books and Literature",
      "value": 0.00027078317064036986
    },
    {
      "category": "Style & Fashion",
      "value": 0.00011770141998920516
    },
    {
      "category": "Pop Culture",
      "value": 0.00006764487171529734
    }
  ],
  "html": "29101",
  "language": "en",
  "status": 200
}Useful resources used in development of website categorizationTensorflowWebsite categorizationSklearnPhysik nachhilfe muenchenIntroduction to URL Categorization DatabaseTop shopify stores