Package: asgi-sitemaps
asgi-sitemapsSitemapgeneration for ASGI applications. Inspired byDjango's sitemap framework.ContentsFeaturesInstallationQuickstartHow-ToSitemap sectionsDynamic generation from database queriesAdvanced web framework integrationAPI ReferenceSitemapSitemapAppFeaturesBuild and compose sitemap sections into a single dynamic ASGI endpoint.Supports drawing sitemap items from a variety of sources (static lists, (async) ORM queries, etc).Compatible with any ASGI framework.Fully type annotated.100% test coverage.InstallationInstall with pip:$pipinstall'asgi-sitemaps==1.*'asgi-sitemapsrequires Python 3.7+.QuickstartLet's build a static sitemap for a "Hello, world!" application. The sitemap will contain a single URL entry for the home/endpoint.Here is the project file structure:.└── server├── __init__.py├── app.py└── sitemap.pyFirst, declare a sitemap section by subclassingSitemap, then wrap it in aSitemapApp:# server/sitemap.pyimportasgi_sitemapsclassSitemap(asgi_sitemaps.Sitemap):defitems(self):return["/"]deflocation(self,item:str):returnitemdefchangefreq(self,item:str):return"monthly"sitemap=asgi_sitemaps.SitemapApp(Sitemap(),domain="example.io")Now, register thesitemapendpoint as a route onto your ASGI app. For example, if using Starlette:# server/app.pyfromstarlette.applicationsimportStarlettefromstarlette.responsesimportPlainTextResponsefromstarlette.routingimportRoutefrom.sitemapimportsitemapasyncdefhome(request):returnPlainTextResponse("Hello, world!")routes=[Route("/",home),Route("/sitemap.xml",sitemap),]app=Starlette(routes=routes)Serve the app using$ uvicorn server.app:app, then request the sitemap:curlhttp://localhost:8000/sitemap.xml<?xml version="1.0" encoding="utf-8"?><urlsetxmlns="http://www.sitemaps.org/schemas/sitemap/0.9"><url><loc>http://example.io/</loc><changefreq>monthly</changefreq><priority>0.5</priority></url></urlset>Tada!To learn more:SeeHow-Tofor more advanced usage, including splitting the sitemap in multiple sections, and dynamically generating entries from database queries.See theSitemapAPI referencefor all supported sitemap options.How-ToSitemap sectionsYou can combine multiple sitemap classes into a single sitemap endpoint. This is useful to split the sitemap in multiple sections that may have differentitems()and/or sitemap attributes. Such sections could be static pages, blog posts, recent articles, etc.To do so, declare multiple sitemap classes, then pass them as a list toSitemapApp:# server/sitemap.pyimportasgi_sitemapsclassStaticSitemap(asgi_sitemaps.Sitemap):...classBlogSitemap(asgi_sitemaps.Sitemap):...sitemap=asgi_sitemaps.SitemapApp([StaticSitemap(),BlogSitemap()],domain="example.io")Entries from each sitemap will be concatenated when building the finalsitemap.xml.Dynamic generation from database queriesSitemap.items()supports consuming any async iterable. This means you can easily integrate with an async database client or ORM so thatSitemap.items()fetches and returns relevant rows for generating your sitemap.Here's an example usingDatabases, assuming you have aDatabaseinstance inserver/resources.py:# server/sitemap.pyimportasgi_sitemapsfrom.resourcesimportdatabaseclassSitemap(asgi_sitemaps.Sitemap):asyncdefitems(self):query="SELECT permalink, updated_at FROM articles;"returnawaitdatabase.fetch_all(query)deflocation(self,row:dict):returnrow["permalink"]Advanced web framework integrationWhileasgi-sitemapsis framework-agnostic, you can use the.scopeattributeavailable onSitemapinstances to feed the ASGI scope into your framework-specific APIs for inspecting and manipulating request information.Here is an example withStarlettewhere we build sitemap of static pages. To decouple from the raw URL paths, pages are referred to by view name. We reverse-lookup their URLs by building aRequestinstance from the ASGI.scope, and using.url_for():# server/sitemap.pyimportasgi_sitemapsfromstarlette.datastructuresimportURLfromstarlette.requestsimportRequestclassStaticSitemap(asgi_sitemaps.Sitemap):defitems(self):return["home","about","blog:home"]deflocation(self,name:str):request=Request(scope=self.scope)url=request.url_for(name)returnURL(url).pathThe corresponding Starlette routing table could look something like this:# server/routes.pyfromstarlette.routingimportMount,Routefrom.importviewsfrom.sitemapimportsitemaproutes=[Route("/",views.home,name="home"),Route("/about",views.about,name="about"),Route("/blog/",views.blog_home,name="blog:home"),Route("/sitemap.xml",sitemap),]API ReferenceclassSitemapRepresents a source of sitemap entries.You can specify the typeTof sitemap items for extra type safety:importasgi_sitemapsclassMySitemap(asgi_sitemaps.Sitemap[str]):...asyncitemsSignature:async def () -> Union[Iterable[T], AsyncIterable[T]](Required)Return aniterableor anasynchronous iterableof items of the same type. Each item will be passed as-is to.location(),.lastmod(),.changefreq(), and.priority().Examples:# Simplest usage: return a listdefitems(self)->List[str]:return["/","/contact"]# Async operations are also supportedasyncdefitems(self)->List[dict]:query="SELECT permalink, updated_at FROM pages;"returnawaitdatabase.fetch_all(query)# Sync and async generators are also supportedasyncdefitems(self)->AsyncIterator[dict]:query="SELECT permalink, updated_at FROM pages;"asyncforrowindatabase.aiter_rows(query):yieldrowlocationSignature:def (item: T) -> str(Required)Return the absolute path of a sitemap item."Absolute path" means an URL path without a protocol or domain. For example:/blog/my-article. (Sohttps://mydomain.com/blog/my-articleis not a valid location, nor ismydomain.com/blog/my-article.)lastmodSignature:def (item: T) -> Optional[datetime.datetime](Optional)Return thedate of last modificationof a sitemap item as adatetimeobject, orNone(the default) for nolastmodfield.changefreqSignature:def (item: T) -> Optional[str](Optional)Return thechange frequencyof a sitemap item.Possible values are:None- Nochangefreqfield (the default)."always""hourly""daily""weekly""monthly""yearly""never"prioritySignature:def (item: T) -> float(Optional)Return thepriorityof a sitemap item. Must be between 0 and 1. Defaults to0.5.protocolType:str(Optional)This attribute defines the protocol used to build URLs of the sitemap.Possible values are:"auto"- The protocol with which the sitemap was requested (the default)."http""https"scopeThis property returns theASGI scopeof the current HTTP request.classSitemapAppAn ASGI application that responds to HTTP requests with thesitemap.xmlcontents of the sitemap.Parameters:(Required)sitemaps- ASitemapobject or a list ofSitemapobjects, used to generate sitemap entries.(Required)domain- The domain to use when generating sitemap URLs.Examples:sitemap=SitemapApp(Sitemap(),domain="mydomain.com")sitemap=SitemapApp([StaticSitemap(),BlogSitemap()],domain="mydomain.com")LicenseMITChangelogAll notable changes to this project will be documented in this file.The format is based onKeep a Changelog.1.0 - 2022-02-13AddedNow marked as Production/Stable software.(Pull #14)Add official support for Python 3.9 and Python 3.10. (Pull #13)0.3.2 - 2020-07-07FixedFix support for async items. (Pull #9)0.3.1 - 2020-07-05FixedFixScopetype hint: values are nowAny.0.3.0 - 2020-07-05This release changes the approach from "scrape the ASGI app to gather URLs" to a programmatic class-based API inspired by Django's sitemap framework.As such, the command line application does not exist anymore. Users are expected to defineSitemapclasses, compose them into aSitemapAppendpoint, and add that to their ASGI app routing table.See the newREADME.mddocumentation for more information.ChangedSwitch to a class-based dynamic endpoint API. (Pull #4)0.2.0 - 2020-06-01ChangedProject was renamed fromsitemapstoasgi-sitemaps- sitemap generation for ASGI apps. (Pull #2)Change options of CLI and programmatic API to fit new "ASGI-only" project scope. (Pull #2)CLI now reads from stdin (for--checkmode) and outputs sitemap to stdout. (Pull #2)RemovedDrop support for crawling arbitrary remote servers. (Pull #2)FixedDon't include non-200 or non-HTML URLs in sitemap. (Pull #2)0.1.0 - 2020-05-31AddedInitial implementation: CLI and programmatic async API.