Package: doc-warden
Doc WardenEvery CI build owned by the Azure-SDK team also needs to verify that the documentation within the target repo meets a set of standards.Doc-wardenis intended to ease theimplementationof these checks in CI builds.Features:Enforces Readme StandardsReadmes presentReadmes have appropriate contentEnforces Changelog StandardsChangelogs PresentChangelogs contain entry and content for the latest package versionGenerates report for included observed packagesThis package is tested on Python 3.4 -> 3.8. This package went python3-only starting with0.7.0.PrerequisitesThis package is intended to be run as part of a pipeline within Azure DevOps. As such,Pythonmust be installed prior to attempting to install or useDoc-Warden.Whilepipcomes pre-installed on most modern Python installs, ifpipis an unrecognized command when attempting to installwarden, run the following commandafteryour Python installation is complete.In addition,wardenis distributed usingsetuptoolsandwheel, so those packages should also be present prior to install./:> python -m ensurepip
/:> pip install setuptools wheelUsageRight now,wardensupports two main purposes. 1. Readme and Changelog enforcement (scan,content,presence), and 2. package indexing (index).Example usage (for any of the above commands):<pre-step, clone target repository>
...
/:> pip install setuptools wheel
/:> pip install doc-warden
...
<next task, because PATH doesn't update without another one>
/:> ward scan -d $(Build.SourcesDirectory)Notes for example aboveAssumption is that the.docsettingsfile is placed at the root of the repository.To provide a different path (likeazure-sdk-for-javadoes...), use:/:> ward scan -d $(Build.SourcesDirectory) -c $(Build.SourcesDirectory)/eng/.docsettings.ymlParameter OptionscommandCurrently supports 3 commands. Values:['scan', 'presence', 'content',index]Required.scanRun bothcontentandpresenceenforcement on the targeted directory.contentRun onlycontentenforcement on the target directory. Ensures that:The content in each readme matches the regex patterns defined in the .docsettings fileEach changelog contains entry for the latest version.presenceRun onlypresenceenforcement on the target directory. Ensures readmes and changelogs exist where they should.indexTake inventory of the target folder. Attempts to leverage selected docsettings to discover all packages within the directory, and generate apackages.mdindex file.--scan-directoryThe target directorywardenshould be scanning.Required.--repo-rootThe root of the repo. Entries in the config-file should be relative to this directory.Optional.--scan-languagewardenchecks for packages byconvention, so it needs to understand what language it is looking at. This must be populated either in.docsettings fileor by parameter.Optional.--config-locationBy default,wardenlooks for the.docsettingsfile in the root of the repository. However, populating this location will override this behavior and instead pull the file from the location in this parameter.Optional.--pipeline-stageThe stage of the pipeline. can bepr,ci, orrelease.Optional.--targetSpecify what file to run enforcement onreadmeorchangelog. Used when runningcontentorpresenceverification only.Optional.--package-outputOverride the default location that the generatedpackages.mdfile is dropped to during execution of theindexcommand.--verbose-outputEnable or disable output of an html report. Defaults to false.Optional.Notes for Devops UsageThe-dargument should be$(Build.SourcesDirectory). This will pointwardenat the repo that has been associated with CI.MethodologyEnforcing Readme PresenceWhen should we expect a readme and/or changelog to be present?Always:At the root of the repo (Readme only)Associated with apackagedirectory (Readme and Changelog).NetA package directory is indicated by:a*,csprojfile under thesdkdirectoryNote that this is just a proxy.wardenattempts to omit test projects by convention.PythonA package directory is indicated by:the presence of asetup.pyfileJavaA package directory is indicated by:the presence of apom.xmlfileThe POM<packaging>value within is set toJARNode & JSA package directory is indicated by:The presence of apackage.jsonfileEnforcing Readme Contentdoc-wardenhas the ability to check discovered readme files to ensure that a set of configured sections is present. How does it work?doc-wardenwill ensure that each regex defined inrequired_readme_sectionsmatches against at least one section header in the readme. If all the patterns match at least one header, the readme will pass content verification.Other Notes:Asectionheader is any markdown or RST that will result in a<h1>to<h2>html tag.wardenwill content verify anyreadme.rstorreadme.mdfile found outside theomitted_pathsin the targeted repo.Enforcing Changelog Contentdoc-wardenchecks the latest entry in the changelog file to make sure it matches the latest version of the package. It also checks to make sure that the entry is not empty.Control, the.docsettings.ymlFile, and YouSpecial cases often need to be configured. It seems logical that there needs be a central location (per repo) to override conventional settings. To that end, a new.docsettings.ymlfile will be added to each repo.<repo-root>
│   README.md
│   .docsettings.yml
│
└───.azure-pipelines
│   │   <build def>
│   
└───<other files and folders>The presence of this file allows each repository to customize how enforcement takes place within their repo.Example DocSettings File for Java Repoomitted_paths:
  - archive/*
  - sdk/eventhub/
language: java
root_check_enabled: True
required_readme_sections:
  - "(Client Library for Azure .*|Microsoft Azure SDK for .*)"
  - Getting Started
    - Install Package
    - Prerequisites
    - Authenticate the Client
known_presence_issues:
  - ['cognitiveservices/data-plane/language/bingspellcheck/README.md', '#2847']
  - ['cognitiveservices/data-plane/language/bingspellcheck/CHANGELOG.md', '#2847']
known_content_issues:
  - ['sdk/template/azure-sdk-template/README.md','#1368']
  - ['sdk/template/azure-sdk-template/CHANGELOG.md','#1368']The above configuration tellswarden...The language within the repo isjavaTo ensure that aREADME.mdis present at the root of the repository.To omitanypaths underarchive/from the readme checks.To omit paths founddirectlyundersdk/eventhub/.This means that if there is a readme content issue undersdk/eventhub/azure-messaging/, it will still throw an error!Possible values forlanguageright now are['net', 'java', 'js', 'python']. Greater than one target language is not currently supported.required_readme_sectionsConfigurationThis section instructswardento verify that there is at least one matching section title for each providedsectionpattern in any discovered readme. Notice thatnestedspecifications are supported. Regex is fully supported.The two items listed from the example.docsettingsfile will:Match a header matched by a simple regex expressionMatch a header exactly titled "Getting Started"
- Under the header "Getting Started" validate that 3 additional headings are present.
-doc-wardenwill search up to the next header of equivalent importantance for the sub-headings.
- This means that when searching under header# Getting Started, doc-warden will scan up to the nextH1header.Note that the regex is surrounded by quotation marks where the regex will breakymlparsing of the configuration file.known_presence_issuesandknown_content_issuesConfigurationdoc-wardenis designed to crash builds if it detects failures. However, the vast majority of the time, these issues cannot be fixed immediately. In the above configuration, there are two paths highlighted as known issues.The first,known_presence_issues, tells warden that a presence failure detected at the specified pathsshould be ignoredand should not result in a crashed build. Atupledescribing each known issue specifies both what the known issue is, as well as some sort of justification. Having an exception with an issueId attached is a good justification for not failing the build.We're aware of this issue, and it is tracked in the following github issue.Theknown_content_issuesparameter functionsidenticallyto theknown_presence_issuescheck. If a readme is listed as "already known" to have failures, the entire CI build will not be crashed by Warden.package_indexing_exclusion_listandpackage_indexing_traversal_stopsConfigurationIndexing packages is often done as part of nightly (or triggered) automation. With this being the case, sometimeswardenmay detect a PackageId that users wish to omit from the generatedpackages.mdfile. The Azure SDK team leverages
thepackage_indexing_exclusion_listarray members to enable just this sort of scenario.package_indexing_traversal_stopsis used during parse of .NET language reposonly. This is due to how the discovery logic for readme and changelog is implemented for .NET projects. Specifically, readmes for a .csproj are often a couple directories up from their parent .csproj location!For .net,wardenwill traverseupone directory at a time, looking for the readme and changelog files in each traversed directory.wardenwill continue to traverse until...It discovers a folder with a.slnwithin itIt encounters a folder that exactly matches one present inpackage_indexing_traversal_stopsNote thatwardenwill not even execute an index against a .NET repounless the traversal stops are set.SDK for net .docsettingsis a great example for both the exclusion list as well as the traversal stops.Provide FeedbackIf you encounter any bugs or have suggestions, please file an issuehereand assign toscbedd.