Package: saml2-mdq
Django SAML MDQA lightweight SAML2 MDQ server that implementsdraft-young-md-query implementation:Runs on top of the metadata downloaded and validated by pyFF (batch pipeline)Have signing features (on top of xmlsec)Have ValidUntil definitions featureSupports the following Entities Identifiers: urlencoded, {sha1} and {base64}Remember that pyFF is needed for metadata downloading, it can run as daemon or as a scheduled process (batch)Table of contentsInstallationRequirementsConfigure pyFFTest the pipelineConfigure Django MDQClient configurationsShibboleth IdP ConfigurationPySAML2PerformancesPerformancesAuthorsCreditsRequirementsapt install build-essential xmlsec1 python3-dev libxml2-dev libxslt1-dev libyaml-dev python3-pip
pip3 install --upgrade pip
pip3 install virtualenv django lxml xmlsecInstallpyFFvirtualenv -p python3 python-pyff
source python-pyff/bin/activate
pip3 install git+https://github.com/IdentityPython/pyFF.gitConfigure pyFF# Create a folder for the configuration
mkdir pyff-configuration
cd pyff-configuration

# create folder for the certificates
mkdir certificates

# create certificates
openssl req -nodes -new -x509 -days 3650 -keyout certificates/private.key -out certificates/public.cert -subj '/CN=your.own.fqdn.com'

# create a pipeline directory
mkdir pipelinesCreate a pipelines to fetch and handle all the Idem + eduGAIN metadata, this would be similar to the following.
Name itpipelines/garr_batch.fd:# Metadata download and validation
- load xrd garr-loaded.xrd:
  - ./pipelines/garr.xrd
# select can even filter entity by IDPSSO or SPSSO Description and things ...
# - select: "!//md:EntityDescriptor[md:SPSSODescriptor]"
- select
- store:
     directory: ./garr
- publish:
     output: ./garr/garr-loaded.xml
- statsNow create the XRD file where to configure the URLs where the Metadata can be downloaded.
Name itpipelines/garr.xrd<?xml version="1.0" encoding="UTF-8"?>
<XRDS xmlns="http://docs.oasis-open.org/ns/xri/xrd-1.0">
  <XRD>
    <Link rel="urn:oasis:names:tc:SAML:2.0:metadata" href="http://md.idem.garr.it/metadata/idem-test-metadata-sha256.xml"/>
  </XRD>
  <XRD>
    <Subject>http://md.idem.garr.it/metadata/edugain2idem-metadata-sha256.xml</Subject>
    <Link rel="urn:oasis:names:tc:SAML:2.0:metadata" href="http://md.idem.garr.it/metadata/edugain2idem-metadata-sha256.xml">
        <Title>IDEM+eduGAIN</Title>
        <ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
            <ds:X509Data>
                <ds:X509Certificate>
                MIIDWzCCAkOgAwIBAgIJALo/EGIq8rgNMA0GCSqGSIb3DQEBCwUAMEQxCzAJBgNV
                BAYTAklUMRYwFAYDVQQKDA1JREVNIEdBUlIgQUFJMR0wGwYDVQQDDBRJREVNIE1l
                dGFkYXRhIFNpZ25lcjAeFw0xOTAxMjIxNjA5MjBaFw0yMjAxMjExNjA5MjBaMEQx
                CzAJBgNVBAYTAklUMRYwFAYDVQQKDA1JREVNIEdBUlIgQUFJMR0wGwYDVQQDDBRJ
                REVNIE1ldGFkYXRhIFNpZ25lcjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
                ggEBAMay3N21fswu3AE6hqCPUVjvCyol5OKTHs9CXDIFyAoigP+YSdloLSGwx6n6
                ks9aBbJqlzRBIEd3CpByvX7GmBuITl3ElhxMY40Cv/ULok1GbDmQMhPScU6J1f9b
                526R9Ks+BbYZYmBRX9gqmpX1R867IES4z+JhXnXr5K8HTPjfaDGh2xORL6msXjww
                DJgaJCOpBCctLvCWcmUp0ucpl8VHGjFAAI5Eb6pwQEEPj1yqW52ggM+AHNFY6bAC
                9RX7Qv8MonQZwXpNNBNL+UcnGLVBXtBftd4zq7XxPNN9F/Ele3YJGaOVk8cCEJt5
                SfTeguzUaAyh8f/BfEs6CwucCSsCAwEAAaNQME4wHQYDVR0OBBYEFCZQVW7g6mc9
                3zaJP/p0lGbVQ4O6MB8GA1UdIwQYMBaAFCZQVW7g6mc93zaJP/p0lGbVQ4O6MAwG
                A1UdEwQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAF6OKKdWyeI385ZS5i29mSMA
                4BoPCVAhyXDMLMdqTQqvZp3PAL/zjLYRYKgGH53d4uN/EztWM8YBdyzBzdbpFWpd
                wRGzwyfXzt6l2luElWb59PacNqHbBkyFO2YZmgqLzgrVX1gA3/3ij9zrLqd1lHVH
                MHPUpqv98KYXnttyzhacdYaRGDO/2A28U9QeRq2/HgVScklhJvoySeNyXNspYfte
                ePRxeHBj21DgiQb+X1+ovKASM+RULa6cA1TJBCop+VqZMZiRJ3Rj6RML63ckEO8H
                Md/XFvxlr+P2JcVKzHaZEEUGGINUCCuDABqKBZOqykGWXDastVw6/I0OIdLmWNI=
                </ds:X509Certificate>
            </ds:X509Data>
      </ds:KeyInfo>
    </Link>
  </XRD>
</XRDS>Test the pipelinepyff pipelines/garr.fdYou should have an output of this kindtotal size:     6003
selected:       6003
          idps: 3257
           sps: 2744Configure Django MDQIf you need to integratesaml2_mdqin a preexisting django project you can install it as an app:pip install saml2_mdqThen addsaml2_mdqinto yoursettings.INSTALLED_APPS, it doesn't need migrations.
Add alsodjango.middleware.http.ConditionalGetMiddlewareinsettings.MIDDLEWAREto enableETagin the HttpResponse headers.If you instead just need a fully working MDQ server you can copy the entire project this waygit clone https://github.com/UniversitaDellaCalabria/Django-MDQ.git
cd Django-MDQ
pip install -r requirementsThenCopydjango_mdq/settingslocal.py.exampletodjango_mdq/settingslocal.pyand edit itindjango_mdq/settingslocal.pyconfigure:PYFF_METADATA_FOLDERmust point to the folder where the pyFF downloads periodically the metadata xml filesPYFF_METADATA_LOADEDmust point to the full metadata xml published by pyff, containing all the entitiesMETADATA_SIGNER_KEYandMETADATA_SIGNER_CERTto enable Metadata signing features (optional, not required)METADATA_CACHE_CONTROLto set the Http-Header Cache control max-ageMETADATA_VALID_UNTILto set the freshness of the metadataThis projects doesn't need of any database configurationRun it in development mode./manage.py runserver 0.0.0.0:8001or in production one (see gunicorn or uwsgi examples to do that)To create your Metadata RSA keys you can even use this example command:openssl req -nodes -new -x509 -days 3650 -keyout certificates/private.key -out certificates/public.cert -subj '/CN=your.own.fqdn.com'Shibboleth IdP ConfigurationThis is a metadata-provider definition file that can be included in/opt/shibboleth-idp/conf/services.xml.
In the MetadataProviders resources, called<util:list id="shibboleth.MetadataResolverResources">we can put it as the child element<value>%{idp.home}/conf/metadata-providers-mdq.xml</value>.
Just changedjango_mdq.urlin your production url.<?xml version="1.0" encoding="UTF-8"?>
<!-- This file is an EXAMPLE metadata configuration file. -->
<MetadataProvider id="ShibbolethMetadataMdq" xsi:type="ChainingMetadataProvider"
    xmlns="urn:mace:shibboleth:2.0:metadata"
    xmlns:resource="urn:mace:shibboleth:2.0:resource"
    xmlns:security="urn:mace:shibboleth:2.0:security"
    xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="urn:mace:shibboleth:2.0:metadata http://shibboleth.net/schema/idp/shibboleth-metadata.xsd
                        urn:mace:shibboleth:2.0:resource http://shibboleth.net/schema/idp/shibboleth-resource.xsd
                        urn:mace:shibboleth:2.0:security http://shibboleth.net/schema/idp/shibboleth-security.xsd
                        urn:oasis:names:tc:SAML:2.0:metadata http://docs.oasis-open.org/security/saml/v2.0/saml-schema-metadata-2.0.xsd">

    <MetadataProvider id="DynamicEntityMetadata" xsi:type="DynamicHTTPMetadataProvider"
                  connectionRequestTimeout="PT5S"
                  connectionTimeout="PT5S"
                  socketTimeout="PT3S">

    <!-- Enable this if have configured METADATA_SIGNER_KEY and METADATA_SIGNER_CERT in Django-MDQ settingslocal.py
    <MetadataFilter xsi:type="SignatureValidation" requireSignedRoot="true"
                    certificateFile="%{idp.home}/credentials/mdq-cert.pem"/>
    -->

    <!-- Enable this if have configured METADATA_VALID_UNTIL in Django-MDQ settingslocal.py
    <MetadataFilter xsi:type="RequiredValidUntil" maxValidityInterval="P14D"/>
    -->

    <MetadataQueryProtocol>https://django_mdq.url/</MetadataQueryProtocol>
    </MetadataProvider>

</MetadataProvider>Test the configuration# reload ShibbolethIdP or Metadata Service
touch /opt/jetty/webapps/idp.xml

# do a mdquery
/opt/shibboleth-idp/bin/mdquery.sh -e https://coco.release-check.edugain.org/shibboleth --saml2 -u http://localhost:8080/idpA test with PySAML2import io
import json
import urllib.request

from saml2.mdstore import MetaDataMDX

def b64(entity_name):
    return '{base64}'+base64.b64encode(entity_name.encode()).decode()

# when available
mdq_url = "http://localhost:8001"
mdq_cert = "certificates/public.cert"

entity2check = 'https://idp.unical.it/idp/shibboleth'

cert = open(mdq_cert)

# omit cert if unavailable
mdx = MetaDataMDX(mdq_url, cert=cert)
# base64 entity name trasformation
# mdx = MetaDataMDX(mdq_url, cert=cert, entity_transform=b64)

# certificati
mdx.certs(entity2check, "idpsso", use="encryption")

# get certs from idp
mdx.service(entity2check, 'idpsso_descriptor', 'single_sign_on_service')
mdx.certs(entity2check, "idpsso", use="signing")AuthorsGiuseppe De Marcogiuseppe.demarco@unical.itCreditsIdentityPython