Package: message-tagging-service
Message tagging service is a microservice to tag build with proper tags, which
is triggered by specific message from a message bus. Currently, service
supports to tag module build according to a module build state change event.This service works with Python 3.6 and 3.7.WorkflowThis is the service workflow, for example of a module build.Listen on message bus, which isfedora-messagingin Fedora, and waiting
for module build state change events.Once received a message, MTS checks predefined rule definitions one by one in
the order of presence in rule file.If the module build matches a rule, apply tags defined in matched rule to
this build and its associated-develbuild.Send message to message bus to announce a module build is tagged with
specific tags. Please note that MTS does not actually wait for the tag build
task to finish. The message is sent just aftertagBuildreturns, which
just creates a tag build task and returns immediately.Rule DefinitionRule definition is documented in aModularity documentFor detailed information on how the rules are matched, please refer toparagraphin that document.ConfigurationApplication configurationconf/config.py. Each config is well-documented,
please refer to this file’s content for detailed information. Following three
config section are defined for different purpose to run MTS.BaseConfigurationprovides default options which could be reused for
running in production.DevConfigurationcontains anything for running in development mode.TestConfigurationcontains any config for test purpose.Messaging configuration for fedora-messaging included in fileconf/mts.toml,
which includes configs for receiving and sending messages from and to
fedora-messaging broker inTOMLformat. The default config values is defined
well for interacting with a local RabbitMQ broker, which is good for testing
locally. For deployment, copy this file and adjust config values for the Fedora
messaging broker.MessagingEventsbuild.tag.requestedMessage is sent when atagBuildtask is requested in Koji. An example message:{
  "build": {
    "id": id,
    "name": name,
    "stream": stream,
    "version": version,
    "context": context,
  },
  "nvr": N-V-R,
  "destination_tags": [
    {"tag": name_1, "task_id": 1},
    {"tag": name_2, "task_id": 2},
    ...
  ]
}where,destination_tagsis a list of mappings each of them contains the tag
to apply and corresponding task ID returned from Koji.build.tag.unmatchedMessage is sent if a module build does not match any predefined rules. An
example message:{
  "build": {
    "id": id,
    "name": name,
    "stream": stream,
    "version": version,
    "context": context,
  },
}The message simply contains the module build information.Topic PrefixFor Fedora, messages are sent to topics with prefixorg.fedoraproject.prod,
e.g.org.fedoraproject.prod.mts.build.tag.requested.For internal, the prefix isVirtualTopic.eng.mts, e.g.VirtualTopic.eng.mts.build.tag.requested.Environment VariablesMTS_DRY_RUNDry run mode. Currently, no build is tagged actually in dry run mode. No
particular is required. Just defineMTS_DRY_RUNin environment variables.MTS_DEVSwitch service to run in development mode as long asMTS_DEVis defined.
For example,export MTS_DEV=1.MTS_CONFIG_FILESpecify an alternative config file instead of reading theconf/config.pywhenMTS_DEVis enabled or/etc/mts/config.pyif MTS is installed in
system. Either an absolute path or relative path works.ContributionReport issue athttps://github.com/fedora-modularity/message-tagging-service/issues.Before making a pull request, ensure the changes do not break anything and are
covered by tests. Run tests:toxChange Logs0.8.1 (2020-06-22)Fix the field from build_state to state_name for state filter (Yu Ming Zhu)0.8 (2020-06-18)Only receive msgs with build_state is ready or done by default (Yu Ming Zhu)Unittest: the latest koji use gssapi_login instead of krb_login (Yu Ming Zhu)CI: Fix missing command to run tests (Chenxiong Qi)Use image fedora:31 in CI (Chenxiong Qi)0.7 (2020-02-07)Upgrade base image to Fedora 30Do not attempt to tag a scratch build (Chenxiong Qi)Do not handle scratch from rule (Chenxiong Qi)Add support for complex destinations (Luiz Carvalho)Run mts in the foreground (Luiz Carvalho)0.6 (2019-09-27)Revert “Log the exception when tag failures occur” (Chenxiong Qi)Allow durable queues when using rhmsg (Luiz Carvalho)Fix param type of dest_tags (Chenxiong Qi)Better logs (Chenxiong Qi)Log the exception when tag failures occur (mprahl)Set image version to latest (Chenxiong Qi)Fix consumer and add tests (Chenxiong Qi)Massive updates to README.rst (Chenxiong Qi)Fix variable name inside log message format (Chenxiong Qi)Include mts.toml config in sdist package (Chenxiong Qi)Do not include removed fedmsg.d/ directory (Chenxiong Qi)Remove fedmsg related lines from Dockerfile (Chenxiong Qi)Remove duplicate code handling rule scratch and development (Chenxiong Qi)Migrate to fedora-messaging (Chenxiong Qi)Refactor function is_file_readable (Chenxiong Qi)No need to set koji_cert if not use ssl auth (Chenxiong Qi)Ensure logout koji session if something wrong on session (Chenxiong Qi)Revert “Set USER for fedmsg to publish message” (Chenxiong Qi)Revert “Install packages for Fedora container” (Chenxiong Qi)Set USER for fedmsg to publish message (Chenxiong Qi)Install packages for Fedora container (Chenxiong Qi)Use new pytest.raises argument match (Chenxiong Qi)0.5 (2019-04-15)Change metric name to be in sync with other services (Filip Valder)Reduce the number of tries to match rules (Chenxiong Qi)Match module build by build state (Chenxiong Qi)0.4.1 (2019-04-01)Ignore https verify while downloading ca cert (Chenxiong Qi)0.4 (2019-03-30)Adjust gunicorn command line options (Chenxiong Qi)Increase the number of workers to run the web app (Chenxiong Qi)Set gunicorn log level to debug (Chenxiong Qi)Test image build and container in Travis-CI (Chenxiong Qi)Refactor Dockerfile (Chenxiong Qi)Add missing deps to Dockerfile (Chenxiong Qi)Add missing to break dnf-install command properly in Dockerfile (Chenxiong Qi)Expose metrics endpoint for monitoring (Chenxiong Qi)Add container badge in README (Chenxiong Qi)Include failed tagBuild task request in build.tag.requested message (Chenxiong Qi)0.3 (2019-02-20)Refine event topics (Chenxiong Qi)Fix badges in README (Chenxiong Qi)Better log when module build in init state (Chenxiong Qi)Use known good version of moksha-hub (Luiz Carvalho)Tag -devel CG Koji build (Luiz Carvalho)Handle multiple tags for single rule (Luiz Carvalho)Refine code for the first match wins (Chenxiong Qi)Use dedent in tests when mocking modulemd data (Luiz Carvalho)Only allow a single rule match (Luiz Carvalho)Make docker/install-ca.sh executable (mprahl)Add missing docker/install-ca.sh (mprahl)Add back the volumes for improved UX in OpenShift (mprahl)Connect over http when using the rcm-tools repo since the CA isn’t trusted (mprahl)Fix a comment in the Jenkinsfile (mprahl)Add a Jenkins job to build container images and push them to quay.io (mprahl)Install rhmsg in the container image (mprahl)Add the ability to install a custom CA in the container image (mprahl)Set the default container user to 1001 to mimic OpenShift (mprahl)Add additional DNF arguments to make the container image slightly smaller (mprahl)Remove the volumes in the Dockerfile that MTS doesn’t write to (mprahl)Don’t rely on default fedmsg configuration files in the container image (mprahl)Add Dockerfile for building prod image (Chenxiong Qi)Allow set None to a config (Chenxiong Qi)Refine configuration section in README (Chenxiong Qi)Fix consumer_topics in config (Chenxiong Qi)Install MTS and fedmsg.d config files (Chenxiong Qi)Fix grammar issues in README (Chenxiong Qi)Support multiple authtype to login a Koji session (Chenxiong Qi)Refactor fedmsg.d config (Chenxiong Qi)Convert README to RST format (Chenxiong Qi)Login koji session by calling koji_cli.lib.activate_session (Chenxiong Qi)Config update and reset methods. Extending tests for Config (Valerij Maljulin)Merge __getattr__ with __getitem__ in Config class (Valerij Maljulin)Adding support for profile parameter (Valerij Maljulin)Base class for configuration profiles (Valerij Maljulin)0.2 (2019-01-22)Add missing files to tarball generated by sdist0.1 (2019-01-21)First release that MTS is able to handle specific message to tag build.