Package: martian
A library to grok configuration from Python code.Martian tutorialIntroduction“There was so much to grok, so little to grok from.” – Stranger in a
Strange Land, by Robert A. HeinleinMartian provides infrastructure for declarative configuration of
Python code. Martian is especially useful for the construction of
frameworks that need to provide a flexible plugin
infrastructure. Martian doesn’t actually provide infrastructure for
plugin registries (except for itself). Many frameworks have their own
systems for this, and if you need a generic one, you might want to
considerzope.component. Martian just allows you to make the
registration of plugins less verbose.You can see Martian as doing something that you can also solve with
metaclasses, with the following advantages:the developer of the framework doesn’t have to write a lot of ad-hoc
metaclasses anymore; instead we offer an infrastructure to make life
easier.configuration doesn’t need to happen at import time, but can happen at
program startup time. This also makes configuration more tractable for
a developer.we don’t bother the developer thatusesthe framework with the
surprising behavior that metaclasses sometimes bring. The classes
the user has to deal with are normal classes.Why is this package namedmartian? In the novel “Stranger in a
Strange Land”, the verbgrokis introduced:Grok means to understand so thoroughly that the observer becomes a
part of the observed – to merge, blend, intermarry, lose identity
in group experience.In the context of this package, “grokking” stands for the process of
deducing declarative configuration actions from Python code. In the
novel, grokking is originally a concept that comes from the planet
Mars. Martiansgrok. Since this package helps you grok code, it’s
called Martian.Martian provides a framework that allows configuration to be expressed
in declarative Python code. These declarations can often be deduced
from the structure of the code itself. The idea is to make these
declarations so minimal and easy to read that even extensive
configuration does not overly burden the programmers working with the
code.Themartianpackage is a spin-off from theGrok project, in the
context of which this codebase was first developed. While Grok uses
it, the code is completely independent of Grok.Motivation“Deducing declarative configuration actions from Python code” - that
sounds very abstract. What does it actually mean? What is
configuration?  What is declarative configuration? In order to explain
this, we’ll first take a look at configuration.Larger frameworks often offer a lot of points where you can modify
their behavior: ways to combine its own components with components you
provide yourself to build a larger application. A framework offers
points where it can beconfiguredwith plugin code. When you plug
some code into a plugin point, it results in the updating of some
registry somewhere with the new plugin. When the framework uses a
plugin, it will first look it up in the registry. The action of
registering some component into a registry can be calledconfiguration.Let’s look at an example framework that offers a plugin point. We
introduce a very simple framework for plugging in different template
languages, where each template language uses its own extension. You
can then supply the framework with the template body and the template
extension and some data, and render the template.Let’s look at the framework:>>> import string
>>> class templating(FakeModule):
...
...   class InterpolationTemplate(object):
...      "Use %(foo)s for dictionary interpolation."
...      def __init__(self, text):
...          self.text = text
...      def render(self, **kw):
...          return self.text % kw
...
...   class TemplateStringTemplate(object):
...      "PEP 292 string substitutions."
...      def __init__(self, text):
...          self.template = string.Template(text)
...      def render(self, **kw):
...          return self.template.substitute(**kw)
...
...   # the registry, we plug in the two templating systems right away
...   extension_handlers = { '.txt': InterpolationTemplate,
...                          '.tmpl': TemplateStringTemplate }
...
...   def render(data, extension, **kw):
...      """Render the template at filepath with arguments.
...
...      data - the data in the file
...      extension - the extension of the file
...      keyword arguments - variables to interpolate
...
...      In a real framework you could pass in the file path instead of
...      data and extension, but we don't want to open files in our
...      example.
...
...      Returns the rendered template
...      """
...      template = extension_handlers[extension](data)
...      return template.render(**kw)Since normally we cannot create modules in a doctest, we have emulated
thetemplatingPython module using theFakeModuleclass. Whenever you seeFakeModulesubclasses, imagine you’re
looking at a module definition in a.pyfile. Now that we have
defined a moduletemplating, we also need to be able to import
it. Fake modules are always placed automatically into themartiantest.fakenamespace so you can import them from there:>>> from martiantest.fake import templatingNow let’s try therenderfunction for the registered template
types, to demonstrate that our framework works:>>> templating.render('Hello %(name)s!', '.txt', name="world")
'Hello world!'
>>> templating.render('Hello ${name}!', '.tmpl', name="universe")
'Hello universe!'File extensions that we do not recognize cause aKeyErrorto be
raised:>>> templating.render('Hello', '.silly', name="test")
Traceback (most recent call last):
...
KeyError: '.silly'We now want to plug into this filehandler framework and provide a
handler for.sillyfiles. Since we are writing a plugin, we cannot
change thetemplatingmodule directly. Let’s write an extension
module instead:>>> class sillytemplating(FakeModule):
...   class SillyTemplate(object):
...      "Replace {key} with dictionary values."
...      def __init__(self, text):
...          self.text = text
...      def render(self, **kw):
...          text = self.text
...          for key, value in kw.items():
...              text = text.replace('{%s}' % key, value)
...          return text
...
...   templating.extension_handlers['.silly'] = SillyTemplate
>>> from martiantest.fake import sillytemplatingIn the extension module, we manipulate theextension_handlersdictionary of thetemplatingmodule (in normal code we’d need to
import it first), and plug in our own function..sillyhandling
works now:>>> templating.render('Hello {name}!', '.silly', name="galaxy")
'Hello galaxy!'Above we plug into ourextension_handlerregistry using Python
code. Using separate code to manually hook components into registries
can get rather cumbersome - each time you write a plugin, you also
need to remember you need to register it.Doing template registration in Python code also poses a maintenance
risk. It is tempting to start doing fancy things in Python code such
as conditional configuration, making the configuration state of a
program hard to understand. Another problem is that doing
configuration at import time can also lead to unwanted side effects
during import, as well as ordering problems, where you want to import
something that really needs configuration state in another module that
is imported later. Finally, it can also make code harder to test, as
configuration is loaded always when you import the module, even if in
your test perhaps you don’t want it to be.Martian provides a framework that allows configuration to be expressed
in declarative Python code. Martian is based on the realization that
what to configure where can often be deduced from the structure of
Python code itself, especially when it can be annotated with
additional declarations. The idea is to make it so easy to write and
register a plugin so that even extensive configuration does not overly
burden the developer.Configuration actions are executed during a separate phase (“grok
time”), not at import time, which makes it easier to reason about and
easier to test.Configuration the Martian WayLet’s now transform the abovetemplatingmodule and thesillytemplatingmodule to use Martian. First we must recognize
that every template language is configured to work for a particular
extension. With Martian, we annotate the classes themselves with this
configuration information. Annotations happen usingdirectives,
which look like function calls in the class body.Let’s create anextensiondirective that can take a single string
as an argument, the file extension to register the template class
for:>>> import martian
>>> class extension(martian.Directive):
...   scope = martian.CLASS
...   store = martian.ONCE
...   default = NoneWe also need a way to easily recognize all template classes. The normal
pattern for this in Martian is to use a base class, so let’s define aTemplatebase class:>>> class Template(object):
...   passWe now have enough infrastructure to allow us to change the code to use
Martian style base class and annotations:>>> class templating(FakeModule):
...
...   class InterpolationTemplate(Template):
...      "Use %(foo)s for dictionary interpolation."
...      extension('.txt')
...      def __init__(self, text):
...          self.text = text
...      def render(self, **kw):
...          return self.text % kw
...
...   class TemplateStringTemplate(Template):
...      "PEP 292 string substitutions."
...      extension('.tmpl')
...      def __init__(self, text):
...          self.template = string.Template(text)
...      def render(self, **kw):
...          return self.template.substitute(**kw)
...
...   # the registry, empty to start with
...   extension_handlers = {}
...
...   def render(data, extension, **kw):
...      # this hasn't changed
...      template = extension_handlers[extension](data)
...      return template.render(**kw)
>>> from martiantest.fake import templatingAs you can see, there have been very few changes:we made the template classes inherit fromTemplate.we use theextensiondirective in the template classes.we stopped pre-filling theextension_handlersdictionary.So how do we fill theextension_handlersdictionary with the right
template languages? Now we can use Martian. We define agrokkerforTemplatethat registers the template classes in theextension_handlersregistry:>>> class meta(FakeModule):
...   class TemplateGrokker(martian.ClassGrokker):
...     martian.component(Template)
...     martian.directive(extension)
...     def execute(self, class_, extension, **kw):
...       templating.extension_handlers[extension] = class_
...       return True
>>> from martiantest.fake import metaWhat does this do? AClassGrokkerhas itsexecutemethod
called for subclasses of what’s indicated by themartian.componentdirective. You can also declare what directives aClassGrokkerexpects on this component by usingmartian.directive()(thedirectivedirective!) one or more times.Theexecutemethod takes the class to be grokked as the first
argument, and the values of the directives used will be passed in as
additional parameters into theexecutemethod. The framework can
also pass along an arbitrary number of extra keyword arguments during
the grokking process, so we need to declare**kwto make sure we
can handle these.All our grokkers will be collected in a special Martian-specific
registry:>>> reg = martian.GrokkerRegistry()We will need to make sure the system is aware of theTemplateGrokkerdefined in themetamodule first, so let’s
register it first. We can do this by simply grokking themetamodule:>>> reg.grok('meta', meta)
TrueBecauseTemplateGrokkeris now registered, our registry now knows
how to grokTemplatesubclasses. Let’s grok thetemplatingmodule:>>> reg.grok('templating', templating)
TrueLet’s try therenderfunction of templating again, to demonstrate
we have successfully grokked the template classes:>>> templating.render('Hello %(name)s!', '.txt', name="world")
'Hello world!'
>>> templating.render('Hello ${name}!', '.tmpl', name="universe")
'Hello universe!'.sillyhasn’t been registered yet:>>> templating.render('Hello', '.silly', name="test")
Traceback (most recent call last):
...
KeyError: '.silly'Let’s now register.sillyfrom an extension module:>>> class sillytemplating(FakeModule):
...   class SillyTemplate(Template):
...      "Replace {key} with dictionary values."
...      extension('.silly')
...      def __init__(self, text):
...          self.text = text
...      def render(self, **kw):
...          text = self.text
...          for key, value in kw.items():
...              text = text.replace('{%s}' % key, value)
...          return text
>>> from martiantest.fake import sillytemplatingAs you can see, the developer that uses the framework has no need
anymore to know abouttemplating.extension_handlers. Instead we can
simply grok the module to haveSillyTemplatebe register appropriately:>>> reg.grok('sillytemplating', sillytemplating)
TrueWe can now use the.sillytemplating engine too:>>> templating.render('Hello {name}!', '.silly', name="galaxy")
'Hello galaxy!'Admittedly it is hard to demonstrate Martian well with a small example
like this. In the end we have actually written more code than in the
basic framework, after all. But even in this small example, thetemplatingandsillytemplatingmodule have become more
declarative in nature. The developer that uses the framework will not
need to know anymore about things liketemplating.extension_handlersor an API to register things
there. Instead the developer can registering a new template system
anywhere, as long as he subclasses fromTemplate, and as long as
his code is grokked by the system.Finally note how Martian was used to define theTemplateGrokkeras
well. In this way Martian can use itself to extend itself.Grokking instancesAbove we’ve seen how you can grok classes. Martian also supplies a way
to grok instances. This is less common in typical frameworks, and has
the drawback that no class-level directives can be used, but can still
be useful.Let’s imagine a case where we have a zoo framework with anAnimalclass, and we want to track instances of it:>>> class Animal(object):
...   def __init__(self, name):
...     self.name = name
>>> class zoo(FakeModule):
...   horse = Animal('horse')
...   chicken = Animal('chicken')
...   elephant = Animal('elephant')
...   lion = Animal('lion')
...   animals = {}
>>> from martiantest.fake import zooWe define anInstanceGrokkersubclass to grokAnimalinstances:>>> class meta(FakeModule):
...   class AnimalGrokker(martian.InstanceGrokker):
...     martian.component(Animal)
...     def execute(self, instance, **kw):
...       zoo.animals[instance.name] = instance
...       return True
>>> from martiantest.fake import metaLet’s create a new registry with theAnimalGrokkerin it:>>> reg = martian.GrokkerRegistry()
>>> reg.grok('meta', meta)
TrueWe can now grok thezoomodule:>>> reg.grok('zoo', zoo)
TrueThe animals will now be in theanimalsdictionary:>>> sorted(zoo.animals.items())
[('chicken', <Animal object at ...>),
 ('elephant', <Animal object at ...>),
 ('horse', <Animal object at ...>),
 ('lion', <Animal object at ...>)]More informationFor many more details and examples of more kinds of grokkers, please
seesrc/martian/core.txt. For more information on directives seesrc/martian/directive.txt.CHANGES2.0.post1 (2023-03-23)Add missingpython_requiresinsetup.pyto prevent installing on too
old versions.2.0 (2023-03-23)Add support for Python 3.11.Drop support for Python 2.7, 3.5, 3.6.Fix test suite incompatibility withzope.interface >= 6.0.1.5 (2022-02-11)Title directive now correctly handles non-ascii characters.Fix test suite incompatibility with zope.interface >= 5.0.Drop support for Python 3.4.Add support for Python 3.9 and 3.10.1.4 (2020-02-23)Check forbuiltins(Python 3) everywhere that we check for__builtin__(Python 2).1.3.post1 (2019-03-14)Fix rendering of PyPI page.1.3 (2019-03-14)Add support for Python 3.7 and 3.8.1.2 (2018-05-09)Add a new directivemartian.ignore()to explicitly not grok
something in a module:class Example:
    pass

martian.ignore('Example')Fix the code to be pep 8 compliant.1.1 (2018-01-25)Bypass bootstrap, add coverage to toxFixinspect.getargspec()deprecation in python31.0 (2017-10-19)Add support for Python 3.5, 3.6, PyPy2 and PyPy3.Drop support for Python 2.6 and 3.3.0.15 (2015-04-21)compatibility for python 3adjust egg to work with newer version of setuptoolsFix an encoding issue under Python-2.7 in the doctests.0.14 (2010-11-03)Feature changesThe computation of the default value for a directive can now be defined inside
the directive class definition. Whenever there is aget_defaultclassmethod, it is used for computing the default:class name(Directive):
    scope = CLASS
    store = ONCE

    @classmethod
    def get_default(cls, component, module=None, **data):
       return component.__name__.lower()When binding the directive, the default-default behaviour can still be
overriden by passing aget_defaultfunction:def another_default(component, module=None, **data):
   return component.__name__.lower()

name.bind(get_default=another_default).get(some_component)Making the default behaviour intrinsic to the directive, prevents having to
pass theget_defaultfunction over and over when getting values, for
example in the grokkers.0.13 (2010-11-01)Feature changesIgnore all __main__ modules.List zope.testing as a test dependency.0.12 (2009-06-29)Feature changesChanges to better support various inheritance scenarios in combination with
directives. Details follow.CLASS_OR_MODULEscope directives will be aware of inheritance of
values that are defined in module-scope. Consider the following case:module a:
  some_directive('A')
  class Foo(object):
    pass

module b:
  import a
  class Bar(a.Foo):
    passAs before,Foowill have the valueAconfigured for it.Bar,
since it inherits fromFoo, will inherit this value.CLASS_OR_MODULEandCLASSscope directives will be aware of
inheritance of computed default values. Consider the following case:module a:
  class Foo(object):
     pass

module b:
  import a
  class Bar(a.Foo):
     pass

def get_default(component, module, **data):
    if module.__name__ == 'a':
       return "we have a default value for module a"
    return martian.UNKNOWNWhen we now do this:some_directive.bind(get_default=get_default).get(b.Bar)We will get the value “we have a default value for module a”. This
is because when trying to compute the default value forBarwe
returnedmartian.UNKNOWNto indicate the value couldn’t be found
yet. The system then looks at the base class and tries again, and in
this case it succeeds (as the module-name isa).martian.ONCE_IFACEstorage option to allow the creation of
directives that store their value onzope.interfaceinterfaces. This was originally ingrokcore.viewbut was of
wider usefulness.Bugs fixedIgnore things that look like Python modules and packages but aren’t.
These are sometimes created by editors, operating systems and
network file systems and we don’t want to confuse them.Ignore .pyc and .pyo files that don’t have a matching .py file viamodule_info_from_dotted_nameif itsignore_nonsourceparameter isTrue.  The default isTrue.  To revert to the
older behavior where .pyc files were honored, passignore_nonsource=False.Pass alongexclude_filter(and the newignore_nonsourceflag) to ModuleInfo constructor when it calls itself recursively.Replacefake_importto import fake modules in tests with a real
python import statement (from martiantest.fake import my_fake_module). This works by introducing a metaclass forFakeModulethat automatically registers it as a module. The
irony does not escape us. This also means thatmartian.scan.resolve()will now work on fake modules.0.11 (2008-09-24)Feature changesAdded MULTIPLE_NOBASE option for directive store. This is like MULTIPLE
but doesn’t inherit information from the base class.0.10 (2008-06-06)Feature changesAdd avalidateClassvalidate function for directives.MovedFakeModuleandfake_importinto amartian.testingmodule so that they can be reused by external packages.Introduce new tutorial text as README.txt. The text previously inREADME.txtwas rather too detailed for a tutorial, so has been
moved intocore.txt.Introduce aGrokkerRegistryclass that is aModuleGrokkerwith aMetaMultiGrokkerin it. This is the convenient thing to
instantiate to start working with Grok and is demonstrated in the
tutorial.Introduced three new martian-specific directives:martian.component,martian.directiveandmartian.priority. These replace thecomponent_class,directivesandpriorityclass-level attributes. This way
Grokkers look the same as what they grok. This breaks backwards
compatibility again, but it’s an easy replace operation. Note thatmartian.directivetakes the directive itself as an argument, and
then optionally the same arguments as thebindmethod of
directives (name,defaultandget_default). It may be
used multiple times. Note thatmartian.baseclasswas already a
Martian-specific directive and this has been unchanged.For symmetry, add anexecutemethod toInstanceGrokker.0.9.7 (2008-05-29)Feature changesAdded aMethodGrokkerbase class for grokkers that want to grok
methods of a class rather than the whole class itself.  It works
quite similar to theClassGrokkerregarding directive
definition, except that directives evaluated not only on class (and
possibly module) level but also for each method.  That way,
directives can also be applied to methods (as decorators) in case
they support it.0.9.6 (2008-05-14)Feature changesRefactored themartian.Directivebase class yet again to allow
more declarative (rather than imperative) usage in grokkers.
Directives themselves no longer have aget()method nor a
default value factory (get_default()).  Instead you will have to
“bind” the directive first which is typically done in a grokker.Extended theClassGrokkerbaseclass with a standardgrok()method that allows you to simply declare a set of directives that
are used on the grokked classes.  Then you just have to implement anexecute()method that will receive the data from those
directives as keyword arguments.  This simplifies the implementation
of class grokkers a lot.0.9.5 (2008-05-04)scan_for_classesjust needs a single second argument specifying
an interface. The support for scanning for subclasses directly has
been removed as it became unnecessary (due to changes in
grokcore.component).0.9.4 (2008-05-04)Features changesReplaced the various directive base classes with a singlemartian.Directivebase class:The directive scope is now defined with thescopeclass
attribute using one ofmartian.CLASS,martian.MODULE,martian.CLASS_OR_MODULE.The type of storage is defined with thestoreclass attribute
using one ofmartian.ONCE,martian.MULTIPLE,martian.DICT.Directives have now gained the ability to read the value that they
have set on a component or module using aget()method.  Theclass_annotationandclass_annotation_listhelpers have
been removed as a consequence.Moved thebaseclass()directive from Grok to Martian.Added amartian.util.check_provides_onehelper, in analogy tocheck_implements_one.Thescan_for_classeshelper now also accepts aninterfaceargument which allows you to scan for classes based on interface
rather than base classes.Bug fixesadded dummypackage_dotted_nametoBuiltinModuleInfo. This
allows the grokking of views in test code using Grok’sgrok.testing.grok_componentwithout a failure when it sets up thestaticattribute.no longer use the convention that classes ending in -Base will be considered
base classes. You must now explicitly use the grok.baseclass() directive.The type check of classes uses isinstance() instead of type(). This means
Grok can work with Zope 2 ExtensionClasses and metaclass programming.0.9.3 (2008-01-26)Feature changesAdded an OptionalValueDirective which allows the construction of
directives that take either zero or one argument. If no arguments
are given, thedefault_valuemethod on the directive is
called. Subclasses need to override this to return the default value
to use.RestructuringMove some util functions that were really grok-specific out of Martian
back into Grok.0.9.2 (2007-11-20)Bug fixesscan.module_info_from_dotted_name() now has special behavior when it
runs into __builtin__. Previously, it would crash with an error. Now
it will return an instance of BuiltinModuleInfo. This is a very
simple implementation which provides just enough information to make
client code work. Typically this client code is test-related so that
the module context will be __builtin__.0.9.1 (2007-10-30)Feature changesGrokkers now receive amodule_infokeyword argument.  This
change is completely backwards-compatible since grokkers which don’t
takemodule_infoexplicitly will absorb the extra argument in**kw.0.9 (2007-10-02)Feature changesReverted the behaviour where modules called tests or ftests were skipped
by default and added an API to provides a filtering function for skipping
modules to be grokked.0.8.1 (2007-08-13)Feature changesDon’t grok tests or ftests modules.Bugs fixedFix a bug where if a class had multiple base classes, this could end up
in the resultant list multiple times.0.8 (2007-07-02)Feature changesInitial public release.