Repository: ipython/ipython

Issue:
Dynamic Traits for HasTraits Instances and Classes
Currently the installation of dynamic traits for `HasTraits` objects is something that remains absent from the IPython Traitlets module. From what I can see this problem has two prongs: one with installation at a class level (relatively simple) and the other with installation at an instance level (more nuanced). The first prong is for creating traits that are accessed by all the instances of a particular class. The other is for creating traits that are only accessed by a single instance and which remain invisible to all others. I currently have solutions for both problems and will be making a pull request shortly.

---

In either case there will probably need to be a function which takes a `TraitType`, default value, and other arguments to initialize a new `TraitType` instance that will later be installed. It might something look like what's shown below, where `args` and `kw` would act as ways for initializing the default value for a `ClassBasedTraitType`.

``` python
def trait_init(type, entry, args=None, kw=None, **metadata):
    if issubclass(type, TraitType):
        if issubclass(type, ClassBasedTraitType):
            if args is None:
                args=()
            if kw is None:
                kw={}
            instc = type(entry,args,kw,**metadata)
        else:
            # type is not class based
            instc = type(entry,**metadata)
            instc.default_value = entry
        return instc
    else:
        raise TraitError('{0} must inherit from TraitType'.format(type))
```

---

For the class level problem, the function that would actually install the trait would be fairly simple (I have a method for removing them as well but describing how they get there should give hints as to how removal works). The only real bit that needs to be done is to set the default value in each of the classes instances. I have come up with a way to accomplish this using two things. The first is a class attribute `_instances` which is a `set` associated with a decorator that uses weakrefs to store `HasTraits` instances. So the `__init__` for `HasTraits` would now include a line of code like the following `self.__class__._instances.add(weakref.ref(self))`. The second would then be the decorator for the attribute  `_instances`. This is shown below and would be used to return all the instances of a particular class:

``` python
    @classmethod
    def instances(cls):
        """Get a list of class instances"""
        dead = set()
        for ref in cls._instances:
            obj = ref()
            if (obj is not None):
                if isinstance(obj,cls):
                    yield obj
                else:
                    pass
            else:
                dead.add(ref)
        cls._instances -= dead
```

With this, the next step is to actually implement the function which will do the installing. With a method that will return all the instances of a particular class the function shown below only needs to iterate and set default values. It might look something like what's shown below:

``` python
def install_trait(klass, name, trait):
    if issubclass(klass,HasTraits):
        trait.this_class = klass
        trait.name = name
        for obj in klass.instances():
            trait.set_default_value(obj)
        setattr(klass, name, trait)
    else:
        raise TraitError()
```

---

Now moving on to the instance level problem. From what I can see there are two solutions; one involving metaclasses and another using a dictionary attribute in the instance to store `TraitType` objects.

For the metaclass solution, I was talking with @jdfreder who suggested a method where one could use a `HasTraits` object, that was created via a metaclass and stored as an instance level attribute, to hold dynamic traits. At this point it would simply be a matter of using the methods I described above to install `TraitType` objects at a class level.

The non-metaclass solution would override `__getattr__` and `__setattr__` in `HasTraits` to force the instance to utilize `__set__` and `__get__` from a trait in its dynamic trait dictionary (if such a trait existed). With this, the only thing required to install a new dynamic trait would be to add an initialized trait to the dynamic trait dictionary of an instance. I have a rough version of this working at the moment, however it could use some touch ups.

I was told by @ellisonbg and Jonathan that @sylvaincorlay and @jasongrout might have some suggestions on either the class or instance level problems of installing dynamic traits for `HasTraits` objects. This was already getting a bit long to write so let me know if there are questions on what I've written.



Discussion:
Can you describe the use case for this? It's not clear to me what it would be used for.

@minrk I have a bunch of usecases for dynamic traitlets. (I played a bit with @jdfreder 's implementation.)
- First usecase: a data store in notebook metadata (or elsewhere, it is not the point here). 

``` python
from IPython.html import DataStore

# DataStore() returns a singleton. It inherits from HasTrait and represent a
# traitlet-addressable section of the notebook metadata.
store = DataStore()
if not hasattr(store, 'value'):
    add_trait(store, 'value', Float(0))
slider = FloatSlider(value=store.value)
dlink((slider, 'value'), (store, 'value'))
# The slider is changes the value in the data store
# On notebook save, the value is saved in the ipynb metadata
# Next time you run the notebook, you find back the slider value. 
```

(I know how questionable storing in the notebook metadata is, this is just a nice API to access it if you like.)

I can put a couple of other usecases examples tomorrow if you are interested. 

@rmorshea this is a huge wall of text!! I'm glad that you opened the issue to talk about it though.

@minrk and @takluyver,
@rmorshea is a student of @ellisonbg who is working on an in notebook SVG rendering library (not widget based).  He is using traitlets extensively and has the need to be able to add traits to instances, not classes.  @rmorshea can you describe, briefly, why this need exists?

I'm interested in this question for another reason, which is to improve symmetry with the widgets.  I'd like to allow Javascript instantiated models to declare new synced traits.

My solution to this is a small, simple hack:

``` python
    def add_trait(self, traitname, trait):
        """Dynamically add a trait attribute to the HasTraits instance."""
        self.__class__ = type(self.__class__.__name__, (self.__class__,), 
                              {traitname: trait})
        trait.set_default_value(self)
```

(thanks for the commit @sylvaincorlay , https://github.com/SylvainCorlay/ipython/commit/24871d4d1aae9261f3ffc577474bf5813f1b6198)

@rmorshea 's idea, which is less of a hack but requires much more effort, is to implement the logic for instance level traits in `getattr` `setattr` overrides.

I wasn't quite sure how much information I should include so I probably over did it. But that aside, the main reason I ended up working on this is mainly because I _thought_ I needed to use it. So at the moment it might not be absolutely necessary in my use case. However I have implemented a case in which one might want to add a `slope` attribute to a `Group` object containing a series of shapes to rotate them a given number of degrees. I've demonstrated this with a [notebook](http://nbviewer.ipython.org/github/rmorshea/misc/blob/testing/docs/SVG.ipynb) in cells 13 and 14. In this particular case one probably wouldn't want `slope` to be an attribute that would be common to all `Group` objects.

Thanks @rmorshea 

@jdfreder I'm not sure that there's a use for dynamically installing traits at a class level though. I'd already written code to do it so I figured I'd include it in the discussion and see if someone had a need. The only purpose I can see it serving is avoiding inheritance or some obscure case where you might need to add an arbitrary number of traits. If this isn't relevant I can remove it from the issue description.

>  has the need to be able to add traits to instances

Aha, this is the part I didn't get from the description. I just didn't know what the goal was. That makes sense. I don't think adding traits to classes does, though. That's what subclasses are for.


Resolving patch:
diff --git a/IPython/html/widgets/widget.py b/IPython/html/widgets/widget.py
--- a/IPython/html/widgets/widget.py
+++ b/IPython/html/widgets/widget.py
@@ -292,6 +292,13 @@ def on_displayed(self, callback, remove=False):
             True if the callback should be unregistered."""
         self._display_callbacks.register_callback(callback, remove=remove)
 
+    def add_trait(self, traitname, trait):
+        """Dynamically add a trait attribute to the Widget."""
+        super(Widget, self).add_trait(traitname, trait)
+        if trait.get_metadata('sync'):
+             self.keys.append(traitname)
+             self.send_state(traitname)
+
     #-------------------------------------------------------------------------
     # Support methods
     #-------------------------------------------------------------------------
diff --git a/IPython/utils/traitlets.py b/IPython/utils/traitlets.py
--- a/IPython/utils/traitlets.py
+++ b/IPython/utils/traitlets.py
@@ -801,6 +801,12 @@ def trait_metadata(self, traitname, key, default=None):
         else:
             return trait.get_metadata(key, default)
 
+    def add_trait(self, traitname, trait):
+        """Dynamically add a trait attribute to the HasTraits instance."""
+        self.__class__ = type(self.__class__.__name__, (self.__class__,),
+                              {traitname: trait})
+        trait.set_default_value(self)
+
 #-----------------------------------------------------------------------------
 # Actual TraitTypes implementations/subclasses
 #-----------------------------------------------------------------------------


Test patch:
diff --git a/IPython/utils/tests/test_traitlets.py b/IPython/utils/tests/test_traitlets.py
--- a/IPython/utils/tests/test_traitlets.py
+++ b/IPython/utils/tests/test_traitlets.py
@@ -1563,3 +1563,55 @@ def test_klass(self):
 ###
 # End Forward Declaration Tests
 ###
+
+class TestDynamicTraits(TestCase):
+
+    def setUp(self):
+        self._notify1 = []
+
+    def notify1(self, name, old, new):
+        self._notify1.append((name, old, new))
+
+    def test_notify_all(self):
+
+        class A(HasTraits):
+            pass
+
+        a = A()
+        self.assertTrue(not hasattr(a, 'x'))
+        self.assertTrue(not hasattr(a, 'y'))
+
+        # Dynamically add trait x.
+        a.add_trait('x', Int())
+        self.assertTrue(hasattr(a, 'x'))
+        self.assertTrue(isinstance(a, (A, )))
+
+        # Dynamically add trait y.
+        a.add_trait('y', Float())
+        self.assertTrue(hasattr(a, 'y'))
+        self.assertTrue(isinstance(a, (A, )))
+        self.assertEqual(a.__class__.__name__, A.__name__)
+
+        # Create a new instance and verify that x and y
+        # aren't defined.
+        b = A()
+        self.assertTrue(not hasattr(b, 'x'))
+        self.assertTrue(not hasattr(b, 'y'))
+
+        # Verify that notification works like normal.
+        a.on_trait_change(self.notify1)
+        a.x = 0
+        self.assertEqual(len(self._notify1), 0)
+        a.y = 0.0
+        self.assertEqual(len(self._notify1), 0)
+        a.x = 10
+        self.assertTrue(('x', 0, 10) in self._notify1)
+        a.y = 10.0
+        self.assertTrue(('y', 0.0, 10.0) in self._notify1)
+        self.assertRaises(TraitError, setattr, a, 'x', 'bad string')
+        self.assertRaises(TraitError, setattr, a, 'y', 'bad string')
+        self._notify1 = []
+        a.on_trait_change(self.notify1, remove=True)
+        a.x = 20
+        a.y = 20.0
+        self.assertEqual(len(self._notify1), 0)