Repository: elastic/elasticsearch-dsl-py

Issue:
Search does not support unicode indices
In elasticsearch_dsl/search.py, the `index` function checks to see if the arguments are of type `str` before appending them to the list of search indices. On Python 2, this test fails for unicode strings.

Instead, we can use `six.string_types` for Python 2 and 3 compatibility.

This problem may also occur elsewhere.

To reproduce:

```
from elasticsearch_dsl.search import Search
s = Search()
s.index('twitter-2015.01.01')  # works
s.index(u'twitter-2015.01.01')  # does not work
```

Discussion:

Patch:
diff --git a/Changelog.rst b/Changelog.rst
index f29287b..f001879 100644
--- a/Changelog.rst
+++ b/Changelog.rst
@@ -3,13 +3,39 @@
 Changelog
 =========
 
-5.4.0 (dev)
+6.0.0 (dev)
 -----------
+
+Release compatible with elasticsearch 6.0, changes include:
+
+ * use ``doc`` as default ``DocType`` name, this change includes:
+   * ``DocType._doc_type.matches`` method is now used to determine which
+   ``DocType`` should be used for a hit instead of just checking ``_type``
+ * ``Nested`` and ``Object`` field refactoring using newly introduced
+   ``InnerDoc`` class. To define a ``Nested``/``Object`` field just define the
+   ``InnerDoc`` subclass and then use it when defining the field::
+
+      class Comment(InnerDoc):
+          body = Text()
+          created_at = Date()
+
+      class Blog(DocType):
+          comments = Nested(Comment)
+
+ * methods on ``connections`` singleton are now exposed on the ``connections``
+   module directly.
+ * field values are now only deserialized when coming from elasticsearch (via
+   ``from_es`` method) and not when assigning values in python (either by
+   direct assignment or in ``__init__``).
+
+5.4.0 (2017-12-06)
+------------------
  * fix ``ip_range`` aggregation and rename the class to ``IPRange``.
    ``Iprange`` is kept for bw compatibility
  * fix bug in loading an aggregation with meta data from dict
- * add support for ``normalizer`` paramter of ``Keyword`` fields
+ * add support for ``normalizer`` parameter of ``Keyword`` fields
  * ``IndexTemplate`` can now be specified using the same API as ``Index``
+ * ``Boolean`` field now accepts ``"false"`` as ``False``
 
 5.3.0 (2017-05-18)
 ------------------
diff --git a/docs/configuration.rst b/docs/configuration.rst
index 7266efe..4439885 100644
--- a/docs/configuration.rst
+++ b/docs/configuration.rst
@@ -51,7 +51,7 @@ To define a default connection that will be used globally, use the
 
 .. code:: python
 
-    from elasticsearch_dsl.connections import connections
+    from elasticsearch_dsl import connections
 
     connections.create_connection(hosts=['localhost'], timeout=20)
 
@@ -68,7 +68,7 @@ time using the ``configure`` method:
 
 .. code:: python
 
-    from elasticsearch_dsl.connections import connections
+    from elasticsearch_dsl import connections
 
     connections.configure(
         default={'hosts': 'localhost'},
diff --git a/docs/persistence.rst b/docs/persistence.rst
index 2698c27..5fab1a8 100644
--- a/docs/persistence.rst
+++ b/docs/persistence.rst
@@ -9,7 +9,11 @@ layer for your application.
 Mappings
 --------
 
-The mapping definition follows a similar pattern to the query dsl:
+If you wish to create mappings manually you can use the ``Mapping`` class, for
+more advanced use cases, however, we recommend you use the :ref:`doc_type`
+abstraction in combination with :ref:`index` (or ``IndexTemplate``) to define
+index-level settings and properties. The mapping definition follows a similar
+pattern to the query dsl:
 
 .. code:: python
 
@@ -25,9 +29,11 @@ The mapping definition follows a similar pattern to the query dsl:
     m.field('category', 'text', fields={'raw': Keyword()})
 
     # you can also create a field manually
-    comment = Nested()
-    comment.field('author', Text())
-    comment.field('created_at', Date())
+    comment = Nested(
+                     properties={
+                        'author': Text(),
+                        'created_at': Date()
+                     })
 
     # and attach it to the mapping
     m.field('comments', comment)
@@ -99,6 +105,8 @@ specify type (``nGram`` in our example).
     either not exist or be closed. To create multiple ``DocType``-defined
     mappings you can use the :ref:`index` object.
 
+.. _doc_type:
+
 DocType
 -------
 
@@ -109,7 +117,7 @@ If you want to create a model-like wrapper around your documents, use the
 
     from datetime import datetime
     from elasticsearch_dsl import DocType, Date, Nested, Boolean, \
-        analyzer, InnerObjectWrapper, Completion, Keyword, Text
+        analyzer, InnerDoc, Completion, Keyword, Text
 
     html_strip = analyzer('html_strip',
         tokenizer="standard",
@@ -117,7 +125,11 @@ If you want to create a model-like wrapper around your documents, use the
         char_filter=["html_strip"]
     )
 
-    class Comment(InnerObjectWrapper):
+    class Comment(InnerDoc):
+        author = Text(fields={'raw': Keyword()})
+        content = Text(analyzer='snowball')
+        created_at = Date()
+
         def age(self):
             return datetime.now() - self.created_at
 
@@ -131,21 +143,14 @@ If you want to create a model-like wrapper around your documents, use the
             fields={'raw': Keyword()}
         )
 
-        comments = Nested(
-            doc_class=Comment,
-            properties={
-                'author': Text(fields={'raw': Keyword()}),
-                'content': Text(analyzer='snowball'),
-                'created_at': Date()
-            }
-        )
+        comments = Nested(Comment)
 
         class Meta:
             index = 'blog'
 
         def add_comment(self, author, content):
             self.comments.append(
-              {'author': author, 'content': content})
+              Comment(author=author, content=content, created_at=datetime.now()))
 
         def save(self, ** kwargs):
             self.created_at = datetime.now()
@@ -199,7 +204,7 @@ explicitly:
     first.save()
 
 
-All the metadata fields (``id``, ``parent``, ``routing``, ``index`` etc) can be
+All the metadata fields (``id``, ``routing``, ``index`` etc) can be
 accessed (and set) via a ``meta`` attribute or directly using the underscored
 variant:
 
@@ -271,16 +276,12 @@ accessed through the ``_doc_type`` attribute of the class:
 
 .. code:: python
 
-    # name of the type and index in elasticsearch
-    Post._doc_type.name
+    # name of the index in elasticsearch
     Post._doc_type.index
 
     # the raw Mapping object
     Post._doc_type.mapping
 
-    # the optional name of the parent type (if defined)
-    Post._doc_type.parent
-
 The ``_doc_type`` attribute is also home to the ``refresh`` method which will
 update the mapping on the ``DocType`` from elasticsearch. This is very useful
 if you use dynamic mappings and want the class to be aware of those fields (for
@@ -352,8 +353,8 @@ In the ``Meta`` class inside your document definition you can define various
 metadata for your document:
 
 ``doc_type``
-  name of the doc_type in elasticsearch. By default it will be constructed from
-  the class name (MyDocument -> my_document)
+  name of the doc_type in elasticsearch. By default it will be set to ``doc``,
+  it is not recommended to change.
 
 ``index``
   default index for the document, by default it is empty and every operation
@@ -366,8 +367,15 @@ metadata for your document:
   optional instance of ``Mapping`` class to use as base for the mappings
   created from the fields on the document class itself.
 
+``matches(self, hit)``
+  method that returns ``True`` if a given raw hit (``dict`` returned from
+  elasticsearch) should be deserialized using this ``DocType`` subclass. Can be
+  overriden, by default will just check that values for ``_index`` (including
+  any wildcard expansions) and ``_type`` in the document matches those in
+  ``_doc_type``.
+
 Any attributes on the ``Meta`` class that are instance of ``MetaField`` will be
-used to control the mapping of the meta fields (``_all``, ``_parent`` etc).
+used to control the mapping of the meta fields (``_all``, ``dynamic`` etc).
 Just name the parameter (without the leading underscore) as the field you wish
 to map and pass any parameters to the ``MetaField`` class:
 
@@ -378,7 +386,6 @@ to map and pass any parameters to the ``MetaField`` class:
 
         class Meta:
             all = MetaField(enabled=False)
-            parent = MetaField(type='blog')
             dynamic = MetaField('strict')
 
 .. _index:
diff --git a/elasticsearch_dsl/__init__.py b/elasticsearch_dsl/__init__.py
index aae0c1b..a8890de 100644
--- a/elasticsearch_dsl/__init__.py
+++ b/elasticsearch_dsl/__init__.py
@@ -3,12 +3,12 @@ from .aggs import A
 from .function import SF
 from .search import Search, MultiSearch
 from .field import *
-from .document import DocType, MetaField
+from .document import DocType, MetaField, InnerDoc
 from .mapping import Mapping
 from .index import Index, IndexTemplate
 from .analysis import analyzer, token_filter, char_filter, tokenizer
 from .faceted_search import *
 
-VERSION = (5, 4, 0, 'dev')
+VERSION = (6, 0, 0, 'dev')
 __version__ = VERSION
 __versionstr__ = '.'.join(map(str, VERSION))
diff --git a/elasticsearch_dsl/connections.py b/elasticsearch_dsl/connections.py
index bdc62e0..80d33a2 100644
--- a/elasticsearch_dsl/connections.py
+++ b/elasticsearch_dsl/connections.py
@@ -94,3 +94,8 @@ class Connections(object):
             raise KeyError('There is no connection with alias %r.' % alias)
 
 connections = Connections()
+configure = connections.configure
+add_connection = connections.add_connection
+remove_connection = connections.remove_connection
+create_connection = connections.create_connection
+get_connection = connections.get_connection
diff --git a/elasticsearch_dsl/document.py b/elasticsearch_dsl/document.py
index 8b94f2f..bb8d1ef 100644
--- a/elasticsearch_dsl/document.py
+++ b/elasticsearch_dsl/document.py
@@ -1,26 +1,17 @@
 import collections
-import re
+from fnmatch import fnmatch
 
 from elasticsearch.exceptions import NotFoundError, RequestError
 from six import iteritems, add_metaclass
 
 from .field import Field
 from .mapping import Mapping
-from .utils import ObjectBase, AttrDict, merge
+from .utils import ObjectBase, AttrDict, merge, DOC_META_FIELDS, META_FIELDS
 from .response import HitMeta
 from .search import Search
 from .connections import connections
 from .exceptions import ValidationException, IllegalOperation
 
-DOC_META_FIELDS = frozenset((
-    'id', 'routing', 'version', 'version_type'
-))
-
-META_FIELDS = frozenset((
-    # Elasticsearch metadata fields, except 'type'
-    'index', 'using', 'score',
-)).union(DOC_META_FIELDS)
-
 
 class MetaField(object):
     def __init__(self, *args, **kwargs):
@@ -44,10 +35,8 @@ class DocTypeOptions(object):
         # default cluster alias, can be overriden in doc.meta
         self._using = getattr(meta, 'using', None)
 
-        # get doc_type name, if not defined take the name of the class and
-        # transform it to lower_case
-        doc_type = getattr(meta, 'doc_type',
-                re.sub(r'(.)([A-Z])', r'\1_\2', name).lower())
+        # get doc_type name, if not defined use 'doc'
+        doc_type = getattr(meta, 'doc_type', 'doc')
 
         # create the mapping instance
         self.mapping = getattr(meta, 'mapping', Mapping(doc_type))
@@ -72,6 +61,9 @@ class DocTypeOptions(object):
                 self._using = self._using or b._doc_type._using
                 self.index = self.index or b._doc_type.index
 
+        # custom method to determine if a hit belongs to this DocType
+        self._matches = getattr(meta, 'matches', None)
+
     @property
     def using(self):
         return self._using or 'default'
@@ -95,6 +87,20 @@ class DocTypeOptions(object):
     def refresh(self, index=None, using=None):
         self.mapping.update_from_es(index or self.index, using=using or self.using)
 
+    def matches(self, hit):
+        if self._matches is not None:
+            return self._matches(hit)
+
+        return (
+                self.index is None or fnmatch(hit.get('_index', ''), self.index)
+            ) and self.name == hit.get('_type')
+
+@add_metaclass(DocTypeMeta)
+class InnerDoc(ObjectBase):
+    """
+    Common class for inner documents like Object or Nested
+    """
+
 
 @add_metaclass(DocTypeMeta)
 class DocType(ObjectBase):
@@ -250,27 +256,6 @@ class DocType(ObjectBase):
             raise NotFoundError(404, message, {'docs': missing_docs})
         return objs
 
-    @classmethod
-    def from_es(cls, hit):
-        """
-        Helper method to construct an instance from a dictionary returned by
-        elasticsearch.
-        """
-        # don't modify in place
-        meta = hit.copy()
-        doc = meta.pop('_source', {})
-
-        if 'fields' in meta:
-            for k, v in iteritems(meta.pop('fields')):
-                if k == '_source':
-                    doc.update(v)
-                if k.startswith('_') and k[1:] in META_FIELDS:
-                    meta[k] = v
-                else:
-                    doc[k] = v
-
-        return cls(meta=meta, **doc)
-
     def _get_connection(self, using=None):
         return connections.get_connection(using or self._doc_type.using)
     connection = property(_get_connection)
diff --git a/elasticsearch_dsl/field.py b/elasticsearch_dsl/field.py
index 418ccc5..ab6f7d8 100644
--- a/elasticsearch_dsl/field.py
+++ b/elasticsearch_dsl/field.py
@@ -2,7 +2,7 @@ import collections
 
 from datetime import date, datetime
 from dateutil import parser, tz
-from six import itervalues, string_types
+from six import itervalues, string_types, iteritems
 from six.moves import map
 
 from .utils import DslBase, ObjectBase, AttrDict, AttrList
@@ -99,70 +99,50 @@ class CustomField(Field):
         d['type'] = self.builtin_type
         return d
 
-class InnerObjectWrapper(ObjectBase):
-    def __init__(self, mapping, **kwargs):
-        # mimic DocType behavior with _doc_type.mapping
-        super(AttrDict, self).__setattr__('_doc_type', type('Meta', (), {'mapping': mapping}))
-        super(InnerObjectWrapper, self).__init__(**kwargs)
-
-
-class InnerObject(object):
-    " Common functionality for nested and object fields. "
-    _param_defs = {'properties': {'type': 'field', 'hash': True}}
+class Object(Field):
+    name = 'object'
     _coerce = True
 
-    def __init__(self, *args, **kwargs):
-        self._doc_class = kwargs.pop('doc_class', InnerObjectWrapper)
-        super(InnerObject, self).__init__(*args, **kwargs)
+    def __init__(self, doc_class=None, **kwargs):
+        self._doc_class = doc_class
+        if doc_class is None:
+            # FIXME import
+            from .document import InnerDoc
+            # no InnerDoc subclass, creating one instead...
+            self._doc_class = type('InnerDoc', (InnerDoc, ), {})
+            for name, field in iteritems(kwargs.pop('properties', {})):
+                self._doc_class._doc_type.mapping.field(name, field)
+            if 'dynamic' in kwargs:
+                self._doc_class._doc_type.mapping.meta('dynamic', kwargs.pop('dynamic'))
+
+        self._mapping = self._doc_class._doc_type.mapping
+        super(Object, self).__init__(**kwargs)
+
+    def __getitem__(self, name):
+        return self._mapping[name]
 
-    def field(self, name, *args, **kwargs):
-        self.properties[name] = construct_field(*args, **kwargs)
-        return self
-    # XXX: backwards compatible, will be removed
-    property = field
+    def __contains__(self, name):
+        return name in self._mapping
 
     def _empty(self):
-        return self._doc_class(self.properties)
+        return self._wrap({})
 
     def _wrap(self, data):
-        return self._doc_class(self.properties, **data)
+        return self._doc_class(**data)
 
     def empty(self):
         if self._multi:
             return AttrList([], self._wrap)
         return self._empty()
 
-    def __getitem__(self, name):
-        return self.properties[name]
-
-    def __contains__(self, name):
-        return name in self.properties
+    def to_dict(self):
+        d = self._mapping.to_dict()
+        _, d = d.popitem()
+        d["type"] = self.name
+        return d
 
     def _collect_fields(self):
-        " Iterate over all Field objects within, including multi fields. "
-        for f in itervalues(self.properties.to_dict()):
-            yield f
-            # multi fields
-            if hasattr(f, 'fields'):
-                for inner_f in itervalues(f.fields.to_dict()):
-                    yield inner_f
-            # nested and inner objects
-            if hasattr(f, '_collect_fields'):
-                for inner_f in f._collect_fields():
-                    yield inner_f
-
-    def update(self, other_object):
-        if not hasattr(other_object, 'properties'):
-            # not an inner/nested object, no merge possible
-            return
-
-        our, other = self.properties, other_object.properties
-        for name in other:
-            if name in our:
-                if hasattr(our[name], 'update'):
-                    our[name].update(other[name])
-                continue
-            our[name] = other[name]
+        return self._mapping.properties._collect_fields()
 
     def _deserialize(self, data):
         if data is None:
@@ -171,10 +151,6 @@ class InnerObject(object):
         if isinstance(data, self._doc_class):
             return data
 
-        if isinstance(data, (list, AttrList)):
-            data[:] = list(map(self._deserialize, data))
-            return data
-
         if isinstance(data, AttrDict):
             data = data._d_
 
@@ -183,10 +159,15 @@ class InnerObject(object):
     def _serialize(self, data):
         if data is None:
             return None
+
+        # somebody assigned raw dict to the field, we should tolerate that
+        if isinstance(data, collections.Mapping):
+            return data
+
         return data.to_dict()
 
     def clean(self, data):
-        data = super(InnerObject, self).clean(data)
+        data = super(Object, self).clean(data)
         if data is None:
             return None
         if isinstance(data, (list, AttrList)):
@@ -196,15 +177,17 @@ class InnerObject(object):
             data.full_clean()
         return data
 
+    def update(self, other):
+        if not isinstance(other, Object):
+            # not an inner/nested object, no merge possible
+            return
 
-class Object(InnerObject, Field):
-    name = 'object'
+        self._mapping.update(other._mapping)
 
-class Nested(InnerObject, Field):
+class Nested(Object):
     name = 'nested'
 
     def __init__(self, *args, **kwargs):
-        # change the default for Nested fields
         kwargs.setdefault('multi', True)
         super(Nested, self).__init__(*args, **kwargs)
 
diff --git a/elasticsearch_dsl/mapping.py b/elasticsearch_dsl/mapping.py
index 385416d..bdfa012 100644
--- a/elasticsearch_dsl/mapping.py
+++ b/elasticsearch_dsl/mapping.py
@@ -1,10 +1,10 @@
 import collections
 
-from six import iteritems
+from six import iteritems, itervalues
 from itertools import chain
 
 from .utils import DslBase
-from .field import InnerObject, Text
+from .field import Text, construct_field
 from .connections import connections
 from .exceptions import IllegalOperation
 from .index import Index
@@ -14,7 +14,8 @@ META_FIELDS = frozenset((
     'numeric_detection', 'dynamic_templates', 'enabled'
 ))
 
-class Properties(InnerObject, DslBase):
+class Properties(DslBase):
+    _param_defs = {'properties': {'type': 'field', 'hash': True}}
     def __init__(self, name):
         self._name = name
         super(Properties, self).__init__()
@@ -22,10 +23,46 @@ class Properties(InnerObject, DslBase):
     def __repr__(self):
         return 'Properties(%r)' % self._name
 
+    def __getitem__(self, name):
+        return self.properties[name]
+
+    def __contains__(self, name):
+        return name in self.properties
+
     @property
     def name(self):
         return self._name
 
+    def field(self, name, *args, **kwargs):
+        self.properties[name] = construct_field(*args, **kwargs)
+        return self
+
+    def _collect_fields(self):
+        " Iterate over all Field objects within, including multi fields. "
+        for f in itervalues(self.properties.to_dict()):
+            yield f
+            # multi fields
+            if hasattr(f, 'fields'):
+                for inner_f in itervalues(f.fields.to_dict()):
+                    yield inner_f
+            # nested and inner objects
+            if hasattr(f, '_collect_fields'):
+                for inner_f in f._collect_fields():
+                    yield inner_f
+
+    def update(self, other_object):
+        if not hasattr(other_object, 'properties'):
+            # not an inner/nested object, no merge possible
+            return
+
+        our, other = self.properties, other_object.properties
+        for name in other:
+            if name in our:
+                if hasattr(our[name], 'update'):
+                    our[name].update(other[name])
+                continue
+            our[name] = other[name]
+
 
 class Mapping(object):
     def __init__(self, name):
diff --git a/elasticsearch_dsl/response/__init__.py b/elasticsearch_dsl/response/__init__.py
index 6bbefa0..0041989 100644
--- a/elasticsearch_dsl/response/__init__.py
+++ b/elasticsearch_dsl/response/__init__.py
@@ -3,8 +3,9 @@ from ..utils import AttrDict, AttrList
 from .hit import Hit, HitMeta
 
 class Response(AttrDict):
-    def __init__(self, search, response):
+    def __init__(self, search, response, doc_class=None):
         super(AttrDict, self).__setattr__('_search', search)
+        super(AttrDict, self).__setattr__('_doc_class', doc_class)
         super(Response, self).__init__(response)
 
     def __iter__(self):
@@ -27,30 +28,23 @@ class Response(AttrDict):
         return len(self.hits)
 
     def __getstate__(self):
-        return (self._d_, self._search)
+        return (self._d_, self._search, self._doc_class)
 
     def __setstate__(self, state):
         super(AttrDict, self).__setattr__('_d_', state[0])
         super(AttrDict, self).__setattr__('_search', state[1])
+        super(AttrDict, self).__setattr__('_doc_class', state[2])
 
     def success(self):
         return self._shards.total == self._shards.successful and not self.timed_out
 
-    def _get_result(self, hit):
-        dt = hit.get('_type')
-        for t in hit.get('inner_hits', ()):
-            hit['inner_hits'][t] = Response(self._search, hit['inner_hits'][t])
-        callback = self._search._doc_type_map.get(dt, Hit)
-        callback = getattr(callback, 'from_es', callback)
-        return callback(hit)
-
     @property
     def hits(self):
         if not hasattr(self, '_hits'):
             h = self._d_['hits']
 
             try:
-                hits = AttrList(map(self._get_result, h['hits']))
+                hits = AttrList(map(self._search._get_result, h['hits']))
             except AttributeError as e:
                 # avoid raising AttributeError since it will be hidden by the property
                 raise TypeError("Could not parse hits.", e)
diff --git a/elasticsearch_dsl/search.py b/elasticsearch_dsl/search.py
index f18a4f3..94778df 100644
--- a/elasticsearch_dsl/search.py
+++ b/elasticsearch_dsl/search.py
@@ -92,13 +92,12 @@ class Request(object):
         self._doc_type = []
         self._doc_type_map = {}
         if isinstance(doc_type, (tuple, list)):
-            for dt in doc_type:
-                self._add_doc_type(dt)
+            self._doc_type.extend(doc_type)
         elif isinstance(doc_type, collections.Mapping):
             self._doc_type.extend(doc_type.keys())
             self._doc_type_map.update(doc_type)
         elif doc_type:
-            self._add_doc_type(doc_type)
+            self._doc_type.append(doc_type)
 
         self._params = {}
         self._extra = extra or {}
@@ -148,7 +147,7 @@ class Request(object):
         else:
             indexes = []
             for i in index:
-                if isinstance(i, str):
+                if isinstance(i, string_types):
                     indexes.append(i)
                 elif isinstance(i, list):
                     indexes += i
@@ -159,11 +158,52 @@ class Request(object):
 
         return s
 
-    def _add_doc_type(self, doc_type):
-        if hasattr(doc_type, '_doc_type'):
-            self._doc_type_map[doc_type._doc_type.name] = doc_type
-            doc_type = doc_type._doc_type.name
-        self._doc_type.append(doc_type)
+    def _get_doc_type(self):
+        """
+        Return a list of doc_type names to be used
+        """
+        return list(set(dt._doc_type.name if hasattr(dt, '_doc_type') else dt for dt in self._doc_type))
+
+    def _resolve_nested(self, field, parent_class=None):
+        doc_class = Hit
+        if hasattr(parent_class, '_doc_type'):
+            nested_field = parent_class._doc_type.resolve_field(field)
+
+        else:
+            for dt in self._doc_type:
+                if not hasattr(dt, '_doc_type'):
+                    continue
+                nested_field = dt._doc_type.resolve_field(field)
+                if nested_field is not None:
+                    break
+
+        if nested_field is not None:
+            return nested_field._doc_class
+
+        return doc_class
+
+    def _get_result(self, hit, parent_class=None):
+        doc_class = Hit
+        dt = hit.get('_type')
+
+        if '_nested' in hit:
+            doc_class = self._resolve_nested(hit['_nested']['field'], parent_class)
+
+        elif dt in self._doc_type_map:
+            doc_class = self._doc_type_map[dt]
+
+        else:
+            for doc_type in self._doc_type:
+                if hasattr(doc_type, '_doc_type') and doc_type._doc_type.matches(hit):
+                    doc_class = doc_type
+                    break
+
+        for t in hit.get('inner_hits', ()):
+            hit['inner_hits'][t] = Response(self, hit['inner_hits'][t], doc_class=doc_class)
+
+        callback = getattr(doc_class, 'from_es', doc_class)
+        return callback(hit)
+
 
     def doc_type(self, *doc_type, **kwargs):
         """
@@ -186,8 +226,7 @@ class Request(object):
             s._doc_type = []
             s._doc_type_map = {}
         else:
-            for dt in doc_type:
-                s._add_doc_type(dt)
+            s._doc_type.extend(doc_type)
             s._doc_type.extend(kwargs.keys())
             s._doc_type_map.update(kwargs)
         return s
@@ -528,6 +567,7 @@ class Search(Request):
             Search().highlight('title', fragment_size=50).highlight('body', fragment_size=100)
 
         which will produce::
+
             {
                 "highlight": {
                     "fields": {
@@ -615,7 +655,7 @@ class Search(Request):
         # TODO: failed shards detection
         return es.count(
             index=self._index,
-            doc_type=self._doc_type,
+            doc_type=self._get_doc_type(),
             body=d,
             **self._params
         )['count']
@@ -634,7 +674,7 @@ class Search(Request):
                 self,
                 es.search(
                     index=self._index,
-                    doc_type=self._doc_type,
+                    doc_type=self._get_doc_type(),
                     body=self.to_dict(),
                     **self._params
                 )
@@ -657,12 +697,10 @@ class Search(Request):
                 es,
                 query=self.to_dict(),
                 index=self._index,
-                doc_type=self._doc_type,
+                doc_type=self._get_doc_type(),
                 **self._params
         ):
-            callback = self._doc_type_map.get(hit['_type'], Hit)
-            callback = getattr(callback, 'from_es', callback)
-            yield callback(hit)
+            yield self._get_result(hit)
 
     def delete(self):
         """
@@ -675,7 +713,7 @@ class Search(Request):
             es.delete_by_query(
                 index=self._index,
                 body=self.to_dict(),
-                doc_type=self._doc_type,
+                doc_type=self._get_doc_type(),
                 **self._params
             )
         )
@@ -720,7 +758,7 @@ class MultiSearch(Request):
             if s._index:
                 meta['index'] = s._index
             if s._doc_type:
-                meta['type'] = s._doc_type
+                meta['type'] = s._get_doc_type()
             meta.update(s._params)
 
             out.append(meta)
@@ -737,7 +775,7 @@ class MultiSearch(Request):
 
             responses = es.msearch(
                 index=self._index,
-                doc_type=self._doc_type,
+                doc_type=self._get_doc_type(),
                 body=self.to_dict(),
                 **self._params
             )
diff --git a/elasticsearch_dsl/utils.py b/elasticsearch_dsl/utils.py
index 192111e..1397edb 100644
--- a/elasticsearch_dsl/utils.py
+++ b/elasticsearch_dsl/utils.py
@@ -10,6 +10,15 @@ from .exceptions import UnknownDslObject, ValidationException
 SKIP_VALUES = ('', None)
 EXPAND__TO_DOT=True
 
+DOC_META_FIELDS = frozenset((
+    'id', 'routing', 'version', 'version_type'
+))
+
+META_FIELDS = frozenset((
+    # Elasticsearch metadata fields, except 'type'
+    'index', 'using', 'score',
+)).union(DOC_META_FIELDS)
+
 def _wrap(val, obj_wrapper=None):
     if isinstance(val, collections.Mapping):
         return AttrDict(val) if obj_wrapper is None else obj_wrapper(val)
@@ -326,12 +335,25 @@ class DslBase(object):
 
 class ObjectBase(AttrDict):
     def __init__(self, **kwargs):
-        m = self._doc_type.mapping
-        for k in m:
-            if k in kwargs and m[k]._coerce:
-                kwargs[k] = m[k].deserialize(kwargs[k])
         super(ObjectBase, self).__init__(kwargs)
 
+    @classmethod
+    def from_es(cls, hit):
+        meta = hit.copy()
+        doc = meta.pop('_source', {})
+        if 'fields' in meta:
+            for k, v in iteritems(meta.pop('fields')):
+                if k.startswith('_') and k[1:] in META_FIELDS:
+                    meta[k] = v
+                else:
+                    doc[k] = v
+
+        m = cls._doc_type.mapping
+        for k in m:
+            if k in doc and m[k]._coerce:
+                doc[k] = m[k].deserialize(doc[k])
+        return cls(meta=meta, **doc)
+
     def __getattr__(self, name):
         try:
             return super(ObjectBase, self).__getattr__(name)
@@ -346,11 +368,6 @@ class ObjectBase(AttrDict):
                     return value
             raise
 
-    def __setattr__(self, name, value):
-        if name in self._doc_type.mapping:
-            value = self._doc_type.mapping[name].deserialize(value)
-        super(ObjectBase, self).__setattr__(name, value)
-
     def to_dict(self):
         out = {}
         for k, v in iteritems(self._d_):
diff --git a/setup.py b/setup.py
index c1838ef..fb20ea8 100644
--- a/setup.py
+++ b/setup.py
@@ -3,7 +3,7 @@ import sys
 from os.path import join, dirname
 from setuptools import setup, find_packages
 
-VERSION = (5, 4, 0, 'dev')
+VERSION = (6, 0, 0, 'dev')
 __version__ = VERSION
 __versionstr__ = '.'.join(map(str, VERSION))
 


Test patch:
diff --git a/test_elasticsearch_dsl/conftest.py b/test_elasticsearch_dsl/conftest.py
index 754e41e..0bb3af7 100644
--- a/test_elasticsearch_dsl/conftest.py
+++ b/test_elasticsearch_dsl/conftest.py
@@ -13,7 +13,7 @@ from .test_integration.test_data import DATA, FLAT_DATA, create_git_index, \
 
 
 @fixture(scope='session')
-def client(request):
+def client():
     try:
         connection = get_test_client(nowait='WAIT_FOR_ES' not in os.environ)
         connections.add_connection('default', connection)
@@ -22,13 +22,13 @@ def client(request):
         skip()
 
 @fixture
-def write_client(request, client):
+def write_client(client):
     yield client
     client.indices.delete('test-*', ignore=404)
     client.indices.delete_template('test-template', ignore=404)
 
 @fixture
-def mock_client(request):
+def mock_client():
     client = Mock()
     client.search.return_value = dummy_response()
     connections.add_connection('mock', client)
@@ -37,7 +37,7 @@ def mock_client(request):
     connections._kwargs = {}
 
 @fixture(scope='session')
-def data_client(request, client):
+def data_client(client):
     # create mappings
     create_git_index(client, 'git')
     create_flat_git_index(client, 'flat-git')
@@ -74,7 +74,7 @@ def dummy_response():
             "_type": "employee",
             "_id": "42",
             "_score": 11.123,
-            "_parent": "elasticsearch",
+            "_routing": "elasticsearch",
 
             "_source": {
               "name": {
@@ -90,7 +90,7 @@ def dummy_response():
             "_type": "employee",
             "_id": "47",
             "_score": 1,
-            "_parent": "elasticsearch",
+            "_routing": "elasticsearch",
 
             "_source": {
               "name": {
@@ -106,7 +106,7 @@ def dummy_response():
             "_type": "employee",
             "_id": "53",
             "_score": 16.0,
-            "_parent": "elasticsearch",
+            "_routing": "elasticsearch",
           },
         ],
         "max_score": 12.0,
@@ -119,7 +119,7 @@ def dummy_response():
 @fixture
 def aggs_search():
     from elasticsearch_dsl import Search
-    s = Search(index='git', doc_type='commits')
+    s = Search(index='flat-git')
     s.aggs\
         .bucket('popular_files', 'terms', field='files', size=2)\
         .metric('line_stats', 'stats', field='stats.lines')\
@@ -156,27 +156,23 @@ def aggs_data():
                                 'hits': [
                                     {
                                         '_id': '3ca6e1e73a071a705b4babd2f581c91a2a3e5037',
-                                        '_type': 'commits',
+                                        '_type': 'doc',
                                         '_source': {
                                             'stats': {'files': 4, 'deletions': 7, 'lines': 30, 'insertions': 23},
                                             'committed_date': '2014-05-02T13:47:19'
                                         },
                                         '_score': 1.0,
-                                        '_parent': 'elasticsearch-dsl-py',
-                                        '_routing': 'elasticsearch-dsl-py',
-                                        '_index': 'git'
+                                        '_index': 'flat-git'
                                     },
                                     {
                                         '_id': 'eb3e543323f189fd7b698e66295427204fff5755',
-                                        '_type': 'commits',
+                                        '_type': 'doc',
                                         '_source': {
                                             'stats': {'files': 1, 'deletions': 0, 'lines': 18, 'insertions': 18},
                                             'committed_date': '2014-05-01T13:32:14'
                                         },
                                         '_score': 1.0,
-                                        '_parent': 'elasticsearch-dsl-py',
-                                        '_routing': 'elasticsearch-dsl-py',
-                                        '_index': 'git'
+                                        '_index': 'flat-git'
                                     }
                                 ],
                                 'max_score': 1.0
@@ -193,26 +189,22 @@ def aggs_data():
                                 'hits': [
                                     {
                                         '_id': '3ca6e1e73a071a705b4babd2f581c91a2a3e5037',
-                                        '_type': 'commits',
+                                        '_type': 'doc',
                                         '_source': {
                                             'stats': {'files': 4, 'deletions': 7, 'lines': 30, 'insertions': 23},
                                             'committed_date': '2014-05-02T13:47:19'
                                         },
                                         '_score': 1.0,
-                                        '_parent': 'elasticsearch-dsl-py',
-                                        '_routing': 'elasticsearch-dsl-py',
-                                        '_index': 'git'
+                                        '_index': 'flat-git'
                                     }, {
                                         '_id': 'dd15b6ba17dd9ba16363a51f85b31f66f1fb1157',
-                                        '_type': 'commits',
+                                        '_type': 'doc',
                                         '_source': {
                                             'stats': {'files': 3, 'deletions': 18, 'lines': 62, 'insertions': 44},
                                             'committed_date': '2014-05-01T13:30:44'
                                         },
                                         '_score': 1.0,
-                                        '_parent': 'elasticsearch-dsl-py',
-                                        '_routing': 'elasticsearch-dsl-py',
-                                        '_index': 'git'
+                                        '_index': 'flat-git'
                                     }
                                 ],
                                 'max_score': 1.0
diff --git a/test_elasticsearch_dsl/test_document.py b/test_elasticsearch_dsl/test_document.py
index d65a674..1d31457 100644
--- a/test_elasticsearch_dsl/test_document.py
+++ b/test_elasticsearch_dsl/test_document.py
@@ -3,19 +3,19 @@ import codecs
 from hashlib import md5
 from datetime import datetime
 
-from elasticsearch_dsl import document, field, Mapping, utils
+from elasticsearch_dsl import document, field, Mapping, utils, InnerDoc
 from elasticsearch_dsl.exceptions import ValidationException, IllegalOperation
 
 from pytest import raises
 
-class MyInner(field.InnerObjectWrapper):
-    pass
+class MyInner(InnerDoc):
+    old_field = field.Text()
 
 class MyDoc(document.DocType):
     title = field.Keyword()
     name = field.Text()
     created_at = field.Date()
-    inner = field.Object(properties={'old_field': field.Text()}, doc_class=MyInner)
+    inner = field.Object(MyInner)
 
 class MySubDoc(MyDoc):
     name = field.Keyword()
@@ -30,13 +30,12 @@ class MyDoc2(document.DocType):
 class MyMultiSubDoc(MyDoc2, MySubDoc):
     pass
 
+class Comment(document.InnerDoc):
+    title = field.Text()
+    tags = field.Keyword(multi=True)
+
 class DocWithNested(document.DocType):
-    comments = field.Nested(
-        properties={
-            'title': field.Text(),
-            'tags': field.Keyword(multi=True)
-        }
-    )
+    comments = field.Nested(Comment)
 
 class SimpleCommit(document.DocType):
     files = field.Text(multi=True)
@@ -61,11 +60,47 @@ class SecretDoc(document.DocType):
     title = SecretField(index='no')
 
 class NestedSecret(document.DocType):
-    secrets = field.Nested(properties={'title': SecretField()})
+    secrets = field.Nested(SecretDoc)
 
 class OptionalObjectWithRequiredField(document.DocType):
     comments = field.Nested(properties={'title': field.Keyword(required=True)})
 
+def test_matches_uses_index_name_and_doc_type():
+    assert SimpleCommit._doc_type.matches({
+        '_type': 'doc',
+        '_index': 'test-git'
+    })
+    assert not SimpleCommit._doc_type.matches({
+        '_type': 'doc',
+        '_index': 'not-test-git'
+    })
+    assert MySubDoc._doc_type.matches({
+        '_type': 'my_custom_doc',
+        '_index': 'default-index'
+    })
+    assert not MySubDoc._doc_type.matches({
+        '_type': 'doc',
+        '_index': 'default-index'
+    })
+    assert not MySubDoc._doc_type.matches({
+        '_type': 'my_custom_doc',
+        '_index': 'test-git'
+    })
+
+def test_matches_accepts_wildcards():
+    class MyDoc(document.DocType):
+        class Meta:
+            index = 'my-*'
+
+    assert MyDoc._doc_type.matches({
+        '_type': 'doc',
+        '_index': 'my-index'
+    })
+    assert not MyDoc._doc_type.matches({
+        '_type': 'doc',
+        '_index': 'not-my-index'
+    })
+
 def test_assigning_attrlist_to_field():
     sc = SimpleCommit()
     l = ['README', 'README.rst']
@@ -84,13 +119,13 @@ def test_custom_field():
     assert {'title': 'Uryyb'} == s.to_dict()
     assert s.title == 'Hello'
 
-    s.title = 'Uryyb'
+    s = SecretDoc.from_es({'_source': {'title': 'Uryyb'}})
     assert s.title == 'Hello'
     assert isinstance(s.title, Secret)
 
 def test_custom_field_mapping():
     assert {
-        'secret_doc': {
+        'doc': {
             'properties': {
                 'title': {'index': 'no', 'type': 'text'}
             }
@@ -113,7 +148,7 @@ def test_multi_works_after_doc_has_been_saved():
 
 def test_multi_works_in_nested_after_doc_has_been_serialized():
     # Issue #359
-    c = DocWithNested(comments=[{'title': 'First!'}])
+    c = DocWithNested(comments=[Comment(title='First!')])
 
     assert [] == c.comments[0].tags
     assert {'comments': [{'title': 'First!'}]} == c.to_dict()
@@ -160,7 +195,7 @@ def test_attribute_can_be_removed():
     assert 'title' not in d._d_
 
 def test_doc_type_can_be_correctly_pickled():
-    d = DocWithNested(title='Hello World!', comments=[{'title': 'hellp'}], meta={'id': 42})
+    d = DocWithNested(title='Hello World!', comments=[Comment(title='hellp')], meta={'id': 42})
     s = pickle.dumps(d)
 
     d2 = pickle.loads(s)
@@ -187,7 +222,7 @@ def test_meta_field_mapping():
             dynamic_templates = document.MetaField([42])
 
     assert {
-        'user': {
+        'doc': {
             'properties': {
                 'username': {'type': 'text'}
             },
@@ -236,12 +271,14 @@ def test_docs_with_properties():
         u.password
 
 def test_nested_can_be_assigned_to():
-    d1 = DocWithNested(comments=[{'title': 'First!'}])
+    d1 = DocWithNested(comments=[Comment(title='First!')])
     d2 = DocWithNested()
 
     d2.comments = d1.comments
+    assert isinstance(d1.comments[0], Comment)
     assert d2.comments == [{'title': 'First!'}]
     assert {'comments': [{'title': 'First!'}]} == d2.to_dict()
+    assert isinstance(d2.comments[0], Comment)
 
 def test_nested_can_be_none():
     d = DocWithNested(comments=None, title='Hello World!')
@@ -258,7 +295,7 @@ def test_nested_defaults_to_list_and_can_be_updated():
 
 def test_to_dict_is_recursive_and_can_cope_with_multi_values():
     md = MyDoc(name=['a', 'b', 'c'])
-    md.inner = [{'old_field': 'of1'}, {'old_field': 'of2'}]
+    md.inner = [MyInner(old_field='of1'), MyInner(old_field='of2')]
 
     assert isinstance(md.inner[0], MyInner)
 
@@ -276,9 +313,8 @@ def test_to_dict_ignores_empty_collections():
 def test_declarative_mapping_definition():
     assert issubclass(MyDoc, document.DocType)
     assert hasattr(MyDoc, '_doc_type')
-    assert 'my_doc' == MyDoc._doc_type.name
     assert {
-        'my_doc': {
+        'doc': {
             'properties': {
                 'created_at': {'type': 'date'},
                 'name': {'type': 'text'},
@@ -331,8 +367,9 @@ def test_document_can_be_created_dynamically():
 
 def test_invalid_date_will_raise_exception():
     md = MyDoc()
+    md.created_at = 'not-a-date'
     with raises(ValidationException):
-        md.created_at = 'not-a-date'
+        md.full_clean()
 
 def test_document_inheritance():
     assert issubclass(MySubDoc, MyDoc)
@@ -367,13 +404,11 @@ def test_meta_inheritance():
     assert issubclass(MyMultiSubDoc, MyDoc2)
     assert issubclass(MyMultiSubDoc, document.DocType)
     assert hasattr(MyMultiSubDoc, '_doc_type')
-    # doc_type should not be inherited
-    assert 'my_multi_sub_doc' == MyMultiSubDoc._doc_type.name
     # index and using should be
     assert MyMultiSubDoc._doc_type.index == MySubDoc._doc_type.index
     assert MyMultiSubDoc._doc_type.using == MySubDoc._doc_type.using
     assert {
-        'my_multi_sub_doc': {
+        'doc': {
             'properties': {
                 'created_at': {'type': 'date'},
                 'name': {'type': 'keyword'},
diff --git a/test_elasticsearch_dsl/test_field.py b/test_elasticsearch_dsl/test_field.py
index 61466a8..ee74297 100644
--- a/test_elasticsearch_dsl/test_field.py
+++ b/test_elasticsearch_dsl/test_field.py
@@ -64,20 +64,8 @@ def test_multi_fields_are_accepted_and_parsed():
         }
     } == f.to_dict()
 
-def test_modifying_nested():
-    f = field.Nested()
-    f.field('name', 'text', index='not_analyzed')
-
-    assert {
-        'type': 'nested',
-        'properties': {
-            'name': {'type': 'text', 'index': 'not_analyzed'}
-        },
-    } == f.to_dict()
-
 def test_nested_provides_direct_access_to_its_fields():
-    f = field.Nested()
-    f.field('name', 'text', index='not_analyzed')
+    f = field.Nested(properties={'name': {'type': 'text', 'index': 'not_analyzed'}})
 
     assert 'name' in f
     assert f['name'] == field.Text(index='not_analyzed')
diff --git a/test_elasticsearch_dsl/test_index.py b/test_elasticsearch_dsl/test_index.py
index 17aa6b7..9d0269d 100644
--- a/test_elasticsearch_dsl/test_index.py
+++ b/test_elasticsearch_dsl/test_index.py
@@ -64,7 +64,7 @@ def test_registered_doc_type_included_in_to_dict():
     assert Post._doc_type.index == 'i'
     assert {
         'mappings': {
-            'post': {
+            'doc': {
                 'properties': {
                     'title': {'type': 'text'},
                     'published_from': {'type': 'date'},
@@ -79,7 +79,7 @@ def test_registered_doc_type_included_in_search():
 
     s = i.search()
 
-    assert s._doc_type_map == {'post': Post}
+    assert s._doc_type == [Post]
 
 
 def test_aliases_add_to_object():
diff --git a/test_elasticsearch_dsl/test_integration/test_document.py b/test_elasticsearch_dsl/test_integration/test_document.py
index f87e259..aceaa81 100644
--- a/test_elasticsearch_dsl/test_integration/test_document.py
+++ b/test_elasticsearch_dsl/test_integration/test_document.py
@@ -3,22 +3,23 @@ from pytz import timezone
 
 from elasticsearch import ConflictError, NotFoundError, RequestError
 
-from elasticsearch_dsl import DocType, Date, Text, Keyword, construct_field, Mapping
+from elasticsearch_dsl import DocType, Date, Text, Keyword, Mapping, InnerDoc, \
+    Object, Nested, MetaField, Q
 from elasticsearch_dsl.utils import AttrList
 
-from pytest import raises
+from pytest import raises, fixture
 
-user_field = construct_field('object')
-user_field.field('name', 'text', fields={'raw': construct_field('keyword')})
+class User(InnerDoc):
+    name = Text(fields={'raw': Keyword()})
 
 class Wiki(DocType):
-    owner = user_field
+    owner = Object(User)
 
     class Meta:
         index = 'test-wiki'
 
 class Repository(DocType):
-    owner = user_field
+    owner = Object(User)
     created_at = Date()
     description = Text(analyzer='snowball')
     tags = Keyword()
@@ -41,9 +42,49 @@ class Commit(DocType):
         doc_type = 'doc'
         mapping = Mapping('doc')
 
+class Comment(InnerDoc):
+    content = Text()
+    author = Object(User)
+    class Meta:
+        dynamic = MetaField(False)
+
+class PullRequest(DocType):
+    comments = Nested(Comment)
+    class Meta:
+        index = 'test-prs'
+
+@fixture
+def pull_request(write_client):
+    PullRequest.init()
+    pr = PullRequest(_id=42, comments=[Comment(content='Hello World!', author=User(name='honzakral'))])
+    pr.save(refresh=True)
+    return pr
+
+def test_nested_inner_hits_are_wrapped_properly(pull_request):
+    s = PullRequest.search().query('nested', inner_hits={}, path='comments',
+                                   query=Q('match', comments__content='hello'))
+
+    response = s.execute()
+    pr = response.hits[0]
+    assert isinstance(pr, PullRequest)
+    assert isinstance(pr.comments[0], Comment)
+
+    comment = pr.meta.inner_hits.comments.hits[0]
+    assert isinstance(comment, Comment)
+
+def test_nested_top_hits_are_wrapped_properly(pull_request):
+    s = PullRequest.search()
+    s.aggs.bucket('comments', 'nested', path='comments').metric('hits', 'top_hits', size=1)
+
+    r = s.execute()
+
+    print(r._d_)
+    assert isinstance(r.aggregations.comments.hits.hits[0], Comment)
+
+
 def test_update_object_field(write_client):
     Wiki.init()
-    w = Wiki(owner={'name': 'Honza Kral'}, _id='elasticsearch-py')
+    w = Wiki(owner=User(name='Honza Kral'), _id='elasticsearch-py')
     w.save()
 
     w.update(owner=[{'name': 'Honza'}, {'name': 'Nick'}])
diff --git a/test_elasticsearch_dsl/test_integration/test_index.py b/test_elasticsearch_dsl/test_integration/test_index.py
index 88c7550..2a2eb53 100644
--- a/test_elasticsearch_dsl/test_integration/test_index.py
+++ b/test_elasticsearch_dsl/test_integration/test_index.py
@@ -16,7 +16,7 @@ def test_index_template_works(write_client):
     assert {
         'test-blog': {
             'mappings': {
-                'post': {
+                'doc': {
                     'properties': {
                         'title': {'type': 'text', 'analyzer': 'my_analyzer'},
                         'published_from': {'type': 'date'},
@@ -40,7 +40,7 @@ def test_index_can_be_created_with_settings_and_mappings(write_client):
     assert {
         'test-blog': {
             'mappings': {
-                'post': {
+                'doc': {
                     'properties': {
                         'title': {'type': 'text', 'analyzer': 'my_analyzer'},
                         'published_from': {'type': 'date'}
diff --git a/test_elasticsearch_dsl/test_integration/test_search.py b/test_elasticsearch_dsl/test_integration/test_search.py
index 78435fa..f42342d 100644
--- a/test_elasticsearch_dsl/test_integration/test_search.py
+++ b/test_elasticsearch_dsl/test_integration/test_search.py
@@ -21,11 +21,9 @@ class Repository(DocType):
 
     class Meta:
         index = 'git'
-        doc_type = 'doc'
 
 class Commit(DocType):
     class Meta:
-        doc_type = 'doc'
         index = 'flat-git'
 
 def test_filters_aggregation_buckets_are_accessible(data_client):
diff --git a/test_elasticsearch_dsl/test_mapping.py b/test_elasticsearch_dsl/test_mapping.py
index 367005f..e1dcacc 100644
--- a/test_elasticsearch_dsl/test_mapping.py
+++ b/test_elasticsearch_dsl/test_mapping.py
@@ -20,17 +20,16 @@ def test_mapping_update_is_recursive():
     m1 = mapping.Mapping('article')
     m1.field('title', 'text')
     m1.field('author', 'object')
-    m1['author'].field('name', 'text')
+    m1.field('author', 'object', properties={'name': {'type': 'text'}})
     m1.meta('_all', enabled=False)
     m1.meta('dynamic', False)
 
     m2 = mapping.Mapping('article')
     m2.field('published_from', 'date')
-    m2.field('author', 'object')
+    m2.field('author', 'object', properties={'email': {'type': 'text'}})
     m2.field('title', 'text')
     m2.field('lang', 'keyword')
     m2.meta('_analyzer', path='lang')
-    m2['author'].field('email', 'text')
 
     m1.update(m2, update_only=True)
 
diff --git a/test_elasticsearch_dsl/test_result.py b/test_elasticsearch_dsl/test_result.py
index e7b36ed..bb11f65 100644
--- a/test_elasticsearch_dsl/test_result.py
+++ b/test_elasticsearch_dsl/test_result.py
@@ -90,7 +90,7 @@ def test_iterating_over_response_gives_you_hits(dummy_response):
     assert 'elasticsearch' == h.meta.id
     assert 12 == h.meta.score
 
-    assert hits[1].meta.parent == 'elasticsearch'
+    assert hits[1].meta.routing == 'elasticsearch'
 
 def test_hits_get_wrapped_to_contain_additional_attrs(dummy_response):
     res = response.Response(Search(), dummy_response)
diff --git a/test_elasticsearch_dsl/test_search.py b/test_elasticsearch_dsl/test_search.py
index 122bc7d..f8f1caf 100644
--- a/test_elasticsearch_dsl/test_search.py
+++ b/test_elasticsearch_dsl/test_search.py
@@ -162,6 +162,8 @@ def test_search_index():
     assert s._index == ['i']
     s = s.index('i2')
     assert s._index == ['i', 'i2']
+    s = s.index(u'i3')
+    assert s._index == ['i', 'i2', 'i3']
     s = s.index()
     assert s._index is None
     s = search.Search(index=('i', 'i2'))
@@ -209,13 +211,14 @@ def test_doc_type_can_be_document_class():
         pass
 
     s = search.Search(doc_type=MyDocType)
-    assert s._doc_type == ['my_doc_type']
-    assert s._doc_type_map == {'my_doc_type': MyDocType}
+    assert s._doc_type == [MyDocType]
+    assert s._doc_type_map == {}
+    assert s._get_doc_type() == ['doc']
 
     s = search.Search().doc_type(MyDocType)
-    assert s._doc_type == ['my_doc_type']
-    assert s._doc_type_map == {'my_doc_type': MyDocType}
-
+    assert s._doc_type == [MyDocType]
+    assert s._doc_type_map == {}
+    assert s._get_doc_type() == ['doc']
 
 def test_sort():
     s = search.Search()
diff --git a/test_elasticsearch_dsl/test_validation.py b/test_elasticsearch_dsl/test_validation.py
index c84d671..f30c194 100644
--- a/test_elasticsearch_dsl/test_validation.py
+++ b/test_elasticsearch_dsl/test_validation.py
@@ -1,25 +1,21 @@
 from datetime import datetime
 
-from elasticsearch_dsl import DocType, Nested, Text, Date, Object, Boolean, Integer
-from elasticsearch_dsl.field import InnerObjectWrapper
+from elasticsearch_dsl import DocType, Nested, Text, Date, Object, Boolean, Integer, InnerDoc
 from elasticsearch_dsl.exceptions import ValidationException
 
 from pytest import raises
 
-class Author(InnerObjectWrapper):
+class Author(InnerDoc):
+    name = Text(required=True)
+    email = Text(required=True)
+
     def clean(self):
+        print(self, type(self), self.name)
         if self.name.lower() not in self.email:
             raise ValidationException('Invalid email!')
 
 class BlogPost(DocType):
-    authors = Nested(
-        required=True,
-        doc_class=Author,
-        properties={
-            'name': Text(required=True),
-            'email': Text(required=True)
-        }
-    )
+    authors = Nested(Author, required=True)
     created = Date()
     inner = Object()
 
@@ -57,8 +53,9 @@ def test_validation_works_for_lists_of_values():
     class DT(DocType):
         i = Date(required=True)
 
+    dt = DT(i=[datetime.now(), 'not date'])
     with raises(ValidationException):
-        DT(i=[datetime.now(), 'not date'])
+        dt.full_clean()
 
     dt = DT(i=[datetime.now(), datetime.now()])
     assert None is dt.full_clean()
@@ -101,7 +98,7 @@ def test_boolean_doesnt_treat_false_as_empty():
 
 
 def test_custom_validation_on_nested_gets_run():
-    d = BlogPost(authors=[{'name': 'Honza', 'email': 'king@example.com'}], created=None)
+    d = BlogPost(authors=[Author(name='Honza', email='king@example.com')], created=None)
 
     assert isinstance(d.authors[0], Author)