Repository: web2py/web2py

Issue:
AttributeError: 'DAL' object has no attribute
Hello,

web2py: Version 2.14.3-stable+timestamp.2016.03.26.23.02.02
dal: 16.03

I've just updated to the last version and received the bug:
(previous version worked fine)

Table - page
Field('name', notnull=True),
Field('parent', 'integer', default=0, requires=IS_EMPTY_OR(IS_IN_DB(self.db, 'page.id', '%(name)s')),

AttributeError: 'DAL' object has no attribute 'page'



Discussion:
Shouldn't parent be "reference page", not integer, and not have a default value at all?  

Anyway it does seem like a DAL bug since page is indeed defined.

same here!
upgrading from 2.12.3 to 2.14.3 broke compatibility with my application. 
Tried all versions so found out that last good versions was 2.13.4 and first broken is 2.14.1

My exact error:
AttributeError: 'DAL' object has no attribute 'trunks'

my relevant db.py sections:

``` python
db  = DAL('mysql://user:password@192.168.1.111/dbname',  pool_size=myconf.take('db.pool_size', cast=int), check_reserved=['mysql'],lazy_tables = True,
db.define_table('sedi',
                Field('name',            length=100, requires=IS_NOT_EMPTY()),
                Field('dominio',         'reference domini', writable=False, readable=False, default=lambda:session.domain ),
                Field('dipartimento',    'reference dipartimenti'),
                Field('trunk',           'reference trunks', ondelete='SET NULL', requires=IS_EMPTY_OR(IS_IN_DB(db, 'trunks.id','%(id)s - %(name)s'))),
                format='%(name)s',
                common_filter = lambda query: db.sedi.dominio==session.domain,
                )
db.define_table('trunks',
                Field('dominio',            'reference domini', writable=False, readable=False, default=lambda:session.domain ),
                Field('name',               length = 100, requires=IS_NOT_EMPTY()),
                Field('enabled',            'boolean', default = False),
                format='%(name)s',
                common_filter = lambda query: db.trunks.dominio == session.domain,
                )
```

Tried to isolate the problematic file.
Seems to be `gluon/validators.py`
Butsyk, try to downgrade just that file with the  2.13.4  version  one. 

trunks should maybe come before sedi??

On Tue, Apr 12, 2016 at 9:04 AM, Marvi notifications@github.com wrote:

> same here!
> upgrading from 2.12.3 to 2.14.3 broke compatibility with my application.
> Tried all versions so found out that last good versions was 2.13.4 and
> first broken is 2.14.1
> 
> My exact error:
> AttributeError: 'DAL' object has no attribute 'trunks'
> 
> my relevant db.py sections:
> 
> db  = DAL('mysql://user:password@192.168.1.111/dbname',  pool_size=myconf.take('db.pool_size', cast=int), check_reserved=['mysql'],lazy_tables = True,
> db.define_table('sedi',
>                 Field('name',            length=100, requires=IS_NOT_EMPTY()),
>                 Field('dominio',         'reference domini', writable=False, readable=False, default=lambda:session.domain ),
>                 Field('dipartimento',    'reference dipartimenti'),
>                 Field('trunk',           'reference trunks', ondelete='SET NULL', requires=IS_EMPTY_OR(IS_IN_DB(db, 'trunks.id','%(id)s - %(name)s'))),
>                 format='%(name)s',
>                 common_filter = lambda query: db.sedi.dominio==session.domain,
>                 )
> db.define_table('trunks',
>                 Field('dominio',            'reference domini', writable=False, readable=False, default=lambda:session.domain ),
>                 Field('name',               length = 100, requires=IS_NOT_EMPTY()),
>                 Field('enabled',            'boolean', default = False),
>                 format='%(name)s',
>                 common_filter = lambda query: db.trunks.dominio == session.domain,
>                 )
> 
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly or view it on GitHub
> https://github.com/web2py/web2py/issues/1266#issuecomment-208894775

In this simple case would resolve the problem. But the complete application has many other tables and relations in a way that at least one relation must come before the table that refers to. 
With the parameter `lazy_tables = True` would not be an issue, until version 2.14.1....

Yes apparently IS_IN_DB stopped being lazy when evaluating the field. I'm looking at it, this issue is very easy to reproduce. Just put this pretty commonly used table in your db.py

```
db.define_table('category',
    Field('parent_id', 'reference category', requires=IS_EMPTY_OR(IS_IN_DB(db, 'category.id', '%(name)s'))),
    Field('name')
)
```

I think it  can come from 2.13.4, I remember having noticed something
strange when try using a old app removing migrate=False... It may come from
earlier version since I use to set migrate=False since a long time...

On Wed, Apr 13, 2016 at 7:41 PM, Leonel Câmara notifications@github.com
wrote:

> Yes apparently IS_IN_DB stopped being lazy when evaluating the field. I'm
> looking at it, this issue is very easy to reproduce. Just put this pretty
> commonly used table in your db.py
> 
> db.define_table('category',
>     Field('parent_id', 'reference category', requires=IS_EMPTY_OR(IS_IN_DB(db, 'category.id', '%(name)s'))),
>     Field('name')
> )
> 
> —
> You are receiving this because you commented.
> Reply to this email directly or view it on GitHub
> https://github.com/web2py/web2py/issues/1266#issuecomment-209691850

I have a fix for it, let me just add a test for it to make sure it doesn't happen again.


Resolving patch:
diff --git a/gluon/validators.py b/gluon/validators.py
--- a/gluon/validators.py
+++ b/gluon/validators.py
@@ -376,8 +376,8 @@ def __init__(self, error_message='Invalid json', native_json=False):
     def __call__(self, value):
         try:
             if self.native_json:
-                simplejson.loads(value) # raises error in case of malformed json
-                return (value, None) #  the serialized value is not passed
+                simplejson.loads(value)  # raises error in case of malformed json
+                return (value, None)  # the serialized value is not passed
             else:
                 return (simplejson.loads(value), None)
         except JSONErrors:
@@ -459,7 +459,7 @@ def options(self, zero=True):
 
     def __call__(self, value):
         if self.multiple:
-            ### if below was values = re.compile("[\w\-:]+").findall(str(value))
+            # if below was values = re.compile("[\w\-:]+").findall(str(value))
             if not value:
                 values = []
             elif isinstance(value, (tuple, list)):
@@ -523,8 +523,8 @@ def __init__(
             field = field._id
         elif isinstance(field, str):
             items = field.split('.')
-            if len(items)==1: items+=['id']
-            field = self.dbset.db[items[0]][items[1]]
+            if len(items) == 1:
+                field = items[0] + '.id'
 
         (ktable, kfield) = str(field).split('.')
         if not label:
@@ -534,16 +534,16 @@ def __init__(
                 label = '%%(%s)s' % str(label).split('.')[-1]
             fieldnames = regex2.findall(label)
             if kfield not in fieldnames:
-                fieldnames.append(kfield) # kfield must be last
+                fieldnames.append(kfield)  # kfield must be last
         elif isinstance(label, Field):
-            fieldnames = [label.name, kfield] # kfield must be last
+            fieldnames = [label.name, kfield]  # kfield must be last
             label = '%%(%s)s' % label.name
         elif callable(label):
             fieldnames = '*'
         else:
             raise NotImplementedError
-        self.field = field # the lookup field
-        self.fieldnames = fieldnames # fields requires to build the formatting
+
+        self.fieldnames = fieldnames  # fields requires to build the formatting
         self.label = label
         self.ktable = ktable
         self.kfield = kfield
@@ -621,16 +621,16 @@ def __call__(self, value):
             if isinstance(value, list):
                 values = value
             elif self.delimiter:
-                values = value.split(self.delimiter) # because of autocomplete
+                values = value.split(self.delimiter)  # because of autocomplete
             elif value:
                 values = [value]
             else:
                 values = []
 
-            if self.field.type in ('id','integer'):
+            if field.type in ('id', 'integer'):
                 new_values = []
                 for value in values:
-                    if not (isinstance(value,(int,long)) or value.isdigit()):
+                    if not (isinstance(value, (int, long)) or value.isdigit()):
                         if self.auto_add:
                             value = str(self.maybe_add(table, self.fieldnames[0], value))
                         else:
@@ -657,8 +657,8 @@ def count(values, s=self.dbset, f=field):
                 elif count(values) == len(values):
                     return (values, None)
         else:
-            if self.field.type in ('id','integer'):
-                if isinstance(value,(int,long)) or value.isdigit():
+            if field.type in ('id', 'integer'):
+                if isinstance(value, (int, long)) or value.isdigit():
                     value = int(value)
                 elif self.auto_add:
                     value = self.maybe_add(table, self.fieldnames[0], value)
@@ -818,7 +818,7 @@ def __call__(self, value):
         if regex_isint.match(str(value)):
             v = int(value)
             if ((self.minimum is None or v >= self.minimum) and
-                (self.maximum is None or v < self.maximum)):
+                    (self.maximum is None or v < self.maximum)):
                 return (v, None)
         return (value, self.error_message)
 
@@ -892,7 +892,7 @@ def __call__(self, value):
             else:
                 v = float(str(value).replace(self.dot, '.'))
             if ((self.minimum is None or v >= self.minimum) and
-                (self.maximum is None or v <= self.maximum)):
+                    (self.maximum is None or v <= self.maximum)):
                 return (v, None)
         except (ValueError, TypeError):
             pass
@@ -978,7 +978,7 @@ def __call__(self, value):
             else:
                 v = decimal.Decimal(str(value).replace(self.dot, '.'))
             if ((self.minimum is None or v >= self.minimum) and
-                (self.maximum is None or v <= self.maximum)):
+                    (self.maximum is None or v <= self.maximum)):
                 return (v, None)
         except (ValueError, TypeError, decimal.InvalidOperation):
             pass
@@ -2248,7 +2248,7 @@ def formatter(self, value):
         y = '%.4i' % year
         format = format.replace('%y', y[-2:])
         format = format.replace('%Y', y)
-        if year < 1900: 
+        if year < 1900:
             year = 2000
         d = datetime.date(year, value.month, value.day)
         return d.strftime(format)
@@ -2347,6 +2347,7 @@ class IS_DATE_IN_RANGE(IS_DATE):
             (datetime.date(2010, 3, 3), 'oops')
 
     """
+
     def __init__(self,
                  minimum=None,
                  maximum=None,
@@ -2400,6 +2401,7 @@ class IS_DATETIME_IN_RANGE(IS_DATETIME):
             (datetime.datetime(2010, 3, 3, 0, 0), 'oops')
 
     """
+
     def __init__(self,
                  minimum=None,
                  maximum=None,
@@ -2513,7 +2515,7 @@ def urlify(s, maxlen=80, keep_underscores=False):
     if keep_underscores:
         s = re.sub('\s+', '-', s)         # whitespace to hyphens
         s = re.sub('[^\w\-]', '', s)
-                   # strip all but alphanumeric/underscore/hyphen
+        # strip all but alphanumeric/underscore/hyphen
     else:
         s = re.sub('[\s_]+', '-', s)      # whitespace & underscores to hyphens
         s = re.sub('[^a-z0-9\-]', '', s)  # strip all but alphanumeric/hyphen
@@ -2705,6 +2707,7 @@ class LazyCrypt(object):
     """
     Stores a lazy password hash
     """
+
     def __init__(self, crypt, password):
         """
         crypt is an instance of the CRYPT validator,
@@ -2760,8 +2763,8 @@ def __eq__(self, stored_password):
         # LazyCrypt objects comparison
         if isinstance(stored_password, self.__class__):
             return ((self is stored_password) or
-                   ((self.crypt.key == stored_password.crypt.key) and
-                   (self.password == stored_password.password)))
+                    ((self.crypt.key == stored_password.crypt.key) and
+                     (self.password == stored_password.password)))
 
         if self.crypt.key:
             if ':' in self.crypt.key:
@@ -3404,14 +3407,14 @@ def __call__(self, value):
                     ok = True
             if not (self.is_localhost is None or self.is_localhost ==
                     (number == self.localhost)):
-                    ok = False
+                ok = False
             if not (self.is_private is None or self.is_private ==
                     (sum([private_number[0] <= number <= private_number[1]
                           for private_number in self.private]) > 0)):
-                    ok = False
+                ok = False
             if not (self.is_automatic is None or self.is_automatic ==
                     (self.automatic[0] <= number <= self.automatic[1])):
-                    ok = False
+                ok = False
             if ok:
                 return (value, None)
         return (value, translate(self.error_message))
@@ -3695,6 +3698,7 @@ class IS_IPADDRESS(Validator):
         >>> IS_IPADDRESS(subnets='invalidsubnet')('2001::8ffa:fe22:b3af')
         ('2001::8ffa:fe22:b3af', 'invalid subnet provided')
     """
+
     def __init__(
             self,
             minip='0.0.0.0',
@@ -3757,7 +3761,7 @@ def __call__(self, value):
                 is_private=self.is_private,
                 is_automatic=self.is_automatic,
                 error_message=self.error_message
-                )(value)
+            )(value)
         elif self.is_ipv6 or isinstance(ip, IPv6Address):
             retval = IS_IPV6(
                 is_private=self.is_private,
@@ -3769,7 +3773,7 @@ def __call__(self, value):
                 is_teredo=self.is_teredo,
                 subnets=self.subnets,
                 error_message=self.error_message
-                )(value)
+            )(value)
         else:
             retval = (value, translate(self.error_message))
 


Test patch:
diff --git a/gluon/tests/test_validators.py b/gluon/tests/test_validators.py
--- a/gluon/tests/test_validators.py
+++ b/gluon/tests/test_validators.py
@@ -232,7 +232,19 @@ def test_IS_IN_DB(self):
         self.assertEqual(sorted(rtn), [('%d' % george_id, 'george'), ('%d' % costanza_id, 'costanza')])
         rtn = IS_IN_DB(db, db.person.id, db.person.name, error_message='oops', sort=True).options(zero=True)
         self.assertEqual(rtn, [('', ''), ('%d' % costanza_id, 'costanza'), ('%d' % george_id, 'george')])
+        # Test it works with self reference
+        db.define_table('category',
+            Field('parent_id', 'reference category', requires=IS_EMPTY_OR(IS_IN_DB(db, 'category.id', '%(name)s'))),
+            Field('name')
+        )
+        ret = db.category.validate_and_insert(name='seinfeld')
+        self.assertFalse(list(ret.errors))
+        ret = db.category.validate_and_insert(name='characters', parent_id=ret.id)
+        self.assertFalse(list(ret.errors))
+        rtn = IS_IN_DB(db, 'category.id', '%(name)s')(ret.id)
+        self.assertEqual(rtn, (ret.id, None))
         db.person.drop()
+        db.category.drop()
 
     def test_IS_NOT_IN_DB(self):
         from gluon.dal import DAL, Field