Repository: getredash/redash

Issue:
Athena query runner doesn't paginate schemas
<!--
#####################################################################
#
# Need support? USE THE FORUM! https://discuss.redash.io/c/support.
#
# Don't have steps to reproduce and actually not sure it's a bug?
# Use the forum! https://discuss.redash.io/c/support.
#
#####################################################################

**Got an idea for a new feature?** Check if it isn't on the roadmap already: http://bit.ly/redash-roadmap and start a new discussion in the features category: https://discuss.redash.io/c/feature-requests 🌟.

Found a bug? Please fill out the sections below... thank you 👍
-->

### Issue Summary
Hi,

AWS Athena query runner paginates only tables but not schemas.
We have ~170 schemas, however, only the first 100 are shown in schema browser.
 
### Steps to Reproduce

1. Connect to Athena with Glue metastore which has more than 100 schemas
2. In the new query window, only 100 schemas are shown.

I've looked into the code and looks like you paginate only on tables but not on schemas (https://github.com/getredash/redash/blob/v4.0.0/redash/query_runner/athena.py#L130).
### Technical details:

* Redash Version: 4.0.0
* Browser/OS:
* How did you install Redash: docker


Kind regards,
Ievgen.


Discussion:
Possible fix (haven't tested it)
```python

schema = {}
schema_paginator = client.get_paginator('get_databases')
table_paginator = client.get_paginator('get_tables')

for databases in schema_paginator.paginate():
    for database in  databases['DatabaseList']:
        iterator = table_paginator.paginate(DatabaseName=database['Name'])
        for table in iterator.search('TableList[]'):
            table_name = '%s.%s' % (database['Name'], table['Name'])
            if table_name not in schema:
                column = [columns['Name'] for columns in table['StorageDescriptor']['Columns']]
                schema[table_name] = {'name': table_name, 'columns': column}
                for partition in table['PartitionKeys']:
                    schema[table_name]['columns'].append(partition['Name'])
```

Resolving patch:
diff --git a/redash/query_runner/athena.py b/redash/query_runner/athena.py
--- a/redash/query_runner/athena.py
+++ b/redash/query_runner/athena.py
@@ -121,18 +121,20 @@ def __get_schema_from_glue(self):
                 region_name=self.configuration['region']
                 )
         schema = {}
-        paginator = client.get_paginator('get_tables')
-
-        for database in client.get_databases()['DatabaseList']:
-            iterator = paginator.paginate(DatabaseName=database['Name'])
-            for table in iterator.search('TableList[]'):
-                table_name = '%s.%s' % (database['Name'], table['Name'])
-                if table_name not in schema:
-                    column = [columns['Name'] for columns in table['StorageDescriptor']['Columns']]
-                    schema[table_name] = {'name': table_name, 'columns': column}
-                    for partition in table.get('PartitionKeys', []):
-                        schema[table_name]['columns'].append(partition['Name'])
 
+        database_paginator = client.get_paginator('get_databases')
+        table_paginator = client.get_paginator('get_tables')
+
+        for databases in database_paginator.paginate():
+            for database in databases['DatabaseList']:
+                iterator = table_paginator.paginate(DatabaseName=database['Name'])
+                for table in iterator.search('TableList[]'):
+                    table_name = '%s.%s' % (database['Name'], table['Name'])
+                    if table_name not in schema:
+                        column = [columns['Name'] for columns in table['StorageDescriptor']['Columns']]
+                        schema[table_name] = {'name': table_name, 'columns': column}
+                        for partition in table.get('PartitionKeys', []):
+                            schema[table_name]['columns'].append(partition['Name'])
         return schema.values()
 
     def get_schema(self, get_stats=False):


Test patch: