Repository: ckan/ckan

Issue:
datastore_info does not return all available datastore info
**CKAN version**
2.9.1

**Describe the bug**
`datastore_info` _["returns information about the data imported, such as column names and types."](https://docs.ckan.org/en/2.9/maintaining/datastore.html#ckanext.datastore.logic.action.datastore_info)_.  For example:

```
$ ckanapi action datastore_info id=5ab4b4de-c970-4619-ab55-ce4338535b24 -r https://data.boston.gov
{
  "meta": {
    "count": 1268020
  },
  "schema": {
    "Agency": "text",
    "DateEntered": "text",
    "Developer": "text",
    "GeneralContractor": "text",
    "MINOR": "text",
    "ProjectAddress_1": "text",
    "ProjectName": "text",
    "ProjectNeigborhood": "text",
    "RESIDENT": "text",
    "Race_Desc": "text",
    "SEX": "text",
    "SubContractor": "text",
    "SubContractorAddress_1": "text",
    "SubContractorAddress_2": "text",
    "TotalHours": "text",
    "Trade": "text"
  }
}
```
However, it doesn't return other relevant available datastore information like aliases and the data dictionary.

Currently, to get an alias, you need to [query a special view `_table_metadata`](https://docs.ckan.org/en/2.9/maintaining/datastore.html#resource-aliases). To [get the data dictionary](https://docs.ckan.org/en/2.9/maintaining/datastore.html#data-dictionary), you need to call `datastore_search`.


**Expected behavior**
Instead of fetching datastore information for a given resource from three different areas, expand 
`datastore_info` so that it also returns the resource's aliases and its data dictionary in addition to what it returns now,

Having the actual data type as stored in the database, and the logical datatype as described by the data dictionary is also useful for Datastore API users.  Returning if a column is indexed is also useful for developers who are using `datastore_search_sql` so they can create performant SQL queries.

**Additional details**
This will facilitate implementing #5801.  It will also help CKAN Datastore API users who are using the datastore to create efficient reports/visualizations and extensions.

datastore_info does not work for aliases
**CKAN version**
2.9.1

**Describe the bug**
`datastore_info`  does not work for resource aliases.

**Steps to reproduce**
1. Create a resource
2. Use `datastore_create` to create aliases for the resource
3. Calling `datastore_info` with a valid alias results in a `NotFound` error

**Expected behavior**
`datastore_info` should also work for aliases.  Further, it should returned the aliased resource id.



Discussion:

Resolving patch:
diff --git a/ckanext/datastore/backend/postgres.py b/ckanext/datastore/backend/postgres.py
--- a/ckanext/datastore/backend/postgres.py
+++ b/ckanext/datastore/backend/postgres.py
@@ -1857,7 +1857,7 @@ def create(self, context, data_dict):
 
         trans = context['connection'].begin()
         try:
-            # check if table already existes
+            # check if table already exists
             context['connection'].execute(
                 u'SET LOCAL statement_timeout TO {0}'.format(timeout))
             result = context['connection'].execute(
@@ -1951,45 +1951,116 @@ def resource_id_from_alias(self, alias):
     #     pass
 
     def resource_fields(self, id):
-        def _type_lookup(t):
-            if t in ['numeric', 'integer']:
-                return 'number'
-            if t.startswith('timestamp'):
-                return "date"
-            return "text"
 
-        info = {'schema': {}, 'meta': {}}
+        info = {'meta': {}, 'fields': []}
 
-        schema_results = None
-        meta_results = None
         try:
+            engine = self._get_read_engine()
+
+            # resource id for deferencing aliases
+            info['meta']['id'] = id
+
+            # count of rows in table
+            meta_sql = sqlalchemy.text(
+                u'SELECT count(_id) FROM "{0}"'.format(id))
+            meta_results = engine.execute(meta_sql)
+            info['meta']['count'] = meta_results.fetchone()[0]
+
+            # table_type - BASE TABLE, VIEW, FOREIGN TABLE, MATVIEW
+            tabletype_sql = sqlalchemy.text(u'''
+                SELECT table_type FROM INFORMATION_SCHEMA.TABLES
+                WHERE table_name = '{0}'
+                '''.format(id))
+            tabletype_results = engine.execute(tabletype_sql)
+            info['meta']['table_type'] = tabletype_results.fetchone()[0]
+            # MATERIALIZED VIEWS show as BASE TABLE, so
+            # we check pg_matviews
+            matview_sql = sqlalchemy.text(u'''
+                SELECT count(*) FROM pg_matviews
+                WHERE matviewname = '{0}'
+                '''.format(id))
+            matview_results = engine.execute(matview_sql)
+            if matview_results.fetchone()[0]:
+                info['meta']['table_type'] = 'MATERIALIZED VIEW'
+
+            # SIZE - size of table in bytes
+            size_sql = sqlalchemy.text(
+                u"SELECT pg_relation_size('{0}')".format(id))
+            size_results = engine.execute(size_sql)
+            info['meta']['size'] = size_results.fetchone()[0]
+
+            # DB_SIZE - size of database in bytes
+            dbsize_sql = sqlalchemy.text(
+                u"SELECT pg_database_size(current_database())".format(id))
+            dbsize_results = engine.execute(dbsize_sql)
+            info['meta']['db_size'] = dbsize_results.fetchone()[0]
+
+            # IDXSIZE - size of all indices for table in bytes
+            idxsize_sql = sqlalchemy.text(
+                u"SELECT pg_indexes_size('{0}')".format(id))
+            idxsize_results = engine.execute(idxsize_sql)
+            info['meta']['idx_size'] = idxsize_results.fetchone()[0]
+
+            # all the aliases for this resource
+            alias_sql = sqlalchemy.text(u'''
+                SELECT name FROM "_table_metadata" WHERE alias_of = '{0}'
+            '''.format(id))
+            alias_results = engine.execute(alias_sql)
+            aliases = []
+            for alias in alias_results.fetchall():
+                aliases.append(alias[0])
+            info['meta']['aliases'] = aliases
+
+            # get the data dictionary for the resource
+            data_dictionary = datastore_helpers.datastore_dictionary(id)
+
             schema_sql = sqlalchemy.text(u'''
-                SELECT column_name, data_type
-                FROM INFORMATION_SCHEMA.COLUMNS
-                WHERE table_name = :resource_id;
-            ''')
-            schema_results = self._get_read_engine().execute(
-                schema_sql, resource_id=id)
+                SELECT
+                f.attname AS column_name,
+                pg_catalog.format_type(f.atttypid,f.atttypmod) AS native_type,
+                f.attnotnull AS notnull,
+                i.relname as index_name,
+                CASE
+                    WHEN i.oid<>0 THEN True
+                    ELSE False
+                END AS is_index,
+                CASE
+                    WHEN p.contype = 'u' THEN True
+                    WHEN p.contype = 'p' THEN True
+                    ELSE False
+                END AS uniquekey
+                FROM pg_attribute f
+                JOIN pg_class c ON c.oid = f.attrelid
+                JOIN pg_type t ON t.oid = f.atttypid
+                LEFT JOIN pg_constraint p ON p.conrelid = c.oid
+                          AND f.attnum = ANY (p.conkey)
+                LEFT JOIN pg_index AS ix ON f.attnum = ANY(ix.indkey)
+                          AND c.oid = f.attrelid AND c.oid = ix.indrelid
+                LEFT JOIN pg_class AS i ON ix.indexrelid = i.oid
+                WHERE c.relkind = 'r'::char
+                      AND c.relname = '{0}'
+                      AND f.attnum > 0
+                ORDER BY c.relname,f.attnum;
+            '''.format(id))
+            schema_results = engine.execute(schema_sql)
+            schemainfo = {}
             for row in schema_results.fetchall():
-                k = row[0]
-                v = row[1]
-                if k.startswith('_'):  # Skip internal rows
+                colname = row.column_name
+                if colname.startswith('_'):  # Skip internal rows
                     continue
-                info['schema'][k] = _type_lookup(v)
+                colinfo = {'native_type': row.native_type,
+                           'notnull': row.notnull,
+                           'index_name': row.index_name,
+                           'is_index': row.is_index,
+                           'uniquekey': row.uniquekey}
+                schemainfo[colname] = colinfo
 
-            # We need to make sure the resource_id is a valid resource_id
-            # before we use it like this, we have done that above.
-            meta_sql = sqlalchemy.text(u'''
-                SELECT count(_id) FROM "{0}";
-            '''.format(id))
-            meta_results = self._get_read_engine().execute(
-                meta_sql, resource_id=id)
-            info['meta']['count'] = meta_results.fetchone()[0]
-        finally:
-            if schema_results:
-                schema_results.close()
-            if meta_results:
-                meta_results.close()
+            for field in data_dictionary:
+                field.update({'schema': schemainfo[field['id']]})
+                info['fields'].append(field)
+
+        except Exception:
+            pass
         return info
 
     def get_all_ids(self):
diff --git a/ckanext/datastore/logic/action.py b/ckanext/datastore/logic/action.py
--- a/ckanext/datastore/logic/action.py
+++ b/ckanext/datastore/logic/action.py
@@ -289,27 +289,57 @@ def datastore_upsert(context, data_dict):
 
 def datastore_info(context, data_dict):
     '''
-    Returns information about the data imported, such as column names
-    and types.
+    Returns detailed metadata about a resource.
+
+    :param resource_id: id or alias of the resource we want info about.
+    :type resource_id: string
+
+    **Results:**
+
+    :rtype: dictionary
+    :returns:
+        **meta**: resource metadata dictionary with the following keys:
+
+        - aliases - aliases (views) for the resource
+        - count - row count
+        - db_size - size of the datastore database (bytes)
+        - id - resource id (useful for dereferencing aliases)
+        - idx_size - size of all indices for the resource (bytes)
+        - size - size of resource (bytes)
+        - table_type - BASE TABLE, VIEW, FOREIGN TABLE or MATERIALIZED VIEW
+
+        **fields**: A list of dictionaries based on :ref:`fields`, with an
+        additional nested dictionary per field called **schema**, with the
+        following keys:
+
+        - native_type - native database data type
+        - index_name
+        - is_index
+        - notnull
+        - uniquekey
 
-    :rtype: A dictionary describing the columns and their types.
-    :param id: Id of the resource we want info about
-    :type id: A UUID
     '''
     backend = DatastoreBackend.get_active_backend()
 
-    p.toolkit.check_access('datastore_info', context, data_dict)
-
     resource_id = _get_or_bust(data_dict, 'id')
-    p.toolkit.get_action('resource_show')(context, {'id': resource_id})
-
     res_exists = backend.resource_exists(resource_id)
     if not res_exists:
-        raise p.toolkit.ObjectNotFound(p.toolkit._(
-            u'Resource "{0}" was not found.'.format(resource_id)
-        ))
+        alias_exists, real_id = backend.resource_id_from_alias(resource_id)
+        if not alias_exists:
+            raise p.toolkit.ObjectNotFound(p.toolkit._(
+                u'Resource/Alias "{0}" was not found.'.format(resource_id)
+            ))
+        else:
+            id = real_id
+    else:
+        id = resource_id
+
+    data_dict['id'] = id
+    p.toolkit.check_access('datastore_info', context, data_dict)
+
+    p.toolkit.get_action('resource_show')(context, {'id': id})
 
-    info = backend.resource_fields(resource_id)
+    info = backend.resource_fields(id)
 
     return info
 


Test patch:
diff --git a/ckanext/datastore/tests/test_info.py b/ckanext/datastore/tests/test_info.py
--- a/ckanext/datastore/tests/test_info.py
+++ b/ckanext/datastore/tests/test_info.py
@@ -19,15 +19,38 @@ def test_info_success():
             {"from": "Brazil", "to": "Italy", "num": 22},
         ],
     }
-    result = helpers.call_action("datastore_create", **data)
+    helpers.call_action("datastore_create", **data)
+
+    # aliases can only be created against an existing resource
+    data = {
+        "resource_id": resource["id"],
+        "force": True,
+        "aliases": "testalias1, testview2",
+
+    }
+    helpers.call_action("datastore_create", **data)
 
     info = helpers.call_action("datastore_info", id=resource["id"])
 
-    assert info["meta"]["count"] == 2, info["meta"]
-    assert len(info["schema"]) == 3
-    assert info["schema"]["to"] == "text"
-    assert info["schema"]["from"] == "text"
-    assert info["schema"]["num"] == "number", info["schema"]
+    assert len(info["meta"]) == 7, info["meta"]
+    assert info["meta"]["count"] == 2
+    assert info["meta"]["table_type"] == "BASE TABLE"
+    assert len(info["meta"]["aliases"]) == 2
+    assert info["meta"]["aliases"] == ["testview2", "testalias1"]
+    assert len(info["fields"]) == 3, info["fields"]
+    assert info["fields"][0]["id"] == "from"
+    assert info["fields"][0]["type"] == "text"
+    assert info["fields"][0]["schema"]["native_type"] == "text"
+    assert not info["fields"][0]["schema"]["is_index"]
+    assert info["fields"][2]["id"] == "num"
+    assert info["fields"][2]["schema"]["native_type"] == "integer"
+
+    # check datastore_info with alias
+    info = helpers.call_action("datastore_info", id='testalias1')
+
+    assert len(info["meta"]) == 7, info["meta"]
+    assert info["meta"]["count"] == 2
+    assert info["meta"]["id"] == resource["id"]
 
 
 @pytest.mark.ckan_config("ckan.plugins", "datastore")