Repository: OCHA-DAP/hdx-ckan

Issue:
Search Page (Amok): Feature Pages - bugs&issues
Pinging @mbellotti @cjhendrix 
I redeployed to dev the latest code and I found the following issues:
1. in feature pages section (and tab) I see countries, organizations and tags instead of topics.
2. I also updated 1 dataset and 1 indicator with "Food Security" topic and I couldn't see it on first page in ALL tab and in "Feature Pages" tab even if the indicator and dataset were there. Check this url: https://dev-data.hdx.rwlabs.org/search?q=food+security
3. Instead of "Country Page" the text says "Topic Page"
4. "Updated on" is not displayed for any entity (org, country or topic/tag).
5. Counter for feature pages is removed. I think @cjhendrix decided to keep it there, but this is more a question if we need or not?
6. When I performed a full search https://dev-data.hdx.rwlabs.org/search?q= it says that there are 1343 results, and when I am going to tab "Feature Pages" I see about 68 feature pages. How do we calculate this number?



Discussion:
Regarding point 5, we did decide to keep it, but since it's not there now, and I don't fully get how we are generating the feature page list, let's leave it out for now.  

1. Can you please provide details on what you mean here? This is not possible and I'm looking at dev right now and all it displays are topics, countries and orgs. Are we talking about the tag icon? A screencap would be helpful if you can.
2. Yes, not sure why this is happening ... although all the lowercase versions of topic tags might help fix it. Incidentally we need to fix the way tags are displayed on the dataset page as the vocabularies will not work with the same url pattern
3. I already mentioned this in my update on.... Tuesday I think. I assume this is because @aalecs doesn't have an icon for countries?
4. That's because there is no information on this and you told me to remove it
5. I thought we did agree to remove it, but I can put it back if you like
6. That is the number of topics + the number of orgs + the number of groups relevant to your search ... what else should it be? If you click on the datasets tab the count changes to reflect the number of datasets, if you click the indicator tab the count changes to reflect the number of indicators, if you click the features tab the count changes to reflect the number of features. Granted if you are on ALL tab the count is just indicators + datasets because it seemed confusing to include features but that can be changed easily if you disagree. 

1. I am pasting you here the topics from dev db

```
ckandb=# select name from tag where vocabulary_id is not null;
              name               
---------------------------------
 Education
 Early Recovery
 Economic
 Emergency Shelter and NFI
 Emergency Telecommunication
 Food Security
 Funding
 Gender-based Violence
 Health
 Humanitarian Profile
 Logistics
 Mine Action
 Nutrition
 Population
 Protection
 Camp Coordination or Management
 Housing Land and Property
 Water Sanitation and Hygiene
(18 rows)
```

.....yes? Still not sure what the problem is. Phone call maybe?

1 and 2 are the same issue (kindof) Dan wants to know why certain topics are not coming up in search.

4) will try to see if I can find a date for this in the group object

5) putting the count back

6) Dan thinks the number of countries is too low (should be over 200+) Marianne thinks it's only the countries that have data in them. Will investigate :)

@cjhendrix - to answer if we going to keep or remove the counter from Features Page section

@mbellotti 
- check why we don't see any topics on search page. It might be related to upper case ?
- display date for "Update on" field for organizations and countries - maybe an existing helper function
- check why we see only 68 feature pages on search results. It should be around 243 countries, and 20 organizations. Is it a limitation maybe

@danmihaila
- get icon for country from @aalecs 
- put text "Country Page" for countries

Marianne can do what she thinks makes sense with the counter for now.  After we are happy with the behavior of the feature page results, we can decide about the counter. 


Resolving patch:
diff --git a/ckanext-hdx_search/ckanext/hdx_search/controllers/search_controller.py b/ckanext-hdx_search/ckanext/hdx_search/controllers/search_controller.py
--- a/ckanext-hdx_search/ckanext/hdx_search/controllers/search_controller.py
+++ b/ckanext-hdx_search/ckanext/hdx_search/controllers/search_controller.py
@@ -195,13 +195,9 @@ def package_search(context, data_dict):
                 if group:
                     new_facet_dict['display_name'] = group.display_name
                     new_facet_dict['description'] = group.description
-                    if hasattr(group, 'revision_timestamp'):
-                        new_facet_dict[
-                            'last_update'] = group.revision_timestamp
                 else:
                     new_facet_dict['display_name'] = key_
                     new_facet_dict['description'] = ''
-                    new_facet_dict['last_update'] = ''
             elif key == 'license_id':
                 license = model.Package.get_license_register().get(key_)
                 if license:
@@ -228,8 +224,8 @@ def package_search(context, data_dict):
     return search_results
 
 
-def sort_features(q, features):
-    return sorted(features, key=lambda x: x['count'])
+def sort_features(features):
+    return sorted(features, key=lambda x: x['count'], reverse=True)
 
 
 def isolate_features(context, facets, q, tab, skip=0, limit=25):
@@ -250,7 +246,7 @@ def isolate_features(context, facets, q, tab, skip=0, limit=25):
     for o in facets['organization']['items']:
         tags.append(o['name'])
         extract[o['name']] = {'name': o['name'], 'display_name': o['display_name'],
-                              'url': h.url_for(controller='organization', action='read', id=o['name']), 'description': o['description'], 'last_update': '', 'feature_type': 'organization', 'count': o['count']}
+                              'url': h.url_for(controller='organization', action='read', id=o['name']), 'description': o['description'], 'feature_type': 'organization', 'count': o['count']}
 
     for p in facets['vocab_Topics']['items']:
         tags.append(p['name'])
@@ -260,7 +256,7 @@ def isolate_features(context, facets, q, tab, skip=0, limit=25):
     for g in facets['groups']['items']:
         tags.append(g['name'])
         extract[g['name']] = {'name': g['name'], 'display_name': g['display_name'],
-                              'url': h.url_for(controller='group', action='read', id=g['name']), 'description': g['description'], 'last_update': '', 'feature_type': 'country', 'count': g['count']}
+                              'url': h.url_for(controller='group', action='read', id=g['name']), 'description': g['description'], 'feature_type': 'country', 'count': g['count']}
 
     if tab == 'all' and len(tags) > 3:
         if q:
@@ -272,7 +268,7 @@ def isolate_features(context, facets, q, tab, skip=0, limit=25):
     for s in selected:
         features.append(extract[s])
     if tab == 'features':
-        feature_list = sort_features(q, features)
+        feature_list = sort_features(features)
         return (feature_list[skip:skip + limit], len(features))
     return features
 
diff --git a/ckanext-hdx_theme/ckanext/hdx_theme/helpers/helpers.py b/ckanext-hdx_theme/ckanext/hdx_theme/helpers/helpers.py
--- a/ckanext-hdx_theme/ckanext/hdx_theme/helpers/helpers.py
+++ b/ckanext-hdx_theme/ckanext/hdx_theme/helpers/helpers.py
@@ -28,6 +28,7 @@
 downloadable_formats = {
     'csv', 'xls', 'xlsx', 'txt', 'jpg', 'jpeg', 'png', 'gif', 'zip', 'xml'
 }
+                    
 
 def is_downloadable(resource):
     format = resource.get('format', 'data').lower()
@@ -92,6 +93,13 @@ def get_last_revision_group(group_id):
         return activity.revision_id
     return None
 
+def get_last_revision_timestamp_group(group_id):
+    activity_objects = model.activity.group_activity_list(group_id, limit=1, offset=0)
+    if len(activity_objects)>0 :
+        activity = activity_objects[0]
+        return h.render_datetime(activity.timestamp)
+    return None
+
 def get_group_followers(grp_id):
     result = logic.get_action('group_follower_count')(
             {'model': model, 'session': model.Session},
@@ -275,4 +283,12 @@ def one_active_item(items):
     for i in items:
         if i['active']:
             return True
-    return False
\ No newline at end of file
+    return False
+
+def feature_count(features):
+    count = 0
+    for name in features:
+        facet = h.get_facet_items_dict(name)
+        for f in facet:
+            count += f['count']
+    return count
\ No newline at end of file
diff --git a/ckanext-hdx_theme/ckanext/hdx_theme/plugin.py b/ckanext-hdx_theme/ckanext/hdx_theme/plugin.py
--- a/ckanext-hdx_theme/ckanext/hdx_theme/plugin.py
+++ b/ckanext-hdx_theme/ckanext/hdx_theme/plugin.py
@@ -94,7 +94,9 @@ def get_helpers(self):
             'hdx_remove_schema_and_domain_from_url': hdx_helpers.hdx_remove_schema_and_domain_from_url,
             'hdx_get_ckan_config': hdx_helpers.hdx_get_ckan_config,
             'get_group_name_from_list': hdx_helpers.get_group_name_from_list,
-            'one_active_item': hdx_helpers.one_active_item
+            'one_active_item': hdx_helpers.one_active_item,
+            'get_last_revision_timestamp_group': hdx_helpers.get_last_revision_timestamp_group,
+            'feature_count': hdx_helpers.feature_count
         }
         
     def get_actions(self):


Test patch:
diff --git a/ckanext-hdx_search/ckanext/hdx_search/tests/test_search.py b/ckanext-hdx_search/ckanext/hdx_search/tests/test_search.py
--- a/ckanext-hdx_search/ckanext/hdx_search/tests/test_search.py
+++ b/ckanext-hdx_search/ckanext/hdx_search/tests/test_search.py
@@ -46,4 +46,27 @@ def test_features(self):
     	offset = h.url_for(
             controller='ckanext.hdx_search.controllers.search_controller:HDXSearchController', action='search')
         response = self.app.get(offset, params={'q': 'health', 'ext_feature':1})
-        assert '200' in response.status    
\ No newline at end of file
+        assert '200' in response.status
+
+    def test_sort(self):
+        import ckanext.hdx_search.controllers.search_controller as search
+        features = [{'count': 1, 'display_name': u'Sierra Leone', 'feature_type': 'country', 'url': '/group/sle', 'description': u'', 'name': 'sle'}, {'count': 1, 'display_name': u'Senegal', 'feature_type': 'country', 'url': '/group/sen', 'description': u'', 'name': 'sen'}, {'count': 50, 'display_name': u'World', 'feature_type': 'country', 'url': '/group/world', 'description': u'', 'name': 'world'}, {'count': 1, 'display_name': u'Nigeria', 'feature_type': 'country', 'url': '/group/nga', 'description': u'', 'name': 'nga'}, {'count': 1, 'display_name': u'Liberia', 'feature_type': 'country', 'url': '/group/lbr', 'description': u'', 'name': 'lbr'}, {'count': 1, 'display_name': u'Kenya', 'feature_type': 'country', 'url': '/group/ken', 'description': u'', 'name': 'ken'}, {'count': 1, 'display_name': u'Guinea', 'feature_type': 'country', 'url': '/group/gin', 'description': u'', 'name': 'gin'}, {'count': 2, 'display_name': u'Colombia', 'feature_type': 'country', 'url': '/group/col', 'description': u'', 'name': 'col'}]
+        sorted_features = search.sort_features(features)
+        assert sorted_features[0]['name'] == 'world'
+
+    def test_package_search(self):
+        import ckanext.hdx_search.controllers.search_controller as search
+        user = model.User.by_name('hdx')
+        data_dict = {
+                'q': '',
+                'fq': '',
+                'facet.field': ['vocab_Topics','groups','organizations'],
+                'rows': 25,
+                'start': (1 - 1) * 25,
+                'sort': None,
+                'extras': {}
+            }
+        context = {'model': model, 'session': model.Session,
+                       'user': user, 'for_view': True, 'auth_user_obj': user}
+        s = search.package_search(context, data_dict)
+        assert len(s['results']) > 0