Repository: akvo/akvo-rsr

Issue:
Move to IATI country list by default
We now have our own country list in RSR, and IATI also uses a country list (which we also use for some fields, such as recipient country). I think we should completely switch to the IATI country list, but that also involves some other changes (filters on RSR pages, etc).
- [x] Projects
  - [x] Remove the rsr_projectlocation.country field from the project editor
  - [x] Remove all mentions of rsr_projectlocation.country in the validation sets
  - [x] Do not hide recipient countries and regions in the RSR validation set
  - [x] Convert any country of a project location to a recipient country
  - [x] Remove the country field as a mandatory field for publishing in RSR
  - [x] Update project page
    - [x] Project header
    - [x] Full report tab (remove country field from location)
    - [x] Donation thanks page
  - [x] Update projects list
    - [x] Update mention of country per project
    - [x] Update location filter
  - [x] Update IATI import
  - [x] Add comment to state that location.country is legacy
- [x] Project updates
  - [x] Remove project update location countries (not used)
  - [x] Remove location filter on updates list
- [x] Organisations
  - [x] Add new iati_country field
  - [x] Convert existing countries to IATI countries
  - [x] Replace country field in 'old' organisation admin
  - [x] Add comment to state that location.country is legacy
  - [x] Update location filter on organisations list
  - [x] Update organisations list
- [x] Employments
  - [x] Convert the employment countries to IATI countries
  - [x] Update 'Request to join an organisation' module
- [x] Make all migrations reversable



Discussion:
Don't forget the API.
Bumping the modification date for projects and updates will make them reload, but Up currently assumes the country list will almost never change. If all the countries suddenly have different IDs it will be very confusing. Maybe we need a mechanism in the API for indicating an invalidation of cached data. 

@stellanl I'm planning the following changes, let me know if you are ok with that:
- The _country_ field for a project location stays in the model and API as it is right now. Basically, as a legacy field.
- The location field gets removed from the project editor and all countries are mapped to a _recipient country_.
- All filters etc are based on the _recipient country_ and _recipient region_ fields.

Should be ok, I only display the country, so the old Up apps should be ok as long as the field stays around. It is maybe an hour's work to switch to a new country field for a new Up release

There is also a country id field in the employment application API.

## Test plan
- [x] After the migrations, make sure that all countries - as specified in locations - are now also specified as a recipient country for projects. And that all organisation locations now have an IATI country based on the old 'country' field.
- [x] In the project editor, make sure the country field does not show for location anymore. And that recipient country and recipient region does always show.
- [x] Make sure that it is still possible to publish a project (location country used to be a requirement).
- [x] On the project page, the project header should display all recipient countries, including the percentage if specified (sorted by the highest percentage, or name otherwise). And the report tab should not show the country for locations.
- [x] On the projects list, each project should display all recipient countries, including the percentage if specified (sorted by the highest percentage, or name otherwise).
- [x] On the projects list, it should be possible to filter on countries like before.
- [x] On the updates list, the countries filter should be removed (there were no updates with countries so far).
- [x] On the organisations list, the first country of an organisation should be shown.
- [x] On the organisations list, it should be possible to filter on countries like before.
- [x] In the 'old' organisation admin, it should be possible to select a country from the IATI country codelist.
- [x] On the My details page, it should be possible to select a country (from the IATI codelist) and request to join an organisation like before.
- [x] It should be possible to reverse the migrations (can only be tested on local dev, not on Test / UAT).

After merging into test, it seems that the migrations are failing:

```
[14:04:33][Step 3/7]   File "/var/akvo/rsr/versions/deploy-RSR_Deploy-830/code/akvo/rsr/migrations/0071_auto_20160525_1036.py", line 121, in convert_locations_to_recipient_countries
[14:04:33][Step 3/7]     RecipientCountry.objects.get_or_create(project=project, country=country_code)
[14:04:33][Step 3/7] __fake__.MultipleObjectsReturned: get() returned more than one RecipientCountry -- it returned 2!
```


Resolving patch:
diff --git a/akvo/rsr/migrations/0071_auto_20160525_1036.py b/akvo/rsr/migrations/0071_auto_20160525_1036.py
--- a/akvo/rsr/migrations/0071_auto_20160525_1036.py
+++ b/akvo/rsr/migrations/0071_auto_20160525_1036.py
@@ -1,6 +1,7 @@
 # -*- coding: utf-8 -*-
 from __future__ import unicode_literals
 
+from django.core.exceptions import MultipleObjectsReturned
 from django.db import models, migrations
 
 
@@ -33,7 +34,7 @@ def add_country_to_rsr_validation_set(apps, schema_editor):
             validation='rsr_projectlocation.country',
             action=1
         )
-    except ProjectEditorValidationSet.DoesNotExist:
+    except (ProjectEditorValidationSet.DoesNotExist, MultipleObjectsReturned):
         # The RSR validation set (id = 1) can not be deleted and should always exist
         pass
 
@@ -72,7 +73,7 @@ def do_not_hide_recipient_countries_and_regions(apps, schema_editor):
                     validation="{0}.{1}".format(model_name, field_name),
                     action=2
                 )
-    except ProjectEditorValidationSet.DoesNotExist:
+    except (ProjectEditorValidationSet.DoesNotExist, MultipleObjectsReturned):
         # The RSR validation set (id = 1) can not be deleted and should always exist
         pass
 
@@ -97,7 +98,7 @@ def hide_recipient_countries_and_regions(apps, schema_editor):
             validation='rsr_recipientregion',
             action=2
         )
-    except ProjectEditorValidationSet.DoesNotExist:
+    except (ProjectEditorValidationSet.DoesNotExist, MultipleObjectsReturned):
         # The RSR validation set (id = 1) can not be deleted and should always exist
         pass
 
@@ -118,7 +119,12 @@ def convert_locations_to_recipient_countries(apps, schema_editor):
                 country_codes.append(country.iso_code.upper())
 
         for country_code in list(set(country_codes)):
-            RecipientCountry.objects.get_or_create(project=project, country=country_code)
+            try:
+                RecipientCountry.objects.get_or_create(project=project, country=country_code)
+            except MultipleObjectsReturned:
+                # This could happen when a project has the same country specified as a recipient
+                # country multiple times
+                pass
 
 
 def revert_locations_to_recipient_countries(apps, schema_editor):
diff --git a/akvo/rsr/migrations/0073_auto_20160525_1541.py b/akvo/rsr/migrations/0073_auto_20160525_1541.py
--- a/akvo/rsr/migrations/0073_auto_20160525_1541.py
+++ b/akvo/rsr/migrations/0073_auto_20160525_1541.py
@@ -58,8 +58,11 @@ def convert_employments_to_iati_countries_reverse(apps, schema_editor):
     for employment in Employment.objects.all():
         new_country_field = employment.new_country_field
         if new_country_field:
-            employment.country = Country.objects.get(iso_code=new_country_field.lower())
-            employment.save(update_fields=['country', ])
+            try:
+                employment.country = Country.objects.get(iso_code=new_country_field.lower())
+                employment.save(update_fields=['country', ])
+            except Country.DoesNotExist:
+                pass
 
 
 class Migration(migrations.Migration):


Test patch: