Change "private sector URL" to "non-government" URL
Merge pull request #430 from rjc123/rjc-edits

Change "private sector URL" to "non-government" URL
Convert config/migrations to PostgreSQL

* Remove structure.sql, add schema.rb, configure appropriately
* Remove mySQL-specific option values from migrations
* Remove mysql2 gem in favour of pg
* Empty the migration which fixed MySQL collation
  This migration was used to fix some collation inconsistencies in
  MySQL but we don't need its functionality on PostgreSQL. The migration
  will remain in the schema_migrations table in the database, so we're
  leaving this file in place but with empty methods to avoid the
  potentially confusing '********** NO FILE **********' message shown by
  `rake db:migrate:status`.
Explicit TNA timestamp conversion

* mySQL used to implicitly convert this
* We aren't in Kansas any more
Use correct type for http_status

* mySQL would handle this implicitly
* Postgres doesn't, thank $DEITY
Fix list ordering in host/mapping_controller specs

* Used to work by coincidence in mySQL
* We're only interested in the set, not the order
Site#default_host depends on host insert order

* Make that explicit, we got away with it in mySQL
Get everything failing for a reason

* Raise RuntimeErrors with suggestions so we can see what
  class of problem breaks what in specs
* Gives us something to fix. Ignoring features right now,
  concentrating on specs
Postgres: fix organisation sites scope

* Digit boolean becomes boolean literal
* Can't alias the COUNT in the HAVING
* ONE, ah ha ha
* TWO, ah ha ha
Fix hits update

* Add upsert support that was missing from the spike
Fix the import of DailyHitTotal

* Split upsert into two queries
* Be specific about the behaviour the specs are testing
Fix sort by hits

* Postgres requires NULLS LAST on an ORDER DESC, apparently
* We can get away with the GROUP BY primary key trick for the rest
Fix HitsMappingsRelations import

* Straight mySQL -> postgreSQL swap
* INSERT IGNORE simulation and ANSI update style
Fix lack of type conversions

* A couple of dynamic columns used for counts on model scopes
  (usually SUM(something) as attr_name) were coming back as strings
  see https://groups.google.com/forum/#!topic/ruby-pg/egzlYNi2OKk)
* Lay a method on top of the attribute to do the conversion on the fly
* Better suggestions welcomed!
Fix mappings from host paths

* We really need the GROUP BY, and the way we tell Postgres that
  the path is effectively represented by the hash is to use MIN
  as the aggregate. We can no longer use #pluck as this will affect the
  SELECTed field, but we can get the same behaviour by simple #map.
Fix hit scope groupings

* Fall back on MIN for path_hash, mapping_id and host_id when grouping hits
Postgres adapter backport

* From ActiveRecord 4.0
* Has path to allow hash of variables
* Allows increase in work_mem - vital for HashAggregate to be used instead of slower GroupAggregate
Widen hits and hits_staging path column

* Would fall over on odd too-long rows from hits
Turn off quoting

* CDN files occasionally have unmatched quotes in the path
* Postgres doesn't allow disabling quoting when treating a file
  as CSV just to skip the header, so pretend the quote is an
  unprintable (E'\b' == backspace)
Use async commits

* Speeds up smaller transactions
* Returns before write-ahead log is persisted to disk
* Gives up the D (durability) in ACID - could result in data loss if the
  server crashed, but in the case of hits this would almost
  immediately be re-imported and in the case of bulk mapping
  adds would be repeatable.