Repository: ckan/ckan

Issue:
Escape special characters in password while db loading
CKAN doesn't escape special symbols within password.

Restore procedure:

```
paster db load -c /etc/ckan/default/production.ini my_database_dump.sql
```

in my case produces the following invalid command:

```
PGPASSWORD=Ab1239idap&u && psql -U ckan_default -h localhost -d ckan_default -f /mnt/portal/tmp/ckan.pg_dump
[1] 26628
-bash: u: command not found
```



Discussion:
I'd prefer to remove the "db load" command and instead just document how to dump and restore using the postgres tools directly.

@wardi: According to [the documentation](http://docs.ckan.org/en/latest/maintaining/paster.html#dumping-and-loading-databases-to-from-a-file), `db load` also automatically performs DB migrations if the data comes from an older CKAN version. That is a really nice feature that you won't get by using the raw postgres tools.

This command prevents users from using a bunch of the useful options available in pg_dump/pg_restore and I don't think it's really much of a burden to run `paster db upgrade` after restoring the db.

If you really like `db load` and are willing to fix this issue I won't refuse the fix. I think it's really useful to know that you don't _need_ ckan in order to perform a backup or restore, though.

Ah, I wasn't aware of the `db upgrade` command (the [online documentation](http://docs.ckan.org/en/latest/maintaining/paster.html) doesn't mention it explicitly and I never checked `paster db --help` before). In that case you're probably right and `db load` can be removed. A section in the docs describing the full database dump/restore/upgrade cycle would be useful then.

+1 for removing `db load` in favour of improved documentation.

@torfsen, it is documented where it needs to be though, albeit slightly hidden: http://docs.ckan.org/en/latest/maintaining/upgrading/upgrade-source.html#upgrading-a-source-install

@mattfullerton: Yeah, it's also described at [database migrations](http://docs.ckan.org/en/latest/contributing/database-migrations.html). However, the [command line interface page](http://docs.ckan.org/en/latest/maintaining/paster.html) always had an "exhaustive" look and feel to me. From there I learned that `db load` does the necessary migrations and that's all that I've ever needed :wink: 

Perhaps the paster reference could be auto-generated so that the reference and `paster --help` / `paster command --help` contain the same information.

Hi, I'm having the same password problem but with `paster pg dump`. 
Is that fixed already? 
If not, I fixed it on my CKAN and can submit a fix for the `paster pg dump`.

Please let me know. Thanks

Can this be re-opened until the discussed documentation improvements have been implemented? This is still affecting users, see for example [this Stack Overflow question that was posted today](http://stackoverflow.com/questions/38242908/dump-the-ckan-database-and-load-it-in-another-ckan-instance/38248006#38248006).


Resolving patch:
diff --git a/ckan/lib/cli.py b/ckan/lib/cli.py
--- a/ckan/lib/cli.py
+++ b/ckan/lib/cli.py
@@ -31,6 +31,18 @@
 #   Otherwise loggers get disabled.
 
 
+def deprecation_warning(message=None):
+    '''
+    Print a deprecation warning to STDERR.
+
+    If ``message`` is given it is also printed to STDERR.
+    '''
+    sys.stderr.write(u'WARNING: This function is deprecated.')
+    if message:
+        sys.stderr.write(u' ' + message.strip())
+    sys.stderr.write(u'\n')
+
+
 def parse_db_config(config_key='sqlalchemy.url'):
     ''' Takes a config key for a database connection url and parses it into
     a dictionary. Expects a url like:
@@ -187,10 +199,10 @@ class ManageDb(CkanCommand):
                                      search index
     db upgrade [version no.]       - Data migrate
     db version                     - returns current version of data schema
-    db dump FILE_PATH              - dump to a pg_dump file
-    db load FILE_PATH              - load a pg_dump from a file
+    db dump FILE_PATH              - dump to a pg_dump file [DEPRECATED]
+    db load FILE_PATH              - load a pg_dump from a file [DEPRECATED]
     db load-only FILE_PATH         - load a pg_dump from a file but don\'t do
-                                     the schema upgrade or search indexing
+                                     the schema upgrade or search indexing [DEPRECATED]
     db create-from-model           - create database from the model (indexes not made)
     db migrate-filestore           - migrate all uploaded data from the 2.1 filesore.
     '''
@@ -291,6 +303,7 @@ def _run_cmd(self, command_line):
             raise SystemError('Command exited with errorcode: %i' % retcode)
 
     def dump(self):
+        deprecation_warning(u"Use PostgreSQL's pg_dump instead.")
         if len(self.args) < 2:
             print 'Need pg_dump filepath'
             return
@@ -300,6 +313,7 @@ def dump(self):
         pg_cmd = self._postgres_dump(dump_path)
 
     def load(self, only_load=False):
+        deprecation_warning(u"Use PostgreSQL's pg_restore instead.")
         if len(self.args) < 2:
             print 'Need pg_dump filepath'
             return


Test patch: