PostgreSQL — Using COPY, pg dump and restore
By: Alex Isaac
Tags: Database, Postgresql, Data Dump, Copy, Psql

Image source : digitalocean.com

The COPY command in the postgreSQL is used for importing data in the files into the database table and also for exporting tables from the database to the file.

Importing from a file into Database Table

For example if we want to import data from a CSV file into the database table, the following command is used.

COPY <table_name> FROM <file_path> DELIMITER ‘,’ CSV;

The above command assumes that the CSV file has the values for all the fields in the table with delimiter as comma(,) and the fields are ordered in the file the same as that of in the table.

If the file contains the values only for the particular fields in the table, we have to explicitly give the field names along with the table_name.

For example, if the file has values only for field1, field2, and field3, but the table has many fields,

COPY <table_name>(field1, field2, field3) FROM <file_path> DELIMITER ‘ , ’ CSV;

If the COPY command was successful, it will return COPY <number of rows copied>

We can use the HEADER in the command line which can be used to ignore the first line from the file while copying because it may contain the name of the fields.

COPY <table_name> FROM <file_path> DELIMITER ‘,’ CSV HEADER;

If the error occurs in between the COPY FROM operation, the operation will be stopped, but the data stored so far will not get deleted. These data are not visible but occupies the space in the disk. In this case, we have to use VACUUM command for cleaning the unwanted disk usages.

Exporting to a file from Database Table

The COPY TO command is used to copy the tables in the database into the file by using the required delimiter.

COPY <table_name> TO <file_path> DELIMITER ‘,’ CSV HEADER;

If you don’t want column_names to be copied into the CSV file, then you can ignore the HEADER.

To export only the particular fields, you can only give the field names with the table name.

COPY <table_name>(field1, field2, field3) TO <file_path> DELIMITER ‘,’ CSV;

We can also use queries to copy the particular records which matches the condition into the file.

COPY ( SELECT * FROM <table>WHERE id=10 ) TO <file_path> DELIMITER ‘,’ CSV;

It is recommended to give the absolute pathname while using COPY. The server itself enforces you to use absolute pathname while using COPY TO. But during reading file by COPY FROM you can use both absolute as well as relative pathnames.

Difference between COPY and \copy command

If you are logged in to a remote PostgreSQL server and want to copy a table records in the file in the local machine. In this case, you can use the \copy command which is a builtin command of PostgreSQL. While using COPY, the server is doing the writing operation but during \copy, the psql writes the CSV file and transfers it into the local machine.

To use the \copy command your local machine should have sufficient privileges. It doesn’t mean the superuser privileges.

pg_dump and pg_restore

The pg_dump is a tool for backing up a database into the file. We can restore it later into another database. To back up the database into a file run,

pg_dump -U <username> -h <host> -W -F t <db_name> > <output_filename>

where,

-W forces to prompt for a password while logging to the database.

-F specifies the format of the dump file created, here t refers tar file.

Now the dump file is created for the database.

Also, we can restore the file into new database by using,

pg_restore -U <username> -h <host> -W -F t -d <db_name> <dump_file_name>

Every parameters are same as that of above.

References

https://www.postgresql.org/docs/10/sql-copy.html

https://coderwall.com/p/b8rksg/backup-and-restore-a-postgresql-database-using-commandline