Repository: cloudspannerecosystem/harbourbridge
Pull request #218 (2021-11-09 04:54:45)

Update all the READMEs to reflect new subcommands and flags structure
- [x] Tests pass
- [x] Appropriate changes to README are included in PR

agasheesh: Thanks @nevinheintze and @bharadwaj-aditya for thorough reviews. There were many open ended questions around how to structure the READMEs and the same is reflected in the number of code review iterations that was required for this PR. But thanks to the same, I feel much more comfortable with the current state. PTAL.

Patch:
diff --git a/README.md b/README.md
index ecbd275..1fc85d8 100644
--- a/README.md
+++ b/README.md
@@ -3,8 +3,8 @@
 [![cloudspannerecosystem](https://circleci.com/gh/cloudspannerecosystem/harbourbridge.svg?style=svg)](https://circleci.com/gh/cloudspannerecosystem/harbourbridge)
 
 HarbourBridge is a stand-alone open source tool for Cloud Spanner evaluation and
-migration, using data from an existing PostgreSQL, MySQL or DynamoDB database. 
-The tool ingests schema and data from either a pg_dump/mysqldump file or directly 
+migration, using data from an existing PostgreSQL, MySQL or DynamoDB database.
+The tool ingests schema and data from either a pg_dump/mysqldump file or directly
 from the source database, and supports both schema and data migration. For schema
 migration, HarbourBridge automatically builds a Spanner schema from the schema
 of the source database. This schema can be customized using the HarbourBridge
@@ -17,13 +17,13 @@ For more details on schema customization and use of the schema assistant, see
 capabilities of HarbourBridge.
 
 HarbourBridge is designed to simplify Spanner evaluation and migration, and in
-particular for migrating moderate-size PostgreSQL/MySQL/DynamoDB datasets to Spanner 
+particular for migrating moderate-size PostgreSQL/MySQL/DynamoDB datasets to Spanner
 (up to about 100GB). Many features of PostgreSQL/MySQL, especially those that don't
 map directly to Spanner features, are ignored, e.g. stored functions and
 procedures, and sequences. Types such as integers, floats, char/text, bools,
 timestamps, and (some) array types, map fairly directly to Spanner, but many
 other types do not and instead are mapped to Spanner's `STRING(MAX)`.
-In the case of DynamoDB, the schema is inferred based on a certain 
+In the case of DynamoDB, the schema is inferred based on a certain
 amount of sampled data.
 
 View HarbourBridge as a way to get up and running fast, so you can focus on
@@ -31,34 +31,13 @@ critical things like tuning performance and getting the most out of
 Spanner. Expect that you'll need to tweak and enhance what HarbourBridge
 produces.
 
-To use the tool on a PostgreSQL database called mydb, run
-
-```sh
-# By default, the driver is "pg_dump".
-pg_dump mydb | harbourbridge
-# Or,
-pg_dump mydb | harbourbridge -driver=pg_dump
-```
-
-To use the tool on a MySQL database called mydb, run
-
-```sh
-mysqldump mydb | harbourbridge -driver=mysqldump
-```
-
-To use the tool on a DynamoDB database, run
-```sh
-harbourbridge -driver=dynamodb
-```
+For some quick starter examples on how to run HarbourBridge, take a look at
+[Quickstart Guide](#quickstart-guide).
 
-HarbourBridge accepts pg_dump/mysqldump's standard plain-text format, but 
-not archive or custom formats. More details on usage can be found in 
-[Example usage](#example-usage) section.
-
-HarbourBridge automatically determines the cloud project and Spanner instance to
-use, and generates a new Spanner database name (prefixed with `{driver}_` and
-today's date). Command-line flags can be used to explicitly set the Spanner
-instance or database name. See [Options](#options).
+HarbourBridge automatically determines the cloud project to use, and generates
+a new Spanner database name.
+Command-line flags can be used to explicitly set the Spanner instance or
+database name. See [Command line flags](#command-line-flags).
 
 **WARNING: Please check that permissions for the Spanner instance used by
 HarbourBridge are appropriate. Spanner manages access control at the database
@@ -117,21 +96,22 @@ Spanner instance.
 
 Install Go ([download](https://golang.org/doc/install)) on your development
 machine if it is not already installed, configure the GOPATH environment
-variable if it is not already configured using 
-`export GOPATH=$(go env GOPATH)`, and 
+variable if it is not already configured, and
 [test your installation](https://golang.org/doc/install#testing).
 
 ### Installing HarbourBridge
 
 You can make a copy of the HarbourBridge codebase from the github repository
-and use "go run":
+and use "go run".
 
 ```sh
 git clone https://github.com/cloudspannerecosystem/harbourbridge
 cd harbourbridge
-go run github.com/cloudspannerecosystem/harbourbridge -driver=pg_dump < my_pg_dump_file
+go run github.com/cloudspannerecosystem/harbourbridge help
 ```
 
+Examples below assume that `harbourbridge` alias is set as following
+
 ```sh
 alias harbourbridge="go run github.com/cloudspannerecosystem/harbourbridge"
 ```
@@ -143,34 +123,43 @@ This workflow also allows you to modify or customize the HarbourBridge codebase.
 To use the tool on a PostgreSQL database called mydb, run
 
 ```sh
-pg_dump mydb | harbourbridge -driver=pg_dump
+pg_dump mydb > mydb.pg_dump
+harbourbridge schema-and-data -source=postgresql < mydb.pg_dump
 ```
 
 To use the tool on a MySQL database called mydb, run
 
 ```sh
-mysqldump mydb | harbourbridge -driver=mysqldump
+mysqldump mydb > mydb.mysqldump
+harbourbridge schema-and-data -source=mysql < mydb.mysqldump
 ```
 
 To use the tool on a DynamoDB database, run
 
 ```sh
-harbourbridge -driver=dynamodb
+harbourbridge schema-and-data -source=dynamodb
 ```
-More details on running harbourbridge can be found in [Example usage](#example-usage) section.
+
+Note: HarbourBridge accepts pg_dump/mysqldump's standard plain-text format,
+but not archive or custom formats. More details on HarbourBridge example usage
+can be found here:
+
+- [PostgreSQL example usage](sources/postgres/README.md#example-postgresql-usage)
+- [MySQL example usage](sources/mysql/README.md#example-mysql-usage)
+- [DynamoDB example usage](sources/dynamodb/README.md#example-dynamodb-usage)
 
 This command will use the cloud project specified by the `GCLOUD_PROJECT`
 environment variable, automatically determine the Cloud Spanner instance
-associated with this project, convert the source schema to a Spanner schema 
+associated with this project, convert the source schema to a Spanner schema
 (For MySQL/Postgres) or infer a schema from the DynamoDB instance, create a
 new Cloud Spanner database with this schema, and finally, populate this new
-database with the data from the source database. 
-If the project has multiple instances, then list of available instances 
-will be shown and you will have to pick one of the available instances and 
-set the `--instance` flag. The new Cloud Spanner database will have a name of 
-the form `{DRIVER}_{DATE}_{RANDOM}`, where`{DRIVER}` is the drivername 
-specified,`{DATE}`is today's date, and`{RANDOM}` is a random suffix for 
-uniqueness.
+database with the data from the source database.
+If the project has multiple instances, then list of available instances
+will be shown and you will have to pick one of the available instances and
+set the `--instance` flag in target-profile. The new Cloud Spanner database
+will have a name of the form `{SOURCE}_{DATE}_{RANDOM}`, where`{SOURCE}` is the
+value of the source flag,`{DATE}`is today's date, and`{RANDOM}` is a random
+suffix for uniqueness.
 
 See the [Troubleshooting Guide](#troubleshooting-guide) for help on debugging
 issues.
@@ -190,11 +179,10 @@ and one for user carts). The files [singers.pg_dump](examples/singers.pg_dump)
 and [singers.mysqldump](examples/singers.mysqldump) contain pg_dump and
 mysqldump output for a version of the [Cloud Spanner
 singers](https://cloud.google.com/spanner/docs/schema-and-data-model#creating_a_table)
-example. To use HarbourBridge on cart.pg_dump, download the file locally and
-run
+example. To use HarbourBridge on cart.pg_dump, download the file locally and run
 
-```
-harbourbridge -driver=pg_dump < cart.pg_dump
+```sh
+harbourbridge schema -source=postgresql < cart.pg_dump
 ```
 
 ### Verifying Results
@@ -207,7 +195,7 @@ it. This will list the tables created by HarbourBridge. Select a table, and take
 a look at its schema and data. Next, go to the query page, and try
 some SQL statements. For example
 
-```
+```sql
 SELECT COUNT(*) from mytable
 ```
 
@@ -217,13 +205,13 @@ to check the number of rows in table `mytable`.
 
 The tables created by HarbourBridge provide a starting point for evaluation of
 Spanner. While they preserve much of the core structure of your PostgreSQL/MySQL
-schema and data, many key features have been dropped, including functions, 
-sequences, procedures,triggers, and views. For DynamoDB, the conversion from 
-schemaless to schema is focused on the use-case where customers use DynamoDB in 
+schema and data, many key features have been dropped, including functions,
+sequences, procedures,triggers, and views. For DynamoDB, the conversion from
+schemaless to schema is focused on the use-case where customers use DynamoDB in
 a consistent, structured way with a fairly well defined set of columns and types.
 
 As a result, the out-of-the-box performance you get from these tables could be
-slower than what you get from PostgreSQL/MySQL/DynamoDB. 
+slower than what you get from PostgreSQL/MySQL/DynamoDB.
 
 To improve performance, also consider using [Interleaved
 Tables](https://cloud.google.com/spanner/docs/schema-and-data-model#creating-interleaved-tables)
@@ -240,11 +228,11 @@ plans and aspirations for developing HarbourBridge further are outlined in the
 Whitepaper](https://github.com/cloudspannerecosystem/harbourbridge/blob/master/whitepaper.md).
 
 You can also change the way HarbourBridge behaves by directly editing the
-pg_dump/mysqldump output. For example, suppose you want to try out different 
-primary keys for a table. First run pg_dump/mysqldump and save the output to 
-a file. Then modify (or add) the relevant 
-`ALTER TABLE ... ADD CONSTRAINT ... PRIMARY KEY ...` statement in the 
-pg_dump/mysqldump output file so that the primary keys match what you need. 
+pg_dump/mysqldump output. For example, suppose you want to try out different
+primary keys for a table. First run pg_dump/mysqldump and save the output to
+a file. Then modify (or add) the relevant
+`ALTER TABLE ... ADD CONSTRAINT ... PRIMARY KEY ...` statement in the
+pg_dump/mysqldump output file so that the primary keys match what you need.
 Then run HarbourBridge on the modified pg_dump/mysqldump output.
 
 ## Files Generated by HarbourBridge
@@ -260,8 +248,8 @@ HarbourBridge generates several files as it runs:
 
 - Report file (ending in `report.txt`): contains a detailed analysis of the
   PostgreSQL/MySQL to Spanner migration, including table-by-table stats and an
-  analysis of PostgreSQL/MySQL types that don't cleanly map onto Spanner types. 
-  Note that PostgreSQL/MySQL types that don't have a corresponding Spanner type 
+  analysis of PostgreSQL/MySQL types that don't cleanly map onto Spanner types.
+  Note that PostgreSQL/MySQL types that don't have a corresponding Spanner type
   are mapped to STRING(MAX).
 
 - Bad data file (ending in `dropped.txt`): contains details of data
@@ -273,75 +261,132 @@ By default, these files are prefixed by the name of the Spanner database (with a
 dot separator). The file prefix can be overridden using the `-prefix`
 [option](#options).
 
-## Options
+## HarbourBridge CLI (command line interface)
 
-HarbourBridge accepts the following options:
+HarbourBridge CLI follows [subcommands](https://github.com/google/subcommands)
+structure with the the following general syntax:
 
-`-dbname` Specifies the name of the Spanner database to create. This must be a
-new database. If dbname is not specified, HarbourBridge creates a new unique
-dbname.
+```sh
+harbourbridge <subcommand> flags
+```
 
-`-instance` Specifies the Spanner instance to use. The new database will be
-created in this instance. If not specified, the tool automatically determines an
-appropriate instance using gcloud.
+### Getting Help
+
+The command hb help displays the available subcommands and the important global flags. The hb flags command displays the list of all available global flags, including flags from other libraries that HarbourBridge uses.
+  
+```text
+    commands   list all subcommand names
+    help   describe subcommands and their syntax
+```
+
+To get help on individual subcommands, use
+
+```sh
+    harbourbridge help <subcommand>
+```
+
+This will print the usage pattern, a few examples, and a list of all available subcommand flags.
+
+### Subcommands
+
+#### harbourbridge `schema`
+
+This subcommand can be used to perform schema migration and report on the quality of the migration. Generated schema mapping file (session.json) can be then further edited using the HarbourBridge web UI to make custom edits to the destination schema. This session file
+is then passed to the data subcommand to perform data migration while honoring the defined
+schema mapping. HarbourBridge also generates Spanner schema which users can modify manually and use directly as well.
+
+#### harbourbridge `data`
 
-`-driver` Specifies the driver to use for schema and data conversion. Supported 
-drivers are _'postgres'_, _'pg_dump'_, _'mysql'_ and _'mysqldump'_. By default, 
-the driver is _'pg_dump'_.
+This subcommand will perform data migration and report on the quality of the same. Rows which could not be migrated are reported in
+dropped.txt file. This subcommand requires users to pass the session file (which contains schema mapping) generated by either the `schema` subcommand or web UI.
 
-`-schema-sample-size` Specifies the number of rows to use for inferring schema 
-(only for DynamoDB). By default, the schema sample size is 100,000.
+#### harbourbridge `schema-and-data`
+
+This subcommand will generate a schema as well as perform data migration and report on the quality of both schema migration and data migration. This subcommand can be used to do a quick evaluation for the migration and get started quickly on Spanner.
+
+### Command line flags
+
+This section describes the flags common across all the subcommands. For flags
+specific to a give subcommand run `harbourbridge help <subcommand>`.
+
+`-source` Required flag. Specifies the source database. Supported source
+databases are _'postgres'_, _'mysql'_ and _'dynamodb'_.
+
+`-target` Optional flag. Specifies the target database. Defaults to _'spanner'_
+, which is the only supported target database today.
 
 `-prefix` Specifies a file prefix for the report, schema, and bad-data files
 written by the tool. If no file prefix is specified, the name of the Spanner
 database (plus a '.') is used.
 
-`-v` or `-verbose` Specifies verbose mode. This will cause HarbourBridge to output detailed
-messages about the conversion.
-
-`-schema-only` Specifies that only schema processing will be performed.
-Any data in the source database will be ignored.
-
-`-data-only` Specifies that only data migration will be performed.
-A spanner database will be created based on the schema state provided
-by a session file (`-session`) and data will be migrated.
+`-v` or `-verbose` Specifies verbose mode. This will cause HarbourBridge to
+output detailed messages about the conversion.
 
 `-skip-foreign-keys` Controls whether we add foreign key constraints after
 data migration is complete. This flag cannot be used with schema-only mode,
 and does not affect the generation of foreign key statements during schema
-processing i.e. foreign key constraints will still appear in the generated 
+processing i.e. foreign key constraints will still appear in the generated
 Spanner DDL files.
 
-`-session` Specifies a session file that contains all schema and data 
+`-session` Specifies a session file that contains all schema and data
 conversion state endcoded as JSON.
 
-`-dump-file` Specifies the path where the dump file is stored. 
-If the location is Google Cloud Storage (GCS), you can use the following format:
-`-dump-file=gs://{bucket_name}/{path/to/dump_file}`. Please ensure you have
-read pemissions to the GCS bucket you would like to use. 
+`-source-profile` Specifies detailed parameters for the source database such as connection parameters. See [Source Profile](#source-profile) for details.
 
-## Example Usage
+`-target-profile` Specifies detailed parameters for the target database. See [Target Profile](#target-profile) for details.
 
-Details on HarbourBridge example usage can be found here: 
-- [PostgreSQL example usage](sources/postgres/README.md#example-postgresql-usage)
-- [MySQL example usage](sources/mysql/README.md#example-mysql-usage)
-- [DynamoDB example usage](sources/dynamodb/README.md#example-dynamodb-usage)
+### Source Profile
 
+HarbourBridge accepts the following params for --source-profile,
+specified as "key1=value1,key2=value,..." pairs:
+
+`file` Specifies the full path of the file to use for reading source database
+schema and/or data. This param is optional, and the file can also be piped to
+stdin, if available locally.
+
+If the file is located in Google Cloud Storage (GCS), you can use the
+following format: `file=gs://{bucket_name}/{path/to/file}`. Please ensure you
+have read pemissions to the GCS bucket you would like to use.
+
+`format` Specifies the format of the file. This param is also optional, and
+defaults to `dump`. This may be extended in future to support other formats
+such as `csv`, `avro` etc.
+
+### Target Profile
+
+HarbourBridge accepts the following options for --target-profile,
+specified as "key1=value1,key2=value,..." pairs:
+
+`dbname` Specifies the name of the Spanner database to create. This must be a
+new database. If dbname is not specified, HarbourBridge creates a new unique
+dbname.
+
+`instance` Specifies the Spanner instance to use. The new database will be
+created in this instance. If not specified, the tool automatically determines an
+appropriate instance using gcloud.
+
+`dialect` Specifies the dialect of Spanner database. By default, Spanner
+databases are created with GoogleSQL dialect. You can override the same by
+setting `dialect=PostgreSQL` in the `-target-profile`. Learn more about support
+for PostgreSQL dialect in Cloud Spanner at
+<https://cloud.google.com/spanner/docs/postgresql-interface>.
 
 ## Schema Conversion
 
 Details on HarbourBridge schema conversion can be found here:
+
 - [PostgreSQL schema conversion](sources/postgres/README.md#schema-conversion)
 - [MySQL schema conversion](sources/mysql/README.md#schema-conversion)
 - [DynamoDB schema conversion](sources/dynamodb/README.md#schema-conversion)
 
 ## Data Conversion
 
-HarbourBridge converts PostgreSQL/MySQL/DynamoDB data to Spanner data based on 
-the Spanner schema it constructs. Conversion for most data types is fairly 
-straightforward, but several types deserve discussion. Note that HarbourBridge 
-is not intended for databases larger than about 100GB. Details on HarbourBridge 
+HarbourBridge converts PostgreSQL/MySQL/DynamoDB data to Spanner data based on
+the Spanner schema it constructs. Conversion for most data types is fairly
+straightforward, but several types deserve discussion. Note that HarbourBridge
+is not intended for databases larger than about 100GB. Details on HarbourBridge
 data conversion can be found here:
+
 - [PostgreSQL data conversion](sources/postgres/README.md#data-conversion)
 - [MySQL data conversion](sources/mysql/README.md#data-conversion)
 - [DynamoDB data conversion](sources/dynamodb/README.md#data-conversion)
@@ -351,14 +396,29 @@ data conversion can be found here:
 The following steps can help diagnose common issues encountered while running
 HarbourBridge.
 
-### 1. Verify driver configuration
+### 1. Verify source profile configuration
+
+First, check that the source profile is correctly configured to connect to your
+database. Source profile configuration varies depending on the database.
+
+#### 1.1 Direct access to PostgreSQL
+
+See [Directly connecting to a PostgreSQL database](sources/postgres/README.md#directly-connecting-to-a-postgresql-database) for troubleshooting direct access to PostgreSQL.
+
+#### 1.2 Direct access to MySQL
+
+See [Directly connecting to a MySQL database](sources/mysql/README.md#directly-connecting-to-a-mysql-database) for troubleshooting direct access to MySQL.
+
+#### 1.3 Direct access to DynamoDB
 
-First, check that the driver is correctly configured to connect to your
-database. Driver configuration varies depending on the driver used.
+See [DynamoDB example usage](sources/dynamodb/README.md#example-dynamodb-usage)
+for troubleshooting direct access to DynamoDB.
 
-#### 1.1 pg_dump
+### 2. Generating dump files
 
-If you are using pg_dump (-driver=pg_dump), check that pg_dump is
+#### 2.1 Generating pg_dump file
+
+If you are using pg_dump , check that pg_dump is
 correctly configured to connect to your PostgreSQL
 database. Note that pg_dump uses the same options as psql to connect to your
 database. See the [psql](https://www.postgresql.org/docs/9.3/app-psql.html) and
@@ -371,45 +431,17 @@ which are standard across PostgreSQL utilities.
 It is also possible to configure access via pg_dump's command-line options
 `--host`, `--port`, and `--username`.
 
-#### 1.2 Direct access to PostgreSQL
-
-In this case, HarbourBridge connects directly to the PostgreSQL
-database to retrieve table schema and data. Set the `-driver=postgres`
-and provide the environment variables `PGHOST`, `PGPORT`, `PGUSER`, 
-`PGDATABASE`. Password can be specified either in the `PGPASSWORD`
-environment variable or provided at the password prompt.
-
-#### 1.3 mysqldump
-
-If you are using mysqldump (-driver=mysqldump), check that mysqldump is
-correctly configured to connect to your MySQL
-database via the command-line options `--host`, `--port`, and `--user`.
-Note that mysqldump uses the same options as mysql to connect to your
-database. See the [mysql](https://dev.mysql.com/doc/refman/8.0/en/mysql-commands.html)
-and [mysqldump](https://dev.mysql.com/doc/refman/8.0/en/mysqldump.html) documentation.
-
-#### 1.4 Direct access to MySQL
-
-In this case, HarbourBridge connects directly to the MySQL
-database to retrieve table schema and data. Set the `-driver=mysql`
-and provide the environment variables `MYSQLHOST`, `MYSQLPORT`, 
-`MYSQLUSER`, `MYSQLDATABASE`. Password can be specified either 
-in the `MYSQLPWD` environment variable or provided at the password 
-prompt.
-
-
-#### 1.5 Direct access to DynamoDB
-
-In this case, HarbourBridge connects directly to the DynamoDB
-database to retrieve table schema and data. Set the `-driver=dynamodb`
-and provide the environment variables `AWS_ACCESS_KEY_ID`, 
-`AWS_SECRET_ACCESS_KEY`, `AWS_REGION`. If you use a custom endpoint for 
-dynamodb, you can specify that using the environment variable 
-`DYNAMODB_ENDPOINT_OVERRIDE`.
+#### 2.2 mysqldump
 
-### 2. Verify dump output
+If you are using mysqldump, check that mysqldump is correctly configured to
+connect to your MySQL database via the command-line options `--host`, `--port`,
+and `--user`. Note that mysqldump uses the same options as mysql to connect to
+your database. See the
+[mysql](https://dev.mysql.com/doc/refman/8.0/en/mysql-commands.html) and
+[mysqldump](https://dev.mysql.com/doc/refman/8.0/en/mysqldump.html)
+documentation.
 
-Next, verify that pg_dump/mysqldump is generating plain-text output. If your 
+Next, verify that pg_dump/mysqldump is generating plain-text output. If your
 database is small, try running
 
 ```sh
@@ -420,10 +452,10 @@ and look at the output file. It should be a plain-text file containing SQL
 commands. If your database is large, consider just dumping the schema via the
 `--schema-only` for pg_dump and `--no-data` for mysqldump command-line option.
 
-pg_dump/mysqldump can export data in a variety of formats, but HarbourBridge 
+pg_dump/mysqldump can export data in a variety of formats, but HarbourBridge
 only accepts `plain` format (aka plain-text). See the
 [pg_dump documentation](https://www.postgresql.org/docs/9.3/app-pgdump.html) and
-[mysqldump documentation](https://dev.mysql.com/doc/refman/8.0/en/mysqldump.html) 
+[mysqldump documentation](https://dev.mysql.com/doc/refman/8.0/en/mysqldump.html)
 for details about formats.
 
 ### 3. Debugging HarbourBridge
@@ -432,12 +464,12 @@ The HarbourBridge tool can fail for a number of reasons.
 
 #### 3.1 No space left on device
 
-HarbourBridge needs to read the pg_dump/mysqldump output twice, once to build 
+HarbourBridge needs to read the pg_dump/mysqldump output twice, once to build
 a schema and once for data ingestion. When pg_dump/mysqldump output is directly
-piped to HarbourBridge, `stdin` is not seekable, and so we write the output to 
-a temporary file. That temporary file is created via Go's ioutil.TempFile. 
-On many systems, this creates a file in `/tmp`, which is sometimes configured 
-with minimal space. A simple workaround is to separately run pg_dump/mysqldump 
+piped to HarbourBridge, `stdin` is not seekable, and so we write the output to
+a temporary file. That temporary file is created via Go's ioutil.TempFile.
+On many systems, this creates a file in `/tmp`, which is sometimes configured
+with minimal space. A simple workaround is to separately run pg_dump/mysqldump
 and write its output to a file in a directory with sufficient space. For example,
 if the current working directory has space, then:
 
@@ -453,9 +485,9 @@ environment variable.
 #### 3.2 Unparsable dump output
 
 HarbourBridge uses the [pg_query_go](https://github.com/pganalyze/pg_query_go)
-library for parsing pg_dump and [pingcap parser](https://github.com/pingcap/parser) 
-for parsing mysqldump. It is possible that the pg_dump/mysqldump output is 
-corrupted or uses features that aren't parseable. Parsing errors should 
+library for parsing pg_dump and [pingcap parser](https://github.com/pingcap/parser)
+for parsing mysqldump. It is possible that the pg_dump/mysqldump output is
+corrupted or uses features that aren't parseable. Parsing errors should
 generate an error message of the form `Error parsing last 54321 line(s) of input`.
 
 #### 3.2 Credentials problems
@@ -463,7 +495,7 @@ generate an error message of the form `Error parsing last 54321 line(s) of input
 HarbourBridge uses standard Google Cloud credential mechanisms for accessing
 Cloud Spanner. If this is mis-configured, you may see errors containing
 "unauthenticated", or "cannot fetch token", or "could not find default
-credentials". You might need to run `gcloud auth application-default login`. 
+credentials". You might need to run `gcloud auth application-default login`.
 See the [Before you begin](#before-you-begin) section for details.
 
 #### 3.4 Can't create database
diff --git a/cmd/README.dynamodb.md b/cmd/README.dynamodb.md
deleted file mode 100644
index 881b95b..0000000
--- a/cmd/README.dynamodb.md
+++ /dev/null
@@ -1,147 +0,0 @@
-# HarbourBridge: Turnkey DynamoDB-to-Spanner Evaluation
-
-HarbourBridge is a stand-alone open source tool for Cloud Spanner evaluation,
-using data from an existing PostgreSQL or MySQL or DynamoDB database. This
-README provides details of the tool's DynamoDB capabilities. For general
-HarbourBridge information see this [README](https://github.com/cloudspannerecosystem/harbourbridge#harbourbridge-turnkey-spanner-evaluation).
-
-## Example DynamoDB Usage
-
-The following examples assume `harbourbridge` has been added to your PATH
-environment variable.
-
-Before running HarbourBridge, make sure that you have
-[set up your AWS credentials/region](https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html)
-correctly (set the environment variables `AWS_ACCESS_KEY_ID`, 
-`AWS_SECRET_ACCESS_KEY`, `AWS_REGION`). HarbourBridge accesses your 
-DynamoDB database via the aws go sdk package. If you use a custom endpoint
-for dynamodb, you can specify that using the environment variable 
-`DYNAMODB_ENDPOINT_OVERRIDE`.
-
-```sh
-export DYNAMODB_ENDPOINT_OVERRIDE=http://dynamodb.us-west-2.amazonaws.com`
-```
-
-To use the tool on a DynamoDB database (it will migrate all tables),
-run
-
-```sh
-harbourbridge schema -source=dynamodb
-```
-
-### Source profile (`-source-profile`)
-
-HarbourBridge accepts the following options for --source-profile for DynamoDB,
-specified as "key1=value1,key2=value,..." pairs:
-
-`-schema-sample-size` Due to the schemaless nature of DynamoDB, the tool infers
-the schema based on a certain amount of sampled data, by default, 100000 rows.
-If a table has more rows than the default value, we only use 100000 rows for 
-estimating the schema. This flag lets you specify the number of rows to use for
-inferring schema. The default value is 100,000.
-
-```sh
-harbourbridge schema -source=dynamodb -source-profile="schema-sample-size=500000"
-```
-
-## Schema Conversion
-
-The HarbourBridge tool maps DynamoDB types to Spanner types as follows:
-
-| PostgreSQL Type    | Spanner Type               | Notes                                     |
-| ------------------ | -------------------------- | ----------------------------------------- |
-| `Number`           | `NUMERIC` or `STRING`      | defaults to NUMERIC, otherwise, STRING    |
-| `String`           | `STRING`                   |                                           |
-| `Boolean`          | `BOOL`                     |                                           |
-| `Binary`           | `BYTES`                    |                                           |
-| `Null`             | A nullable column type     |                                           |
-| `List`             | `STRING`                   | json encoding                             |
-| `Map`              | `STRING`                   | json encoding                             |
-| `StringSet`        | `ARRAY<STRING>`            |                                           |
-| `NumberSet`        | `ARRAY<NUMERIC or STRING>` |                                           |
-| `BinarySet`        | `ARRAY<BYTES>`             |                                           |
-
-We discuss these, as well as other limits and notes on schema conversion, in the
-following sections.
-
-### Schema Inference
-
-DynamoDB is a schemaless database: other than a primary index and optional
-secondary index, column names and types are essentially unconstrained
-and can vary from one row to the next.
-
-However, many customers use DynamoDB in a consistent, structured way
-with a fairly well defined set of columns and types. Our HarbourBridge support
-for DynamoDB focuses on this use-case, and we construct a Spanner schema
-by inspecting table data.
-
-For small tables, we inspect all rows of the table. For large tables, scanning
-the entire table would be extremely expensive and slow, and so we only inspect
-the first N rows (defined by the flag `schema-sample-size`) from the table scan.
-While DynamoDB doesn't return scan results in order, they might not be a truly
-random sample of rows. However, the alternative of randomly sampling rows
-would be much more expensive.
-
-Columns with consistent types are assigned Spanner types as detailed below.
-Columns without a consistent type are mapped to STRING.
-
-#### `Number`
-
-In most cases, we map the Number type in DynamoDB to Spanner's Numeric type.
-However, since the [range of Numeric](https://cloud.google.com/spanner/docs/storing-numeric-data)
-in Cloud Spanner is smaller than the [range of Number](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.NamingRulesDataTypes.html)
-in DynamoDB, this conversion could result in out of range with potential
-precision loss. To address this possibility, we try to convert the sample data,
-and if it consistently fails, we choose STRING type for the column.
-
-#### `Null` Data Type
-
-In DynamoDB, a column can have a Null data type that represents an unknown or
-undefined state. Also, each row defines its own schema for columns (not for
-primary keys). So columns can be absent in rows. 
-
-We treat the above two cases the same as a Null value in Cloud Spanner. The
-cases that a column contains a Null value or a column is not present is an
-indication that this column should be nullable.
-
-#### `List` and `Map`
-
-In Cloud Spanner, the most similar type to List and Map is
-[STRUCT](https://cloud.google.com/spanner/docs/data-types#struct_type), but it
-is not a valid column type (available for query but not for storage).
-Therefore, we encode them into a json string. 
-
-#### Occasional Errors
-
-To prevent a few spurious rows from impacting schema construction, we define an
-error threshold: when building a type for a column, if the percentage of rows
-with a specific data type is lower than or equal to an extremely low
-value (0.1%), then we treat those rows as likely errors. Such rows are ignored
-for schema construction: their type is not considered a candidate type for the
-column.
-
-#### Multi-type Columns
-
-For a special scenario, we may get a column that has equal distribution of two
-data types. E.g., a column has 40% rows in String and 60% rows in Number. If we
-choose Number as its type, it may lead to 40% data loss in the data conversion.
-In the migration, we define a conflicting threshold on rows (after removing Null
-data types and rows that the column is not present). By default, the conflicting
-threshold is 5% and if the percentages of two or more data types are greater
-than it, we would consider that the column has conflicting data types. As a safe
-choice, we define this column as a STRING type in Cloud Spanner. 
-
-## Data Conversion
-
-### A Scan for Entire Table
-
-Data conversion proceeds table by table. For each table, we use the Scan API to
-read data. Each read has a size limit up to 1MB. By using the returned token, we
-make a subsequent call to continue retrieving data from the table.
-
-The row result contains the data type and data itself. According to our
-[inferred schema](#schema-inference), we will parse the row to a format that
-Cloud Spanner can support. If the value parsing fails, we would drop the entire
-row and record it as bad data in the report. If a column does not appear or 
-column has a NULL data type, we would process this as a NULL value in 
-Cloud Spanner. 
diff --git a/cmd/README.md b/cmd/README.md
deleted file mode 100644
index 3c9cc05..0000000
--- a/cmd/README.md
+++ /dev/null
@@ -1,527 +0,0 @@
-# HarbourBridge: Spanner Evaluation and Migration
-
-[![cloudspannerecosystem](https://circleci.com/gh/cloudspannerecosystem/harbourbridge.svg?style=svg)](https://circleci.com/gh/cloudspannerecosystem/harbourbridge)
-
-HarbourBridge is a stand-alone open source tool for Cloud Spanner evaluation and
-migration, using data from an existing PostgreSQL, MySQL or DynamoDB database. 
-The tool ingests schema and data from either a pg_dump/mysqldump file or directly 
-from the source database, and supports both schema and data migration. For schema
-migration, HarbourBridge automatically builds a Spanner schema from the schema
-of the source database. This schema can be customized using the HarbourBridge
-schema assistant. For data migration, HarbourBridge creates a new Spanner
-database using the Spanner schema built during schema migration, and populates
-it with data from the source database.
-
-For more details on schema customization and use of the schema assistant, see
-[web/README](web/README.md). The rest of this README describes the command-line
-capabilities of HarbourBridge.
-
-HarbourBridge is designed to simplify Spanner evaluation and migration, and in
-particular for migrating moderate-size PostgreSQL/MySQL/DynamoDB datasets to Spanner 
-(up to about 100GB). Many features of PostgreSQL/MySQL, especially those that don't
-map directly to Spanner features, are ignored, e.g. stored functions and
-procedures, and sequences. Types such as integers, floats, char/text, bools,
-timestamps, and (some) array types, map fairly directly to Spanner, but many
-other types do not and instead are mapped to Spanner's `STRING(MAX)`.
-In the case of DynamoDB, the schema is inferred based on a certain 
-amount of sampled data.
-
-View HarbourBridge as a way to get up and running fast, so you can focus on
-critical things like tuning performance and getting the most out of
-Spanner. Expect that you'll need to tweak and enhance what HarbourBridge
-produces.
-
-Below are some quick starter examples on how to run HarbourBridge. You can look
-at specific sections for PostgreSQL, MySQL etc for detailed examples.
-
-For example, to generate Spanner schema from a PostgreSQL dump file, run
-
-```sh
-# Generate pg_dump file.
-pg_dump mydb > mydb.pg_dump
-
-harbourbridge schema -source=postgresql -source-profile="format=dump" < mydb.pg_dump
-# Or, (By default, format is "dump" when specifying input files)
-harbourbridge schema -source=postgresql < mydb.pg_dump
-# Or, you can specify full path to the dump file
-harbourbridge schema -source=postgresql -source-profile="format=dump,file=<full path to dump file>"
-# You can also specify path to a file stored in GCS (Google Cloud Storage)
-harbourbridge schema -source=postgresql -source-profile="format=dump,file=gs://<bucket_name>/<file_path>"
-
-# Target is Spanner by default, but you can also specify it explicitly.
-harbourbridge schema -source=postgresql -source-profile="format=dump" -target=spanner < mydb.pg_dump
-```
-
-Similarly, you can use the tool for other source databases.
-
-```sh
-# Generate Spanner schema from a MySQL dump file
-mysqldump mydb > mydb.mysqldump
-harbourbridge schema -source=mysql < mydb.mysqldump
-```
-
-Generate Spanner schema by directly connecting to a DynamoDB database,
-```sh
-# Set the environment variables `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`
-# , `AWS_REGION` and then run,
-harbourbridge schema -source=dynamodb
-```
-
-Note: HarbourBridge accepts pg_dump/mysqldump's standard plain-text format, 
-but not archive or custom formats. More details on usage can be found in 
-[Example usage](#example-usage) section.
-
-HarbourBridge automatically determines the cloud project to use, and generates
-a new Spanner database name (prefixed with `{source}_` and today's date).
-Command-line flags can be used to explicitly set the Spanner instance or
-database name. See [Options](#options).
-
-**WARNING: Please check that permissions for the Spanner instance used by
-HarbourBridge are appropriate. Spanner manages access control at the database
-level, and the database created by HarbourBridge will inherit default
-permissions from the instance. All data written by HarbourBridge is visible to
-anyone who can access the created database.**
-
-As it processes the data, HarbourBridge reports on progress, provides
-stats on the schema and data conversion steps, and an overall assessment of the
-quality of the conversion. It also generates a schema file, report file and
-a session file (and a bad-data file if data was dropped). See
-[Files Generated by HarbourBridge](#files-generated-by-harbourbridge). Details
-of how source database's schema is mapped to Spanner can be found in the
-[Schema Conversion](#schema-conversion) section.
-
-This tool is part of the Cloud Spanner Ecosystem, a community contributed and
-supported open source repository. Please [report
-issues](https://github.com/cloudspannerecosystem/harbourbridge/issues) and send
-pull requests. See the [HarbourBridge
-Whitepaper](https://github.com/cloudspannerecosystem/harbourbridge/blob/master/whitepaper.md)
-for a discussion of our plans for the tool.
-
-Note that the HarbourBridge tool is not an officially supported Google product
-and is not officially supported as part of the Cloud Spanner product.
-
-## Quickstart Guide
-
-### Before you begin
-
-Complete the steps described in
-[Set up](https://cloud.google.com/spanner/docs/getting-started/set-up), which
-covers creating and setting a default Google Cloud project, enabling billing,
-enabling the Cloud Spanner API, and setting up OAuth 2.0 to get authentication
-credentials to use the Cloud Spanner API.
-
-In particular, ensure that you run
-
-```sh
-gcloud auth application-default login
-```
-
-to set up your local development environment with authentication credentials.
-
-Set the GCLOUD_PROJECT environment variable to your Google Cloud project ID:
-
-```sh
-export GCLOUD_PROJECT=my-project-id
-```
-
-If you do not already have a Cloud Spanner instance, or you want to use a
-separate instance specifically for running HarbourBridge, then create a Cloud
-Spanner instance by following the "Create an instance" instructions on the
-[Quickstart using the console](https://cloud.google.com/spanner/docs/quickstart-console)
-guide. HarbourBridge will create a database for you, but it will not create a
-Spanner instance.
-
-Install Go ([download](https://golang.org/doc/install)) on your development
-machine if it is not already installed, configure the GOPATH environment
-variable if it is not already configured, and
-[test your installation](https://golang.org/doc/install#testing).
-
-### Installing HarbourBridge
-
-Download the tool to your machine and install it.
-
-```sh
-GO111MODULE=on go get github.com/cloudspannerecosystem/harbourbridge
-```
-
-The tool should now be installed as `$GOPATH/bin/harbourbridge`.
-See the [Troubleshooting Guide](#troubleshooting-guide) if you run
-into any issues.
-
-### Running HarbourBridge
-
-To use the tool on a PostgreSQL database called mydb, run
-
-```sh
-pg_dump mydb > mydb.pg_dump
-$GOPATH/bin/harbourbridge schema -source=postgresql < mydb.pg_dump
-```
-
-To use the tool on a MySQL database called mydb, run
-
-```sh
-mysqldump mydb > mydb.mysqldump
-$GOPATH/bin/harbourbridge schema -source=mysql < mydb.mysqldump
-```
-
-To use the tool on a DynamoDB database, run
-
-```sh
-$GOPATH/bin/harbourbridge schema -source=dynamodb
-```
-More details on running harbourbridge can be found in [Example usage](#example-usage) section.
-
-This command will use the cloud project specified by the `GCLOUD_PROJECT`
-environment variable, automatically determine the Cloud Spanner instance
-associated with this project, convert the source schema to a Spanner schema 
-(For MySQL/Postgres) or infer a schema from the DynamoDB instance, create a
-new Cloud Spanner database with this schema, and finally, populate this new
-database with the data from the source database. 
-If the project has multiple instances, then list of available instances 
-will be shown and you will have to pick one of the available instances and 
-set the `--instance` flag in target-profile. The new Cloud Spanner database
-will have a name of the form `{SOURCE}_{DATE}_{RANDOM}`, where`{SOURCE}` is
-the source specified,`{DATE}`is today's date, and`{RANDOM}` is a random
-suffix for uniqueness.
-
-See the [Troubleshooting Guide](#troubleshooting-guide) for help on debugging
-issues.
-
-HarbourBridge also [generates several files](#files-generated-by-harbourbridge)
-when it runs: a schema file, a report file (with detailed analysis of the
-conversion), a session file and a bad data file (if any data was dropped).
-
-### Sample Dump Files
-
-If you don't have ready access to a PostgreSQL or MySQL database, some example
-dump files can be found [here](examples). The files
-[cart.pg_dump](examples/cart.pg_dump) and
-[cart.mysqldump](examples/cart.mysqldump) contain pg_dump and mysqldump output
-for a very basic shopping cart application (just two tables, one for products
-and one for user carts). The files [singers.pg_dump](examples/singers.pg_dump)
-and [singers.mysqldump](examples/singers.mysqldump) contain pg_dump and
-mysqldump output for a version of the [Cloud Spanner
-singers](https://cloud.google.com/spanner/docs/schema-and-data-model#creating_a_table)
-example. To use HarbourBridge on cart.pg_dump, download the file locally and
-run
-
-```
-$GOPATH/bin/harbourbridge schema -source=postgresql < cart.pg_dump
-```
-
-### Verifying Results
-
-Once the tool has completed, you can verify the new database and its content
-using the Google Cloud Console. Go to the [Cloud Spanner Instances
-page](https://console.cloud.google.com/spanner/instances), select your Spanner
-instance, and then find the database created by HarbourBridge and select
-it. This will list the tables created by HarbourBridge. Select a table, and take
-a look at its schema and data. Next, go to the query page, and try
-some SQL statements. For example
-
-```
-SELECT COUNT(*) from mytable
-```
-
-to check the number of rows in table `mytable`.
-
-### Next Steps
-
-The tables created by HarbourBridge provide a starting point for evaluation of
-Spanner. While they preserve much of the core structure of your PostgreSQL/MySQL
-schema and data, many key features have been dropped, including functions, 
-sequences, procedures,triggers, and views. For DynamoDB, the conversion from 
-schemaless to schema is focused on the use-case where customers use DynamoDB in 
-a consistent, structured way with a fairly well defined set of columns and types.
-
-As a result, the out-of-the-box performance you get from these tables could be
-slower than what you get from PostgreSQL/MySQL/DynamoDB. 
-
-To improve performance, also consider using [Interleaved
-Tables](https://cloud.google.com/spanner/docs/schema-and-data-model#creating-interleaved-tables)
-to tune performance.
-
-View HarbourBridge as a base set of functionality for Spanner evalution that can
-be readily expanded. Consider forking and modifying the codebase to add the
-functionality you need. Please [file
-issues](https://github.com/cloudspannerecosystem/harbourbridge/issues) and send
-PRs for fixes and new functionality. See our backlog of [open
-issues](https://github.com/cloudspannerecosystem/harbourbridge/issues). Our
-plans and aspirations for developing HarbourBridge further are outlined in the
-[HarbourBridge
-Whitepaper](https://github.com/cloudspannerecosystem/harbourbridge/blob/master/whitepaper.md).
-
-You can also change the way HarbourBridge behaves by directly editing the
-pg_dump/mysqldump output. For example, suppose you want to try out different 
-primary keys for a table. First run pg_dump/mysqldump and save the output to 
-a file. Then modify (or add) the relevant 
-`ALTER TABLE ... ADD CONSTRAINT ... PRIMARY KEY ...` statement in the 
-pg_dump/mysqldump output file so that the primary keys match what you need. 
-Then run HarbourBridge on the modified pg_dump/mysqldump output.
-
-## Files Generated by HarbourBridge
-
-HarbourBridge generates several files as it runs:
-
-- Schema file (ending in `schema.txt`): contains the generated Spanner
-  schema, interspersed with comments that cross-reference to the relevant
-  PostgreSQL/MySQL schema definitions.
-
-- Session file (ending in `session.json`): contains all schema and data
-  conversion state endcoded as JSON. It is basically a snapshot of the session.
-
-- Report file (ending in `report.txt`): contains a detailed analysis of the
-  PostgreSQL/MySQL to Spanner migration, including table-by-table stats and an
-  analysis of PostgreSQL/MySQL types that don't cleanly map onto Spanner types. 
-  Note that PostgreSQL/MySQL types that don't have a corresponding Spanner type 
-  are mapped to STRING(MAX).
-
-- Bad data file (ending in `dropped.txt`): contains details of data
-  that could not be converted and written to Spanner, including sample
-  bad-data rows. If there is no bad-data, this file is not written (and we
-  delete any existing file with the same name from a previous run).
-
-By default, these files are prefixed by the name of the Spanner database (with a
-dot separator). The file prefix can be overridden using the `-prefix`
-[option](#options).
-
-## Command line flags
-
-`-source` Required flag. Specifies the source database. Supported source
-databases are _'postgres'_, _'mysql'_ and _'dynamodb'_.
-
-`-target` Optional flag. Specifies the target database. Defaults to _'spanner'_ 
-, which is the only supported target database today.
-
-`-prefix` Specifies a file prefix for the report, schema, and bad-data files
-written by the tool. If no file prefix is specified, the name of the Spanner
-database (plus a '.') is used.
-
-`-v` or `-verbose` Specifies verbose mode. This will cause HarbourBridge to
-output detailed messages about the conversion.
-
-`-skip-foreign-keys` Controls whether we add foreign key constraints after
-data migration is complete. This flag cannot be used with schema-only mode,
-and does not affect the generation of foreign key statements during schema
-processing i.e. foreign key constraints will still appear in the generated 
-Spanner DDL files.
-
-`-session` Specifies a session file that contains all schema and data 
-conversion state endcoded as JSON.
-
-### Source profile (`-source-profile`)
-
-HarbourBridge accepts the following options for --source-profile,
-specified as "key1=value1,key2=value,..." pairs:
-
-`-file` Specifies the full path of the file to use for reading source database
-schema and/or data. This can also point to a file stored in GCS e.g., 
-gcs://<bucket_name>/<file_path>. This flag is optional, and file can also be
-piped to stdin, if available locally.
-
-`-format` Specifies the format of the file. This flag is also optional, and
-defaults to `dump`. This may be extended in future to support other formats
-such as `csv`, `avro` etc.
-
-### Target profile (`-target-profile`)
-
-HarbourBridge accepts the following options for --target-profile,
-specified as "key1=value1,key2=value,..." pairs:
-
-`-dbname` Specifies the name of the Spanner database to create. This must be a
-new database. If dbname is not specified, HarbourBridge creates a new unique
-dbname.
-
-`-instance` Specifies the Spanner instance to use. The new database will be
-created in this instance. If not specified, the tool automatically determines an
-appropriate instance using gcloud.
-
-## Example Usage
-
-Details on HarbourBridge example usage can be found here: 
-- [PostgreSQL example usage](sources/postgres/README.md#example-postgresql-usage)
-- [MySQL example usage](sources/mysql/README.md#example-mysql-usage)
-- [DynamoDB example usage](sources/dynamodb/README.md#example-dynamodb-usage)
-
-
-## Schema Conversion
-
-Details on HarbourBridge schema conversion can be found here:
-- [PostgreSQL schema conversion](sources/postgres/README.md#schema-conversion)
-- [MySQL schema conversion](sources/mysql/README.md#schema-conversion)
-- [DynamoDB schema conversion](sources/dynamodb/README.md#schema-conversion)
-
-## Data Conversion
-
-HarbourBridge converts PostgreSQL/MySQL/DynamoDB data to Spanner data based on 
-the Spanner schema it constructs. Conversion for most data types is fairly 
-straightforward, but several types deserve discussion. Note that HarbourBridge 
-is not intended for databases larger than about 100GB. Details on HarbourBridge 
-data conversion can be found here:
-- [PostgreSQL data conversion](sources/postgres/README.md#data-conversion)
-- [MySQL data conversion](sources/mysql/README.md#data-conversion)
-- [DynamoDB data conversion](sources/dynamodb/README.md#data-conversion)
-
-## Troubleshooting Guide
-
-HarbourBridge is written using the Go module system, and so it must be
-installed in [module-aware mode](https://golang.org/cmd/go/#hdr-Module_support).
-This can be achived by setting the environment variable `GO111MODULE` to `on`
-before calling "go get" e.g.
-
-```sh
-GO111MODULE=on go get github.com/cloudspannerecosystem/harbourbridge
-```
-
-As an alternative to "go get", you can make a copy of the HarbourBridge
-codebase from the github repository and use "go run":
-
-```sh
-git clone https://github.com/cloudspannerecosystem/harbourbridge
-cd harbourbridge
-pg_dump mydb > mydb.pg_dump
-go run github.com/cloudspannerecosystem/harbourbridge schema -source=postgresql < mydb.pg_dump
-```
-
-This workflow also allows you to modify or customize the HarbourBridge codebase.
-
-The following steps can help diagnose common issues encountered while running
-HarbourBridge.
-
-### 1. Verify driver configuration
-
-First, check that the driver is correctly configured to connect to your
-database. Driver configuration varies depending on the driver used.
-
-#### 1.1 pg_dump
-
-If you are using pg_dump (-driver=pg_dump), check that pg_dump is
-correctly configured to connect to your PostgreSQL
-database. Note that pg_dump uses the same options as psql to connect to your
-database. See the [psql](https://www.postgresql.org/docs/9.3/app-psql.html) and
-[pg_dump](https://www.postgresql.org/docs/9.3/app-pgdump.html) documentation.
-
-Access to a PostgreSQL database is typically configured using the
-_PGHOST_, _PGPORT_, _PGUSER_, _PGDATABASE_ environment variables,
-which are standard across PostgreSQL utilities.
-
-It is also possible to configure access via pg_dump's command-line options
-`--host`, `--port`, and `--username`.
-
-#### 1.2 Direct access to PostgreSQL
-
-In this case, HarbourBridge connects directly to the PostgreSQL
-database to retrieve table schema and data. Set the `-driver=postgres`
-and provide the environment variables `PGHOST`, `PGPORT`, `PGUSER`, 
-`PGDATABASE`. Password can be specified either in the `PGPASSWORD`
-environment variable or provided at the password prompt.
-
-#### 1.3 mysqldump
-
-If you are using mysqldump (-driver=mysqldump), check that mysqldump is
-correctly configured to connect to your MySQL
-database via the command-line options `--host`, `--port`, and `--user`.
-Note that mysqldump uses the same options as mysql to connect to your
-database. See the [mysql](https://dev.mysql.com/doc/refman/8.0/en/mysql-commands.html)
-and [mysqldump](https://dev.mysql.com/doc/refman/8.0/en/mysqldump.html) documentation.
-
-#### 1.4 Direct access to MySQL
-
-In this case, HarbourBridge connects directly to the MySQL
-database to retrieve table schema and data. Set the `-driver=mysql`
-and provide the environment variables `MYSQLHOST`, `MYSQLPORT`, 
-`MYSQLUSER`, `MYSQLDATABASE`. Password can be specified either 
-in the `MYSQLPWD` environment variable or provided at the password 
-prompt.
-
-
-#### 1.5 Direct access to DynamoDB
-
-In this case, HarbourBridge connects directly to the DynamoDB
-database to retrieve table schema and data. Set the `-driver=dynamodb`
-and provide the environment variables `AWS_ACCESS_KEY_ID`, 
-`AWS_SECRET_ACCESS_KEY`, `AWS_REGION`. If you use a custom endpoint for 
-dynamodb, you can specify that using the environment variable 
-`DYNAMODB_ENDPOINT_OVERRIDE`.
-
-### 2. Verify dump output
-
-Next, verify that pg_dump/mysqldump is generating plain-text output. If your 
-database is small, try running
-
-```sh
-{ pg_dump/mysqldump } > file
-```
-
-and look at the output file. It should be a plain-text file containing SQL
-commands. If your database is large, consider just dumping the schema via the
-`--schema-only` for pg_dump and `--no-data` for mysqldump command-line option.
-
-pg_dump/mysqldump can export data in a variety of formats, but HarbourBridge 
-only accepts `plain` format (aka plain-text). See the
-[pg_dump documentation](https://www.postgresql.org/docs/9.3/app-pgdump.html) and
-[mysqldump documentation](https://dev.mysql.com/doc/refman/8.0/en/mysqldump.html) 
-for details about formats.
-
-### 3. Debugging HarbourBridge
-
-The HarbourBridge tool can fail for a number of reasons.
-
-#### 3.1 No space left on device
-
-HarbourBridge needs to read the pg_dump/mysqldump output twice, once to build 
-a schema and once for data ingestion. When pg_dump/mysqldump output is directly
-piped to HarbourBridge, `stdin` is not seekable, and so we write the output to 
-a temporary file. That temporary file is created via Go's ioutil.TempFile. 
-On many systems, this creates a file in `/tmp`, which is sometimes configured 
-with minimal space. A simple workaround is to separately run pg_dump/mysqldump 
-and write its output to a file in a directory with sufficient space. For example,
-if the current working directory has space, then:
-
-```sh
-{ pg_dump/mysqldump } > tmpfile
-harbourbridge < tmpfile
-```
-
-Make sure you cleanup the tmpfile after HarbourBridge has been run. Another
-option is to set the location of Go's TempFile e.g. by setting the `TMPDIR`
-environment variable.
-
-#### 3.2 Unparsable dump output
-
-HarbourBridge uses the [pg_query_go](https://github.com/pganalyze/pg_query_go)
-library for parsing pg_dump and [pingcap parser](https://github.com/pingcap/parser) 
-for parsing mysqldump. It is possible that the pg_dump/mysqldump output is 
-corrupted or uses features that aren't parseable. Parsing errors should 
-generate an error message of the form `Error parsing last 54321 line(s) of input`.
-
-#### 3.2 Credentials problems
-
-HarbourBridge uses standard Google Cloud credential mechanisms for accessing
-Cloud Spanner. If this is mis-configured, you may see errors containing
-"unauthenticated", or "cannot fetch token", or "could not find default
-credentials". You might need to run `gcloud auth application-default login`. 
-See the [Before you begin](#before-you-begin) section for details.
-
-#### 3.4 Can't create database
-
-In this case, the error message printed by the tool should help identify the
-cause. It could be an API permissions issue. For example, the Cloud Spanner API
-may not be appropriately configured. See [Before you begin](#before-you-begin)
-section for details. Alternatively, you have have hit the limit on the number of
-databases per instances (currently 100). This can occur if you re-run the
-HarbourBridge tool many times, since each run creates a new database. In this
-case you'll need to [delete some
-databases](https://cloud.google.com/spanner/docs/getting-started/go/#delete_the_database).
-
-### 4. Database-Specific Issues
-
-The schema, report, and bad-data files [generated by
-HarbourBridge](#files-generated-by-harbourbridge) contain detailed information
-about the schema and data conversion process, including issues and problems
-encountered.
-
-### 5. Reporting Issues
-
-If you are having problems with HarbourBridge, please [submit an
-issue](https://github.com/cloudspannerecosystem/harbourbridge/issues).
diff --git a/cmd/README.mysql.md b/cmd/README.mysql.md
deleted file mode 100644
index 5e9394c..0000000
--- a/cmd/README.mysql.md
+++ /dev/null
@@ -1,268 +0,0 @@
-# HarbourBridge: Turnkey MySQL-to-Spanner Evaluation
-
-HarbourBridge is a stand-alone open source tool for Cloud Spanner evaluation,
-using data from an existing PostgreSQL or MySQL database. This README provides
-details of the tool's MySQL capabilities. For general HarbourBridge information
-see this [README](https://github.com/cloudspannerecosystem/harbourbridge#harbourbridge-turnkey-spanner-evaluation).
-
-## Example MySQL Usage
-
-The following examples assume `harbourbridge` has been added to your PATH
-environment variable.
-
-HarbourBridge can either be used with mysqldump or it can be run directly
-on a MySQL database (via go's database/sql package).
-
-### Using HarbourBridge with mysqldump
-
-To use HarbourBridge on a MySQL database called mydb using mysqldump, run:
-
-```sh
-mysqldump mydb | harbourbridge schema -source=mysql
-```
-
-The tool can also be applied to an existing mysqldump file:
-
-```sh
-harbourbridge schema -source=mysql < my_mysqldump_file
-```
-
-To specify a particular Spanner instance to use during data migration, run:
-
-```sh
-harbourbridge data -source=mysql -target-profile="instance=my-spanner-instance" < my_mysqldump_file
-```
-
-By default, HarbourBridge will generate a new Spanner database name to populate.
-You can override this and specify the database name to use by:
-
-```sh
-harbourbridge data -source=mysql -target-profile="dbname=my-spanner-database-name" < my_mysqldump_file
-```
-
-HarbourBridge generates a report file, a schema file, and a bad-data file (if
-there are bad-data rows). You can control where these files are written by
-specifying a file prefix. For example,
-
-```sh
-harbourbridge schema -prefix=mydb. -source=mysql < my_mysqldump_file
-```
-
-will write files `mydb.report.txt`, `mydb.schema.txt`, and
-`mydb.dropped.txt`. The prefix can also be a directory. For example,
-
-```sh
-harbourbridge schema -prefix=~/spanner-eval-mydb/ -source=mysql < my_mysqldump_file
-```
-
-would write the files into the directory `~/spanner-eval-mydb/`. Note
-that HarbourBridge will not create directories as it writes these files.
-
-### Directly connecting to a MySQL database
-
-To use the tool directly on a MySQL database called mydb, run
-
-```sh
-harbourbridge schema -source=mysql
-```
-
-It is assumed that _MYSQLHOST_, _MYSQLPORT_, _MYSQLUSER_, _MYSQLDATABASE_ environment
-variables are set. Password can be specified either in the _MYSQLPWD_ environment
-variable or provided at the password prompt.
-
-Note that all of the options described in the previous section on read from a
-mysqldump file can also be used when directly connecting to a mysql database.
-
-## Schema Conversion
-
-The HarbourBridge tool maps MySQL types to Spanner types as follows:
-
-| MySQL Type                                        | Spanner Type    | Notes                           |
-| ------------------------------------------------- | --------------- | ------------------------------- |
-| `BOOL`, `BOOLEAN`,<br/>`TINYINT(1)`               | `BOOL`          |                                 |
-| `BIGINT`                                          | `INT64`         |                                 |
-| `BINARY`, `VARBINARY`                             | `BYTES(MAX)`    |                                 |
-| `BLOB`, `MEDIUMBLOB`,<br/>`TINYBLOB`, `LONGBLOB`  | `BYTES(MAX)`    |                                 |
-| `BIT`                                             | `BYTES(MAX)`    |                                 |
-| `CHAR`                                            | `STRING(1)`     | CHAR defaults to length 1       |
-| `CHAR(N)`                                         | `STRING(N)`     | c                               |
-| `DATE`                                            | `DATE`          |                                 |
-| `DATETIME`                                        | `TIMESTAMP`     | t                               |
-| `DECIMAL`, `NUMERIC`                              | `NUMERIC`       | p                               |
-| `DOUBLE`                                          | `FLOAT64`       |                                 |
-| `ENUM`                                            | `STRING(MAX)`   |                                 |
-| `FLOAT`                                           | `FLOAT64`       | s                               |
-| `INTEGER`, `MEDIUMINT`,<br/>`TINYINT`, `SMALLINT` | `INT64`         | s                               |
-| `JSON`                                            | `STRING(MAX)`   |                                 |
-| `SET`                                             | `ARRAY<STRING>` | SET only supports string values |
-| `TEXT`, `MEDIUMTEXT`,<br/>`TINYTEXT`, `LONGTEXT`  | `STRING(MAX)`   |                                 |
-| `TIMESTAMP`                                       | `TIMESTAMP`     |                                 |
-| `VARCHAR`                                         | `STRING(MAX)`   |                                 |
-| `VARCHAR(N)`                                      | `STRING(N)`     | c                               |
-
-Spanner does not support `spatial` datatypes of MySQL. Along with `spatial`
-datatypes, all other types map to `STRING(MAX)`. Some of the mappings in this
-table represent potential changes of precision (marked p), differences in
-treatment of timezones (marked t), differences in treatment of fixed-length
-character types (marked c), and changes in storage size (marked s). We discuss
-these, as well as other limits and notes on schema conversion, in the following
-sections.
-
-### `DECIMAL` and `NUMERIC`
-
-[Spanner's NUMERIC
-type](https://cloud.google.com/spanner/docs/data-types#decimal_type) can store
-up to 29 digits before the decimal point and up to 9 after the decimal point.
-MySQL's NUMERIC type can potentially support higher precision than this, so
-please verify that Spanner's NUMERIC support meets your application needs.  Note
-that in MySQL, NUMERIC is implemented as DECIMAL, so the remarks about DECIMAL
-apply equally to NUMERIC.
-
-### `TIMESTAMP` and `DATETIME`
-
-MySQL has two timestamp types: `TIMESTAMP` and `DATETIME`. Both provide
-microsecond resolution, but neither actually stores a timezone with the data.
-The key difference between the two types is that MySQL converts `TIMESTAMP` values
-from the current time zone to UTC for storage, and back from UTC to the current time
-zone for retrieval. This does not occur for `DATETIME` and data is returned without a
-timezone. For `TIMESTAMP`, timezone can be set by time zone offset parameter.
-
-Spanner has a single timestamp type. Data is stored as UTC (there is no separate
-timezone) Spanner client libraries convert timestamps to UTC before sending them
-to Spanner. Data is always returned as UTC. Spanner's timestamp type is
-essentially the same as `TIMESTAMP`, except that there is no analog of
-MySQL's timezone offset parameter.
-
-In other words, mapping MySQL `DATETIME` to `TIMESTAMP` is fairly
-straightforward, but care should be taken with MySQL `DATETIME` data
-because Spanner clients will not drop the timezone.
-
-### `CHAR(n)` and `VARCHAR(n)`
-
-The semantics of fixed-length character types differ between MySQL and
-Spanner. The `CHAR(n)` type in MySQL is right-padded with spaces. If a string
-value smaller than the limit is stored, spaces will be added to pad it out to
-the specified length. If a string longer than the specified length is stored,
-and the extra characters are all spaces, then it will be silently
-truncated. Moreover, trailing spaces are ignored when comparing two values. In
-constrast, Spanner does not give special treatment to spaces, and the specified
-length simply represents the maximum length that can be stored. This is close to
-the semantics of MySQL's `VARCHAR(n)`. However there are some minor
-differences. For example, even `VARCHAR(n)` has some special treatment of
-spaces: string with trailing spaces in excess of the column length are truncated
-prior to insertion and a warning is generated.
-
-### `SET`
-
-MySQL `SET` is a string object that can hold muliple values, each of which must be
-chosen from a list of permitted values specified when the table is created. `SET`
-is being mapped to Spanner type `ARRAY<STRING>`. Validation of `SET` element values
-will be dropped in Spanner. Thus for production use, validation needs to be done
-in the application.
-
-### `Spatial datatype`
-
-MySQL spatial datatypes are used to represent geographic feature.
-It includes `GEOMETRY`, `POINT`, `LINESTRING`, `POLYGON`, `MULTIPOINT`, `MULTIPOLYGON`
-and `GEOMETRYCOLLECTION` datatypes. Spanner does not support spatial data types.
-This datatype are currently mapped to standard `STRING` Spanner datatype.
-
-### Storage Use
-
-The tool maps several MySQL types to Spanner types that use more storage.
-For example, `SMALLINT` is a two-byte integer, but it maps to Spanner's `INT64`,
-an eight-byte integer.
-
-### Primary Keys
-
-Spanner requires primary keys for all tables. MySQL recommends the use of
-primary keys for all tables, but does not enforce this. When converting a table
-without a primary key, HarbourBridge will create a new primary key of type
-INT64. By default, the name of the new column is `synth_id`. If there is already
-a column with that name, then a variation is used to avoid collisions.
-
-### NOT NULL Constraints
-
-The tool preserves `NOT NULL` constraints. Note that Spanner does not require
-primary key columns to be `NOT NULL`. However, in MySQL, a primary key is a
-combination of `NOT NULL` and `UNIQUE`, and so primary key columns from
-MySQL will be mapped to Spanner columns that are both primary keys and `NOT NULL`.
-
-### Foreign Keys
-
-The tool maps MySQL foreign key constraints into Spanner foreign key constraints, and
-preserves constraint names where possible. Since Spanner doesn't support `ON DELETE`
-and `ON UPDATE` actions, we drop them.
-
-### Default Values
-
-Spanner does not currently support default values. We drop these
-MySQL features during conversion.
-
-### Secondary Indexes
-
-The tool maps MySQL secondary indexes to Spanner secondary indexes, and preserves
-constraint names where possible. Note that Spanner requires index key constraint
-names to be globally unique (within a database), but in MySQL they only have to be
-unique for a table, so we add a uniqueness suffix to a name if needed. The tool also
-maps `UNIQUE` constraint into `UNIQUE` secondary index. Note that due to limitations of our
-mysqldump parser, we are not able to handle key column ordering (i.e. ASC/DESC) in
-mysqldump files. All key columns in mysqldump files will be treated as ASC.
-
-### Other MySQL features
-
-MySQL has many other features we haven't discussed, including functions,
-sequences, procedures, triggers, (non-primary) indexes and views. The tool does
-not support these and the relevant statements are dropped during schema
-conversion.
-
-See
-[Migrating from MySQL to Cloud Spanner](https://cloud.google.com/solutions/migrating-mysql-to-spanner)
-for a general discussion of MySQL to Spanner migration issues.
-HarbourBridge follows most of the recommendations in that guide. The main
-difference is that we map a few more types to `STRING(MAX)`.
-
-## Data Conversion
-
-### Timestamps and Timezones
-
-As noted earlier when discussing [schema conversion of
-TIMESTAMP](#timestamp), there are some subtle differences in how timestamps are
-handled in MySQL and Spanner.
-
-During data conversion, MySQL `TIMESTAMP` values are converted to UTC and
-stored in Spanner. The conversion proceeds as follows. If the value has a
-timezone offset, that timezone is respected during the conversion to UTC. If the value
-does not have a timezone offset, then we look for any `set timezone` statements in the
-mysqldump output and use the timezone offset specified. Otherwise, we use '+00:00' timezone offset (UTC).
-
-### Strings, character set support and UTF-8
-
-Spanner requires that `STRING` values be UTF-8 encoded. All Spanner functions
-and operators that act on `STRING` values operate on Unicode characters rather
-than bytes. Since we map many MySQL types (including `TEXT` and `CHAR`
-types) to Spanner's `STRING` type, HarbourBridge is effectively a UTF-8 based
-tool.
-
-Note that the tool itself does not do any encoding/decoding or UTF-8 checks: it
-passes through data from mysqldump to Spanner. Internally, we use Go's string
-type, which supports UTF-8.
-
-### Spatial datatypes support
-
-As noted earlier when discussing [schema conversion of
-Spatial datatype](#spatial-datatype), Spanner does not support spatial datatypes and are
-mapped to `STRING(MAX)` Spanner type. Data conversion for spatial datatypes is different
-for both `mysql` and `mysqldump` driver.
-
-- MySQL information schema approach (`-driver=mysql`) : Data from MySQL is fetched using
-  'ST_AsText(g)' function which converts a value in internal geometry format to its WKT(Well-Known Text)
-  representation and returns the string result. This value will be stored as `STRING` in Spanner.
-- MySQL dump approach (`-driver=mysqldump`) : Mysqldump will have the internal geometry data in
-  binary format. It cannot be converted to WKT format and there is no proper method for mysqldump
-  generation of spatial datatypes also. Thus, this value will just be fetched as a `TEXT` type and
-  converted to Spanner type `STRING`.
-
-Note that mysql information schema approach would be the recommended approach for data conversion of
-spatial datatypes. For production use, you must store this data using standard data types, and implement
-any searching/filtering logic in the application layer.
diff --git a/cmd/README.postgres.md b/cmd/README.postgres.md
deleted file mode 100644
index ab78960..0000000
--- a/cmd/README.postgres.md
+++ /dev/null
@@ -1,257 +0,0 @@
-# HarbourBridge: Turnkey PostgreSQL-to-Spanner Evaluation
-
-HarbourBridge is a stand-alone open source tool for Cloud Spanner evaluation,
-using data from an existing PostgreSQL or MySQL database. This README provides
-details of the tool's PostgreSQL capabilities. For general HarbourBridge information
-see this [README](https://github.com/cloudspannerecosystem/harbourbridge#harbourbridge-turnkey-spanner-evaluation).
-
-## Example PostgreSQL Usage
-
-The following examples assume `harbourbridge` has been added to your PATH
-environment variable.
-
-HarbourBridge can either be used with pg_dump or it can be run directly
-on a PostgreSQL database (via go's database/sql package).
-
-### Using HarbourBridge with pg_dump
-
-To use HarbourBridge on a PostgreSQL database called mydb using pg_dump output,run:
-
-```sh
-pg_dump mydb | harbourbridge schema -source=postgresql
-```
-
-You can use one of `postgresql`, `postgres`, or `pg` to specify PostgreSQL as
-the source database.
-
-The tool can also be applied to an existing pg_dump file:
-
-```sh
-harbourbridge schema -source=postgres < my_pg_dump_file
-```
-
-To specify a particular Spanner instance to use during data migration, run:
-
-```sh
-harbourbridge data -source=postgres -target-profile="instance=my-spanner-instance" < my_pg_dump_file
-```
-
-By default, HarbourBridge will generate a new Spanner database name to populate.
-You can override this and specify the database name to use by:
-
-```sh
-harbourbridge data -source=postgres -target-profile="dbname=my-spanner-database-name,instance=my-spanner-instance" < my_pg_dump_file
-```
-
-HarbourBridge generates a report file, a schema file, and a bad-data file (if
-there are bad-data rows during data migration). You can control where these
-files are written by specifying a file prefix. For example,
-
-```sh
-harbourbridge schema -prefix=mydb. -source=pg < my_pg_dump_file
-```
-
-will write files `mydb.report.txt`, `mydb.schema.txt`, and
-`mydb.dropped.txt`. The prefix can also be a directory. For example,
-
-```sh
-harbourbridge schema -prefix=~/spanner-eval-mydb/ -source=pg < my_pg_dump_file
-```
-
-would write the files into the directory `~/spanner-eval-mydb/`. Note
-that HarbourBridge will not create directories as it writes these files.
-
-### Directly connecting to a PostgreSQL database
-
-To use the tool directly on a PostgresSQL database called mydb, run
-
-```sh
-harbourbridge schema -source=postgres
-```
-
-It is assumed that _PGHOST_, _PGPORT_, _PGUSER_, _PGDATABASE_ environment
-variables are set. Password can be specified either in the _PGPASSWORD_
-environment variable or provided at the password prompt.
-
-Note that all of the options described in the previous section when specifying
-a dump file can also be used when directly connecting to a PostgreSQL database.
-
-## Schema Conversion
-
-The HarbourBridge tool maps PostgreSQL types to Spanner types as follows:
-
-| PostgreSQL Type    | Spanner Type           | Notes                                     |
-| ------------------ | ---------------------- | ----------------------------------------- |
-| `BOOL`             | `BOOL`                 |                                           |
-| `BIGINT`           | `INT64`                |                                           |
-| `BIGSERIAL`        | `INT64`                | a                                         |
-| `BYTEA`            | `BYTES(MAX)`           |                                           |
-| `CHAR`             | `STRING(1)`            | CHAR defaults to length 1                 |
-| `CHAR(N)`          | `STRING(N)`            | c                                         |
-| `DATE`             | `DATE`                 |                                           |
-| `DOUBLE PRECISION` | `FLOAT64`              |                                           |
-| `INTEGER`          | `INT64`                | s                                         |
-| `NUMERIC`          | `NUMERIC`              | p                                         |
-| `REAL`             | `FLOAT64`              | s                                         |
-| `SERIAL`           | `INT64`                | a, s                                      |
-| `SMALLINT`         | `INT64`                | s                                         |
-| `TEXT`             | `STRING(MAX)`          |                                           |
-| `TIMESTAMP`        | `TIMESTAMP`            | t                                         |
-| `TIMESTAMPTZ`      | `TIMESTAMP`            |                                           |
-| `VARCHAR`          | `STRING(MAX)`          |                                           |
-| `VARCHAR(N)`       | `STRING(N)`            | c                                         |
-| `ARRAY(`pgtype`)`  | `ARRAY(`spannertype`)` | if scalar type pgtype maps to spannertype |
-
-All other types map to `STRING(MAX)`. Some of the mappings in this table
-represent potential changes of precision (marked p), dropped autoincrement
-functionality (marked a), differences in treatment of timezones (marked t),
-differences in treatment of fixed-length character types (marked c), and changes
-in storage size (marked s). We discuss these, as well as other limits and notes
-on schema conversion, in the following sections.
-
-### `NUMERIC`
-
-[Spanner's NUMERIC
-type](https://cloud.google.com/spanner/docs/data-types#decimal_type) can store
-up to 29 digits before the decimal point and up to 9 after the decimal point.
-PostgreSQL's NUMERIC type can potentially support higher precision that this, so
-please verify that Spanner's NUMERIC support meets your application needs.
-
-### `BIGSERIAL` and `SERIAL`
-
-Spanner does not support autoincrementing types, so these both map to `INT64`
-and the autoincrementing functionality is dropped.
-
-### `TIMESTAMP`
-
-PosgreSQL has two timestamp types: `TIMESTAMP` and `TIMESTAMPTZ`. Both have an 8
-byte data representation and provide microsecond resolution, but neither
-actually stores a timezone with the data. The key difference between the two
-types is how string literals are converted to timestamps and queries return
-data. For `TIMESTAMP`, all timezone information is dropped, and data is returned
-without a timezone. For `TIMESTAMPTZ`, string literals are converted to UTC,
-using the literal's timezone if it is specified, or the PostgreSQL's timezone
-paramater if not. When data is printed stored data (in UTC) is converted to the
-timezone from the timezone parameter
-
-Spanner has a single timestamp type. Data is stored as UTC (there is no separate
-timezone) Spanner client libraries convert timestamps to UTC before sending them
-to Spanner. Data is always returned as UTC. Spanner's timestamp type is
-essentially the same as `TIMESTAMPTZ`, except that there is no analog of
-PostgreSQL's timezone parameter.
-
-In other words, mapping PostgreSQL `TIMESTAMPTZ` to `TIMESTAMP` is fairly
-straightforward, but care should be taken with PostgreSQL `TIMESTAMP` data
-because Spanner clients will not drop the timezone.
-
-### `CHAR(n)` and `VARCHAR(n)`
-
-The semantics of fixed-length character types differ between PostgreSQL and
-Spanner. The `CHAR(n)` type in PostgreSQL is padded with spaces. If a string
-value smaller than the limit is stored, spaces will be added to pad it out to
-the specified length. If a string longer than the specified length is stored,
-and the extra characters are all spaces, then it will be silently
-truncated. Moreover, trailing spaces are ignored when comparing two values. In
-constrast, Spanner does not give special treatment to spaces, and the specified
-length simply represents the maximum length that can be stored. This is close to
-the semantics of PostgreSQL's `VARCHAR(n)`. However there are some minor
-differences. For example, even `VARCHAR(n)` has some special treatment of
-spaces: strings longer than the specified length are silently truncated if the
-extra characters are all spaces.
-
-### Storage Use
-
-The tool maps several PostgreSQL types to Spanner types that use more storage.
-For example, `SMALLINT` is a two-byte integer, but it maps to Spanner's `INT64`,
-an eight-byte integer. This additional storage could be significant for large
-arrays.
-
-### Arrays
-
-Spanner does not support multi-dimensional arrays. So while `TEXT[4]` maps to
-`ARRAY<STRING(MAX)>` and `REAL ARRAY` maps to `ARRAY<FLOAT64>`, `TEXT[][]` maps
-to `STRING(MAX)`.
-
-Also note that PosgreSQL supports array limits, but the PostgreSQL
-implementation ignores them. Spanner does not support array size limits, but
-since they have no effect anyway, the tool just drops them.
-
-### Primary Keys
-
-Spanner requires primary keys for all tables. PostgreSQL recommends the use of
-primary keys for all tables, but does not enforce this. When converting a table
-without a primary key, HarbourBridge will create a new primary key of type
-INT64. By default, the name of the new column is `synth_id`. If there is already
-a column with that name, then a variation is used to avoid collisions.
-
-### NOT NULL Constraints
-
-The tool preserves `NOT NULL` constraints. Note that Spanner does not require
-primary key columns to be `NOT NULL`. However, in PostgreSQL, a primary key is a
-combination of `NOT NULL` and `UNIQUE`, and so primary key columns from
-PostgreSQL will be mapped to Spanner columns that are both primary keys and `NOT NULL`.
-
-### Foreign Keys
-
-The tool maps PostgreSQL foreign key constraints into Spanner foreign key constraints, and
-preserves constraint names where possible. Note that Spanner requires foreign key
-constraint names to be globally unique (within a database), but in postgres they only
-have to be unique for a table, so we add a uniqueness suffix to a name if needed.
-Spanner doesn't support `ON DELETE` and `ON UPDATE` actions, so we drop these.
-
-### Default Values
-
-Spanner does not currently support default values. We drop these
-PostgreSQL features during conversion.
-
-### Secondary Indexes
-
-The tool maps PostgresSQL secondary indexes to Spanner secondary indexes, preserving
-constraint names where possible. The tool also maps PostgreSQL `UNIQUE` constraints to
-Spanner `UNIQUE` secondary indexes. Check [here](https://cloud.google.com/spanner/docs/migrating-postgres-spanner#indexes)
-for more details.
-
-### Other PostgreSQL features
-
-PostgreSQL has many other features we haven't discussed, including functions,
-sequences, procedures, triggers, (non-primary) indexes and views. The tool does
-not support these and the relevant statements are dropped during schema
-conversion.
-
-See
-[Migrating from PostgreSQL to Cloud Spanner](https://cloud.google.com/spanner/docs/migrating-postgres-spanner)
-for a general discussion of PostgreSQL to Spanner migration issues.
-HarbourBridge follows most of the recommendations in that guide. The main
-difference is that we map a few more types to `STRING(MAX)`.
-
-## Data Conversion
-
-### Timestamps and Timezones
-
-As noted earlier when discussing [schema conversion of
-TIMESTAMP](#timestamp), there are some subtle differences in how timestamps are
-handled in PostgreSQL and Spanner.
-
-During data conversion, PostgreSQL `TIMESTAMPTZ` values are converted to UTC and
-stored in Spanner. The conversion proceeds as follows. If the value has a
-timezone, that timezone is respected during the conversion to UTC. If the value
-does not have a timezone, then we look for any `set timezone` statements in the
-pg_dump output and use the timezone specified. Otherwise, we use the `TZ`
-environment variable as the timezone, and failing that, we use the local system
-timezone default (as determined by Go).
-
-In constrast, conversion of PostgreSQL `TIMESTAMP` values proceeds by ignoring
-any timezone information and just treating the value as UTC and storing it in
-Spanner.
-
-### Strings, character set support and UTF-8
-
-Spanner requires that `STRING` values be UTF-8 encoded. All Spanner functions
-and operators that act on `STRING` values operate on Unicode characters rather
-than bytes. Since we map many PostgreSQL types (including `TEXT` and `CHAR`
-types) to Spanner's `STRING` type, HarbourBridge is effectively a UTF-8 based
-tool.
-
-Note that the tool itself does not do any encoding/decoding or UTF-8 checks: it
-passes through data from pg_dump to Spanner. Internally, we use Go's string
-type, which supports UTF-8.
diff --git a/cmd/data.go b/cmd/data.go
index 7fb1d7b..922c844 100644
--- a/cmd/data.go
+++ b/cmd/data.go
@@ -41,7 +41,7 @@ func (cmd *DataCmd) Usage() string {
 Migrate data from source db to target db. Source db dump file can be specified
 by either file param in source-profile or piped to stdin. Connection profile
 for source databases in direct connect mode can be specified by setting
-appropriate environment variables. The data flags are:
+appropriate params in source-profile. The data flags are:
 `, path.Base(os.Args[0]))
 }
 
diff --git a/cmd/schema.go b/cmd/schema.go
index 219a9a2..d1e9d44 100644
--- a/cmd/schema.go
+++ b/cmd/schema.go
@@ -38,8 +38,8 @@ func (cmd *SchemaCmd) Usage() string {
 
 Convert schema for source db specified by source and source-profile. Source db
 dump file can be specified by either file param in source-profile or piped to
-stdin. Connection profile for source databases in direct connect mode can be
-specified by setting appropriate environment variables. The schema flags are:
+stdin. Connection profile for source database in direct connect mode can be
+specified by setting appropriate params in source-profile. The schema flags are:
 `, path.Base(os.Args[0]))
 }
 
diff --git a/cmd/eval.go b/cmd/schema_and_data.go
similarity index 86%
rename from cmd/eval.go
rename to cmd/schema_and_data.go
index 95726ce..f24e247 100644
--- a/cmd/eval.go
+++ b/cmd/schema_and_data.go
@@ -13,8 +13,8 @@ import (
 	"github.com/google/subcommands"
 )
 
-// EvalCmd struct with flags.
-type EvalCmd struct {
+// SchemaAndDataCmd struct with flags.
+type SchemaAndDataCmd struct {
 	source          string
 	sourceProfile   string
 	target          string
@@ -24,28 +24,28 @@ type EvalCmd struct {
 }
 
 // Name returns the name of operation.
-func (cmd *EvalCmd) Name() string {
-	return "eval"
+func (cmd *SchemaAndDataCmd) Name() string {
+	return "schema-and-data"
 }
 
 // Synopsis returns summary of operation.
-func (cmd *EvalCmd) Synopsis() string {
-	return "evaluate schema and data migration from source db to target db"
+func (cmd *SchemaAndDataCmd) Synopsis() string {
+	return "schema and data migration from source db to target db in schema-and-data"
 }
 
 // Usage returns usage info of the command.
-func (cmd *EvalCmd) Usage() string {
-	return fmt.Sprintf(`%v eval -source=[source] -target-profile="instance=my-instance"...
+func (cmd *SchemaAndDataCmd) Usage() string {
+	return fmt.Sprintf(`%v schema-and-data -source=[source] -target-profile="instance=my-instance"...
 
-Evaluate schema and data migration from source db to target db. Source db dump
+Migrate schema and data from source db to target db in schema-and-data. Source db dump
 file can be specified by either file param in source-profile or piped to stdin.
 Connection profile for source databases in direct connect mode can be specified
-by setting appropriate environment variables. The eval flags are:
+by setting appropriate params in source-profile. The schema-and-data flags are:
 `, path.Base(os.Args[0]))
 }
 
 // SetFlags sets the flags.
-func (cmd *EvalCmd) SetFlags(f *flag.FlagSet) {
+func (cmd *SchemaAndDataCmd) SetFlags(f *flag.FlagSet) {
 	f.StringVar(&cmd.source, "source", "", "Flag for specifying source DB, (e.g., `PostgreSQL`, `MySQL`, `DynamoDB`)")
 	f.StringVar(&cmd.sourceProfile, "source-profile", "", "Flag for specifying connection profile for source database e.g., \"file=<path>,format=dump\"")
 	f.StringVar(&cmd.target, "target", "Spanner", "Specifies the target DB, defaults to Spanner (accepted values: `Spanner`)")
@@ -54,7 +54,7 @@ func (cmd *EvalCmd) SetFlags(f *flag.FlagSet) {
 	f.StringVar(&cmd.filePrefix, "prefix", "", "File prefix for generated files")
 }
 
-func (cmd *EvalCmd) Execute(ctx context.Context, f *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
+func (cmd *SchemaAndDataCmd) Execute(ctx context.Context, f *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
 	var err error
 	defer func() {
 		if err != nil {
diff --git a/cmd/source_profile.go b/cmd/source_profile.go
index 92312fa..84d09cd 100644
--- a/cmd/source_profile.go
+++ b/cmd/source_profile.go
@@ -173,10 +173,10 @@ type SourceProfileConnectionDynamoDB struct {
 
 func NewSourceProfileConnectionDynamoDB(params map[string]string) (SourceProfileConnectionDynamoDB, error) {
 	dydb := SourceProfileConnectionDynamoDB{}
-	if schemaSampleSize, ok := params["schemaSampleSize"]; ok {
+	if schemaSampleSize, ok := params["schema-sample-size"]; ok {
 		schemaSampleSizeInt, err := strconv.Atoi(schemaSampleSize)
 		if err != nil {
-			return dydb, fmt.Errorf("could not parse schemaSampleSize = %v as a valid int64", schemaSampleSize)
+			return dydb, fmt.Errorf("could not parse schema-sample-size = %v as a valid int64", schemaSampleSize)
 		}
 		dydb.schemaSampleSize = int64(schemaSampleSizeInt)
 	}
@@ -184,16 +184,16 @@ func NewSourceProfileConnectionDynamoDB(params map[string]string) (SourceProfile
 	// Unlike postgres and mysql, there may not be deprecation of env variables, hence it
 	// is better to override env variables optionally via source profile params.
 	var ok bool
-	if dydb.awsAccessKeyID, ok = params["awsAccessKeyID"]; ok {
+	if dydb.awsAccessKeyID, ok = params["aws-access-key-id"]; ok {
 		os.Setenv("AWS_ACCESS_KEY_ID", dydb.awsAccessKeyID)
 	}
-	if dydb.awsSecretAccessKey, ok = params["awsSecretAccessKey"]; ok {
+	if dydb.awsSecretAccessKey, ok = params["aws-secret-access-key"]; ok {
 		os.Setenv("AWS_SECRET_ACCESS_KEY", dydb.awsAccessKeyID)
 	}
-	if dydb.awsRegion, ok = params["awsRegion"]; ok {
+	if dydb.awsRegion, ok = params["aws-region"]; ok {
 		os.Setenv("AWS_REGION", dydb.awsAccessKeyID)
 	}
-	if dydb.dydbEndpoint, ok = params["dydbEndpoint"]; ok {
+	if dydb.dydbEndpoint, ok = params["dydb-endpoint"]; ok {
 		os.Setenv("DYNAMODB_ENDPOINT_OVERRIDE", dydb.awsAccessKeyID)
 	}
 	return dydb, nil
diff --git a/main.go b/main.go
index 0468f17..ffcfb8f 100644
--- a/main.go
+++ b/main.go
@@ -107,7 +107,7 @@ func main() {
 		subcommands.Register(subcommands.CommandsCommand(), "")
 		subcommands.Register(&cmd.SchemaCmd{}, "")
 		subcommands.Register(&cmd.DataCmd{}, "")
-		subcommands.Register(&cmd.EvalCmd{}, "")
+		subcommands.Register(&cmd.SchemaAndDataCmd{}, "")
 		flag.Parse()
 		os.Exit(int(subcommands.Execute(ctx)))
 	}
diff --git a/sources/dynamodb/README.md b/sources/dynamodb/README.md
index 5d076e9..76eba9f 100644
--- a/sources/dynamodb/README.md
+++ b/sources/dynamodb/README.md
@@ -1,54 +1,103 @@
-# HarbourBridge: Turnkey DynamoDB-to-Spanner Evaluation
+# HarbourBridge: DynamoDB-to-Spanner Evaluation and Migration
 
-HarbourBridge is a stand-alone open source tool for Cloud Spanner evaluation,
-using data from an existing PostgreSQL or MySQL or DynamoDB database. This
+HarbourBridge is a stand-alone open source tool for Cloud Spanner evaluation and migration,
+using data from an existing database. This
 README provides details of the tool's DynamoDB capabilities. For general
-HarbourBridge information see this [README](https://github.com/cloudspannerecosystem/harbourbridge#harbourbridge-turnkey-spanner-evaluation).
+HarbourBridge information see this [README](https://github.com/cloudspannerecosystem/harbourbridge#harbourbridge-spanner-evaluation-and-migration).
 
 ## Example DynamoDB Usage
 
-The following examples assume `harbourbridge` alias has been setup as
-following.
-
-```sh
-git clone https://github.com/cloudspannerecosystem/harbourbridge
-cd harbourbridge
-alias harbourbridge="go run github.com/cloudspannerecosystem/harbourbridge"
-```
-
 Before running HarbourBridge, make sure that you have
 [set up your AWS credentials/region](https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html)
-correctly (set the environment variables `AWS_ACCESS_KEY_ID`, 
-`AWS_SECRET_ACCESS_KEY`, `AWS_REGION`). HarbourBridge accesses your 
+correctly (set the environment variables `AWS_ACCESS_KEY_ID`,
+`AWS_SECRET_ACCESS_KEY`, `AWS_REGION`). HarbourBridge accesses your
 DynamoDB database via the aws go sdk package. If you use a custom endpoint
-for dynamodb, you can specify that using the environment variable 
+for dynamodb, you can specify that using the environment variable
 `DYNAMODB_ENDPOINT_OVERRIDE`.
 
+The following examples assume a harbourbridge alias has been setup as described
+in the [Installing HarbourBridge](https://github.com/cloudspannerecosystem/harbourbridge#installing-harbourbridge) section of the main README.
+
+For example, run
+
+```sh
+export DYNAMODB_ENDPOINT_OVERRIDE=http://dynamodb.us-west-2.amazonaws.com
+harbourbridge schema -source=dynamodb 
+```
+
+Instead of setting the environment variables, you
+can also pass corresponding source profile connection parameters `aws-access-key-id`
+, `aws-secret-access-key`, `aws-region`. Custom endpoint can be specified using
+`dydb-endpoint` param.
+
+For example, to convert schema run
+
+```sh
+harbourbridge schema -source=dynamodb -source-profile="aws-access-key-id=<>,aws-secret-access-key=<>,aws-region=<>"
+```
+
+This will generate a session file with `session.json` suffix. This file contains
+schema mapping from source to destination. You will need to specify this file
+during data migration. You can also specify a particular Spanner instance to use
+during data migration.
+
+For example, run
+
+```sh
+harbourbridge data -session=mydb.session.json -source=dynamodb -source-profile="aws-access-key-id=<>,..." -target-profile="instance=my-spanner-instance"
+```
+
+By default, HarbourBridge will generate a new Spanner database name to populate.
+You can override this and specify the database name to use by:
+
 ```sh
-export DYNAMODB_ENDPOINT_OVERRIDE=http://dynamodb.us-west-2.amazonaws.com`
+harbourbridge data -session=mydb.session.json -source=dynamodb -source-profile="aws-access-key-id=<>,..." -target-profile="instance=my-spanner-instance,..."
 ```
 
-To use the tool on a DynamoDB database (it will migrate all tables),
-run
+You can also run HarbourBridge in a schema-and-data mode, where it will perform both
+schema and data migration. This is useful for quick evaluation when source
+database size is small.
 
 ```sh
-harbourbridge -driver=dynamodb
+harbourbridge schema-and-data -source=dynamodb -source-profile="aws-access-key-id=<>,..." -target-profile="instance=my-spanner-instance,..."
 ```
 
-Due the schemaless nature of DynamoDB, the tool infers the schema based on a
-certain amount of sampled data, by default, 100000 rows. If a table has more
-rows than the default value, we only use 100000 rows for estimating the schema.
-You can change this value via the flag `schema-sample-size`. For example,
+HarbourBridge generates a report file, a schema file, and a bad-data file (if
+there are bad-data rows). You can control where these files are written by
+specifying a file prefix. For example,
+
+```sh
+harbourbridge schema -prefix=mydb. -source=dynamodb -source-profile="aws-access-key-id=<>,..."
+```
+
+will write files `mydb.report.txt`, `mydb.schema.txt`, and
+`mydb.dropped.txt`. The prefix can also be a directory. For example,
+
+```sh
+harbourbridge schema -prefix=~/spanner-eval-mydb/ -source=dynamodb -source-profile="aws-access-key-id=<>,..."
+```
+
+would write the files into the directory `~/spanner-eval-mydb/`. Note
+that HarbourBridge will not create directories as it writes these files.
+
+HarbourBridge accepts an additional param `schema-sample-size` for
+`-source-profile` for DynamoDB. Due to the schemaless nature of DynamoDB, the
+tool infers the schema based on a certain amount of sampled data, by default,
+100000 rows. If a table has more rows than the default value, we only use
+100000 rows for estimating the schema. This flag lets you specify the number
+of rows to use for inferring schema. The default value is 100,000.
+
+Sample usage:
 
 ```sh
-harbourbridge -driver=dynamodb -schema-sample-size=500000
+harbourbridge schema -source=dynamodb -source-profile="schema-sample-size=500000,aws-access-key-id=<>,..."
 ```
 
 ## Schema Conversion
 
 The HarbourBridge tool maps DynamoDB types to Spanner types as follows:
 
-| DynamoDB Type    | Spanner Type               | Notes                                     |
+| DynamoDB Type      | Spanner Type               | Notes                                     |
 | ------------------ | -------------------------- | ----------------------------------------- |
 | `Number`           | `NUMERIC` or `STRING`      | defaults to NUMERIC, otherwise, STRING    |
 | `String`           | `STRING`                   |                                           |
@@ -98,7 +147,7 @@ and if it consistently fails, we choose STRING type for the column.
 
 In DynamoDB, a column can have a Null data type that represents an unknown or
 undefined state. Also, each row defines its own schema for columns (not for
-primary keys). So columns can be absent in rows. 
+primary keys). So columns can be absent in rows.
 
 We treat the above two cases the same as a Null value in Cloud Spanner. The
 cases that a column contains a Null value or a column is not present is an
@@ -109,7 +158,7 @@ indication that this column should be nullable.
 In Cloud Spanner, the most similar type to List and Map is
 [STRUCT](https://cloud.google.com/spanner/docs/data-types#struct_type), but it
 is not a valid column type (available for query but not for storage).
-Therefore, we encode them into a json string. 
+Therefore, we encode them into a json string.
 
 #### Occasional Errors
 
@@ -129,7 +178,7 @@ In the migration, we define a conflicting threshold on rows (after removing Null
 data types and rows that the column is not present). By default, the conflicting
 threshold is 5% and if the percentages of two or more data types are greater
 than it, we would consider that the column has conflicting data types. As a safe
-choice, we define this column as a STRING type in Cloud Spanner. 
+choice, we define this column as a STRING type in Cloud Spanner.
 
 ## Data Conversion
 
@@ -142,6 +191,6 @@ make a subsequent call to continue retrieving data from the table.
 The row result contains the data type and data itself. According to our
 [inferred schema](#schema-inference), we will parse the row to a format that
 Cloud Spanner can support. If the value parsing fails, we would drop the entire
-row and record it as bad data in the report. If a column does not appear or 
-column has a NULL data type, we would process this as a NULL value in 
-Cloud Spanner. 
+row and record it as bad data in the report. If a column does not appear or
+column has a NULL data type, we would process this as a NULL value in
+Cloud Spanner.
diff --git a/sources/mysql/README.md b/sources/mysql/README.md
index 4cbe9ba..13a6460 100644
--- a/sources/mysql/README.md
+++ b/sources/mysql/README.md
@@ -1,49 +1,50 @@
-# HarbourBridge: Turnkey MySQL-to-Spanner Evaluation
+# HarbourBridge: MySQL-to-Spanner Evaluation and Migration
 
-HarbourBridge is a stand-alone open source tool for Cloud Spanner evaluation,
-using data from an existing PostgreSQL or MySQL database. This README provides
+HarbourBridge is a stand-alone open source tool for Cloud Spanner evaluation and migration,
+using data from an existing database. This README provides
 details of the tool's MySQL capabilities. For general HarbourBridge information
-see this [README](https://github.com/cloudspannerecosystem/harbourbridge#harbourbridge-turnkey-spanner-evaluation).
+see this [README](https://github.com/cloudspannerecosystem/harbourbridge#harbourbridge-spanner-evaluation-and-migration).
 
 ## Example MySQL Usage
 
-The following examples assume `harbourbridge` alias has been setup as
-following.
-
-```sh
-git clone https://github.com/cloudspannerecosystem/harbourbridge
-cd harbourbridge
-alias harbourbridge="go run github.com/cloudspannerecosystem/harbourbridge"
-```
-
 HarbourBridge can either be used with mysqldump or it can be run directly
 on a MySQL database (via go's database/sql package).
 
+The following examples assume a `harbourbridge` alias has been setup as described
+in the [Installing HarbourBridge](https://github.com/cloudspannerecosystem/harbourbridge#installing-harbourbridge) section of the main README.
+
 ### Using HarbourBridge with mysqldump
 
-To use HarbourBridge on a MySQL database called mydb using mysqldump, run:
+The tool can used to migrate schema from an existing mysqldump file:
 
 ```sh
-mysqldump mydb | harbourbridge -driver=mysqldump
+harbourbridge schema -source=mysql < my_mysqldump_file
 ```
 
-The tool can also be applied to an existing mysqldump file:
+This will generate a session file with `session.json` suffix. This file contains
+schema mapping from source to destination. You will need to specify this file
+during data migration. You can also specify a particular Spanner instance to use
+during data migration.
+
+For example, run
 
 ```sh
-harbourbridge -driver=mysqldump < my_mysqldump_file
+harbourbridge data -session=mydb.session.json -source=mysql -target-profile="instance=my-spanner-instance" < my_mysqldump_file
 ```
 
-To specify a particular Spanner instance to use, run:
+By default, HarbourBridge will generate a new Spanner database name to populate.
+You can override this and specify the database name to use by:
 
 ```sh
-mysqldump mydb | harbourbridge -driver=mysqldump -instance my-spanner-instance
+harbourbridge data -session=mydb.session.json -source=mysql -target-profile="instance=my-spanner-instance,dbname=my-spanner-database-name" < my_mysqldump_file
 ```
 
-By default, HarbourBridge will generate a new Spanner database name to populate.
-You can override this and specify the database name to use by:
+You can also run HarbourBridge in a schema-and-data mode, where it will perform both
+schema and data migration. This is useful for quick evaluation when source
+database size is small.
 
 ```sh
-mysqldump mydb | harbourbridge -driver=mysqldump -dbname my-spanner-database-name
+harbourbridge schema-and-data -source=mysql -target-profile="instance=my-spanner-instance" < my_mysqldump_file
 ```
 
 HarbourBridge generates a report file, a schema file, and a bad-data file (if
@@ -51,14 +52,14 @@ there are bad-data rows). You can control where these files are written by
 specifying a file prefix. For example,
 
 ```sh
-mysqldump mydb | harbourbridge -prefix mydb. -driver=mysqldump
+harbourbridge schema -prefix=mydb. -source=mysql < my_mysqldump_file
 ```
 
 will write files `mydb.report.txt`, `mydb.schema.txt`, and
 `mydb.dropped.txt`. The prefix can also be a directory. For example,
 
 ```sh
-mysqldump mydb | harbourbridge -prefix ~/spanner-eval-mydb/ -driver=mysqldump
+harbourbridge schema -prefix=~/spanner-eval-mydb/ -source=mysql < my_mysqldump_file
 ```
 
 would write the files into the directory `~/spanner-eval-mydb/`. Note
@@ -66,18 +67,25 @@ that HarbourBridge will not create directories as it writes these files.
 
 ### Directly connecting to a MySQL database
 
-To use the tool directly on a MySQL database called mydb, run
+In this case, HarbourBridge connects directly to the MySQL database to retrieve
+table schema and data. Set the `-source=mysql` and corresponding source profile
+connection parameters `host`, `port`, `user`, `db_name` and `password`.
+
+For example to perform schema conversion, run
 
 ```sh
-harbourbridge -driver=mysql
+harbourbridge schema -source=mysql -source-profile="host=<>,port=<>,user=<>,db_name=<>"
 ```
 
-It is assumed that _MYSQLHOST_, _MYSQLPORT_, _MYSQLUSER_, _MYSQLDATABASE_ environment
-variables are set. Password can be specified either in the _MYSQLPWD_ environment
-variable or provided at the password prompt.
+Parameters `port` and `password` are optional. Port (`port`) defaults to `3306`
+for MySQL source. Password can be provided at the password prompt.
+
+(⚠ Deprecated ⚠) Set environment variables `MYSQLHOST`, `MYSQLPORT`,
+`MYSQLUSER`, `MYSQLDATABASE`. Password can be specified either in the
+`MYSQLPWD` environment variable or provided at the password prompt.
 
-Note that all of the options described in the previous section on using mysqldump
-can also be used with "-driver=mysql".
+Note that the various target-profile params described in the previous section
+are also applicable in direct connect mode.
 
 ## Schema Conversion
 
@@ -258,13 +266,13 @@ type, which supports UTF-8.
 
 As noted earlier when discussing [schema conversion of
 Spatial datatype](#spatial-datatype), Spanner does not support spatial datatypes and are
-mapped to `STRING(MAX)` Spanner type. Data conversion for spatial datatypes is different
-for both `mysql` and `mysqldump` driver.
+mapped to `STRING(MAX)` Spanner type. There are differences in data conversion for spatial
+datatypes depending on whether direct connect or a mysqldump file is used.
 
-- MySQL information schema approach (`-driver=mysql`) : Data from MySQL is fetched using
+- MySQL information schema approach (direct connect) : Data from MySQL is fetched using
   'ST_AsText(g)' function which converts a value in internal geometry format to its WKT(Well-Known Text)
   representation and returns the string result. This value will be stored as `STRING` in Spanner.
-- MySQL dump approach (`-driver=mysqldump`) : Mysqldump will have the internal geometry data in
+- MySQL dump approach : Mysqldump will have the internal geometry data in
   binary format. It cannot be converted to WKT format and there is no proper method for mysqldump
   generation of spatial datatypes also. Thus, this value will just be fetched as a `TEXT` type and
   converted to Spanner type `STRING`.
diff --git a/sources/postgres/README.md b/sources/postgres/README.md
index d2cbd8d..a6c35c6 100644
--- a/sources/postgres/README.md
+++ b/sources/postgres/README.md
@@ -1,49 +1,53 @@
-# HarbourBridge: Turnkey PostgreSQL-to-Spanner Evaluation
+# HarbourBridge: PostgreSQL-to-Spanner Evaluation and Migration
 
 HarbourBridge is a stand-alone open source tool for Cloud Spanner evaluation,
 using data from an existing PostgreSQL or MySQL database. This README provides
 details of the tool's PostgreSQL capabilities. For general HarbourBridge information
-see this [README](https://github.com/cloudspannerecosystem/harbourbridge#harbourbridge-turnkey-spanner-evaluation).
+see this [README](https://github.com/cloudspannerecosystem/harbourbridge#harbourbridge-spanner-evaluation-and-migration).
 
 ## Example PostgreSQL Usage
 
-The following examples assume `harbourbridge` alias has been setup as
-following.
-
-```sh
-git clone https://github.com/cloudspannerecosystem/harbourbridge
-cd harbourbridge
-alias harbourbridge="go run github.com/cloudspannerecosystem/harbourbridge"
-```
-
 HarbourBridge can either be used with pg_dump or it can be run directly
 on a PostgreSQL database (via go's database/sql package).
 
+The following examples assume a `harbourbridge` alias has been setup as described
+in the [Installing HarbourBridge](https://github.com/cloudspannerecosystem/harbourbridge#installing-harbourbridge) section of the main README.
+
 ### Using HarbourBridge with pg_dump
 
-To use HarbourBridge on a PostgreSQL database called mydb using pg_dump output,run:
+The tool can used to migrate schema from an existing pg_dump file:
 
 ```sh
-pg_dump mydb | harbourbridge -driver=pg_dump
+harbourbridge schema -source=postgresql < my_pg_dump_file
 ```
 
-The tool can also be applied to an existing pg_dump file:
+You can use any of `postgresql`, `postgres`, or `pg` as the argument to the
+`-source` flag. They all specify PostgreSQL as the source database.
 
-```sh
-harbourbridge -driver=pg_dump < my_pg_dump_file
-```
+This will generate a session file with `session.json` suffix. This file contains
+schema mapping from source to destination. You will need to specify this file
+during data migration. You can also specify a particular Spanner instance to use
+during data migration.
 
-To specify a particular Spanner instance to use, run:
+For example, run
 
 ```sh
-pg_dump mydb | harbourbridge -driver=pg_dump -instance my-spanner-instance
+harbourbridge data -session=mydb.session.json -source=pg -target-profile="instance=my-spanner-instance" < my_pg_dump_file
 ```
 
 By default, HarbourBridge will generate a new Spanner database name to populate.
 You can override this and specify the database name to use by:
 
 ```sh
-pg_dump mydb | harbourbridge -driver=pg_dump -dbname my-spanner-database-name
+harbourbridge data -session=mydb.session.json -source=pg -target-profile="instance=my-spanner-instance,dbname=my-spanner-database-name" < my_pg_dump_file
+```
+
+You can also run HarbourBridge in a schema-and-data mode, where it will perform both
+schema and data migration. This is useful for quick evaluation when source
+database size is small.
+
+```sh
+harbourbridge schema-and-data -source=pg -target-profile="instance=my-spanner-instance" < my_pg_dump_file
 ```
 
 HarbourBridge generates a report file, a schema file, and a bad-data file (if
@@ -51,14 +55,14 @@ there are bad-data rows). You can control where these files are written by
 specifying a file prefix. For example,
 
 ```sh
-pg_dump mydb | harbourbridge -driver=pg_dump -prefix mydb.
+harbourbridge schema -prefix=mydb. -source=postgres < my_pg_dump_file
 ```
 
 will write files `mydb.report.txt`, `mydb.schema.txt`, and
 `mydb.dropped.txt`. The prefix can also be a directory. For example,
 
 ```sh
-pg_dump mydb | harbourbridge -driver=pg_dump -prefix ~/spanner-eval-mydb/
+harbourbridge schema -prefix=~/spanner-eval-mydb/ -source=postgres < my_pg_dump_file
 ```
 
 would write the files into the directory `~/spanner-eval-mydb/`. Note
@@ -66,18 +70,26 @@ that HarbourBridge will not create directories as it writes these files.
 
 ### Directly connecting to a PostgreSQL database
 
-To use the tool directly on a PostgresSQL database called mydb, run
+In this case, HarbourBridge connects directly to the PostgreSQL database to
+retrieve table schema and data. Set the `-source=postgres` and corresponding
+source profile connection parameters `host`, `port`, `user`, `db_name` and
+`password`.
+
+For example to perform schema conversion, run
 
 ```sh
-harbourbridge -driver=postgres
+harbourbridge schema -source=postgres -source-profile="host=<>,port=<>,user=<>,db_name=<>"
 ```
 
-It is assumed that _PGHOST_, _PGPORT_, _PGUSER_, _PGDATABASE_ environment
-variables are set. Password can be specified either in the _PGPASSWORD_ environment
-variable or provided at the password prompt.
+Parameters `port` and `password` are optional. Port (`port`) defaults to `5432`
+for PostgreSQL source. Password can be provided at the password prompt.
+
+Alternatively, you can also set environment variables `PGHOST`, `PGPORT`,
+`PGUSER`, `PGDATABASE` for direct access. Password can be specified either in
+the `PGPASSWORD` environment variable or provided at the password prompt.
 
-Note that all of the options described in the previous section on using pg_dump can
-also be used with "-driver=postgres".
+Note that the various target-profile params described in the previous section
+are also applicable in direct connect mode.
 
 ## Schema Conversion
 


Test patch:
diff --git a/cmd/source_profile_test.go b/cmd/source_profile_test.go
index b7a3a76..6562ee9 100644
--- a/cmd/source_profile_test.go
+++ b/cmd/source_profile_test.go
@@ -120,12 +120,12 @@ func TestNewSourceProfileConnectionDynamoDB(t *testing.T) {
 		},
 		{
 			name:          "valid schema sample size",
-			params:        map[string]string{"schemaSampleSize": "15"},
+			params:        map[string]string{"schema-sample-size": "15"},
 			errorExpected: false,
 		},
 		{
 			name:          "invalid schema sample size",
-			params:        map[string]string{"schemaSampleSize": "a"},
+			params:        map[string]string{"schema-sample-size": "a"},
 			errorExpected: true,
 		},
 	}
diff --git a/testing/mysql/integration_test.go b/testing/mysql/integration_test.go
index 6af891a..aeef596 100644
--- a/testing/mysql/integration_test.go
+++ b/testing/mysql/integration_test.go
@@ -126,20 +126,20 @@ func TestIntegration_MYSQLDUMP_Command(t *testing.T) {
 	checkResults(t, dbURI)
 }
 
-func TestIntegration_MySQL_EvalSubcommand(t *testing.T) {
+func TestIntegration_MYSQL_SchemaAndDataSubcommand(t *testing.T) {
 	onlyRunForEmulatorTest(t)
 	t.Parallel()
 
 	tmpdir := prepareIntegrationTest(t)
 	defer os.RemoveAll(tmpdir)
 
-	dbName := "mysql-dc-eval-src-params"
+	dbName := "mysql-dc-schema-and-data"
 	dbURI := fmt.Sprintf("projects/%s/instances/%s/databases/%s", projectID, instanceID, dbName)
 	filePrefix := filepath.Join(tmpdir, dbName+".")
 
 	host, user, db_name, password := os.Getenv("MYSQLHOST"), os.Getenv("MYSQLUSER"), os.Getenv("MYSQLDATABASE"), os.Getenv("MYSQLPWD")
 	envVars := common.ClearEnvVariables([]string{"MYSQLHOST", "MYSQLUSER", "MYSQLDATABASE", "MYSQLPWD"})
-	args := fmt.Sprintf("eval -source=%s -prefix=%s -source-profile='host=%s,user=%s,db_name=%s,password=%s' -target-profile='instance=%s,dbname=%s'", constants.MYSQL, filePrefix, host, user, db_name, password, instanceID, dbName)
+	args := fmt.Sprintf("schema-and-data -source=%s -prefix=%s -source-profile='host=%s,user=%s,db_name=%s,password=%s' -target-profile='instance=%s,dbname=%s'", constants.MYSQL, filePrefix, host, user, db_name, password, instanceID, dbName)
 	err := common.RunCommand(args, projectID)
 	common.RestoreEnvVariables(envVars)
 	if err != nil {
@@ -246,8 +246,8 @@ func runDataSubcommand(t *testing.T, dbName, dbURI, filePrefix, sessionFile, dum
 	}
 }
 
-func runEvalSubcommand(t *testing.T, dbName, dbURI, filePrefix, dumpFilePath string) {
-	args := fmt.Sprintf("eval -source=mysql -prefix %s -target-profile='instance=%s,dbname=%s' < %s", filePrefix, instanceID, dbName, dumpFilePath)
+func runSchemaAndDataSubcommand(t *testing.T, dbName, dbURI, filePrefix, dumpFilePath string) {
+	args := fmt.Sprintf("schema-and-data -source=mysql -prefix %s -target-profile='instance=%s,dbname=%s' < %s", filePrefix, instanceID, dbName, dumpFilePath)
 	err := common.RunCommand(args, projectID)
 	if err != nil {
 		t.Fatal(err)
@@ -294,17 +294,17 @@ func TestIntegration_MySQLDUMP_DataSubcommand(t *testing.T) {
 	checkResults(t, dbURI)
 }
 
-func TestIntegration_MySQLDUMP_EvalSubcommand(t *testing.T) {
+func TestIntegration_MySQLDUMP_SchemaAndDataSubcommand(t *testing.T) {
 	onlyRunForEmulatorTest(t)
 	tmpdir := prepareIntegrationTest(t)
 	defer os.RemoveAll(tmpdir)
 
-	dbName := "test-eval-subcommand"
+	dbName := "test-schema-and-data"
 	dumpFilePath := "../../test_data/mysqldump.test.out"
 	filePrefix := filepath.Join(tmpdir, dbName+".")
 
 	dbURI := fmt.Sprintf("projects/%s/instances/%s/databases/%s", projectID, instanceID, dbName)
-	runEvalSubcommand(t, dbName, dbURI, filePrefix, dumpFilePath)
+	runSchemaAndDataSubcommand(t, dbName, dbURI, filePrefix, dumpFilePath)
 	defer dropDatabase(t, dbURI)
 	checkResults(t, dbURI)
 }
diff --git a/testing/postgres/integration_test.go b/testing/postgres/integration_test.go
index b859dcb..7673246 100644
--- a/testing/postgres/integration_test.go
+++ b/testing/postgres/integration_test.go
@@ -129,7 +129,7 @@ func TestIntegration_PGDUMP_Command(t *testing.T) {
 	checkResults(t, dbURI)
 }
 
-func TestIntegration_PGDUMP_EvalSubcommand(t *testing.T) {
+func TestIntegration_PGDUMP_SchemaAndDataSubcommand(t *testing.T) {
 	t.Parallel()
 
 	tmpdir := prepareIntegrationTest(t)
@@ -142,7 +142,7 @@ func TestIntegration_PGDUMP_EvalSubcommand(t *testing.T) {
 	dataFilepath := "../../test_data/pg_dump.test.out"
 	filePrefix := filepath.Join(tmpdir, dbName+".")
 
-	args := fmt.Sprintf("eval -prefix %s -source=postgres -target-profile='instance=%s,dbname=%s' < %s", filePrefix, instanceID, dbName, dataFilepath)
+	args := fmt.Sprintf("schema-and-data -prefix %s -source=postgres -target-profile='instance=%s,dbname=%s' < %s", filePrefix, instanceID, dbName, dataFilepath)
 	err := common.RunCommand(args, projectID)
 	if err != nil {
 		t.Fatal(err)
@@ -191,7 +191,7 @@ func TestIntegration_POSTGRES_Command(t *testing.T) {
 	checkResults(t, dbURI)
 }
 
-func TestIntegration_POSTGRES_EvalSubcommand(t *testing.T) {
+func TestIntegration_POSTGRES_SchemaAndDataSubcommand(t *testing.T) {
 	onlyRunForEmulatorTest(t)
 	t.Parallel()
 
@@ -203,7 +203,7 @@ func TestIntegration_POSTGRES_EvalSubcommand(t *testing.T) {
 	dbURI := fmt.Sprintf("projects/%s/instances/%s/databases/%s", projectID, instanceID, dbName)
 	filePrefix := filepath.Join(tmpdir, dbName+".")
 
-	args := fmt.Sprintf("eval -prefix %s -source=postgres -target-profile='instance=%s,dbname=%s'", filePrefix, instanceID, dbName)
+	args := fmt.Sprintf("schema-and-data -prefix %s -source=postgres -target-profile='instance=%s,dbname=%s'", filePrefix, instanceID, dbName)
 	err := common.RunCommand(args, projectID)
 	if err != nil {
 		t.Fatal(err)