Repository: cloudspannerecosystem/harbourbridge
Pull request #86 (2020-09-21 08:02:15)

Fix: Update message for foreign keys.
HarbourBridge currently reports "Column mycol uses foreign keys which Spanner does not support", but this is no longer accurate since Spanner does support foreign keys. We haven't added this to HarbourBridge yet (but it's coming soon as part of user-guided schema migration).

hengfengli: @nevinheintze We probably need to update the `report_test.go`: 

```
    report_test.go:169: 
        	Error Trace:	report_test.go:169
        	Error:      	Not equal: 
        	            	expected: "----------------------------\nSummary of Conversion\n----------------------------\nSchema conversion: OK (some columns did not map cleanly + some missing primary keys).\nData conversion: POOR (66% of 6000 rows written to Spanner).\n\nThe remainder of this report provides stats on the pg_dump statements processed,\nfollowed by a table-by-table listing of schema and data conversion details. For\nbackground on the schema and data conversion process used, and explanations of\nthe terms and notes used in this report, see HarbourBridge's README.\n\n----------------------------\nStatements Processed\n----------------------------\nAnalysis of statements in pg_dump output, broken down by statement type.\n  schema: statements successfully processed for Spanner schema information.\n    data: statements successfully processed for data.\n    skip: statements not relevant for Spanner schema or data.\n   error: statements that could not be processed.\n  --------------------------------------\n  schema   data   skip  error  statement\n  --------------------------------------\n       5      0      0      0  CreateStmt\nSee github.com/lfittl/pg_query_go/nodes for definitions of statement types\n(lfittl/pg_query_go is the library we use for parsing pg_dump output).\n\n----------------------------\nTable bad_schema\n----------------------------\nSchema conversion: POOR (many columns did not map cleanly + missing primary key).\nData conversion: OK (94% of 1000 rows written to Spanner).\n\nWarnings\n1) Column 'synth_id' was added because this table didn't have a primary key.\n   Spanner requires a primary key for every table.\n2) Column 'a': type numeric is mapped to float64. Spanner does not support\n   numeric. This type mapping could lose precision and is not recommended for\n   production use.\n3) Column 'c': type int4[4][2] is mapped to string(max). Spanner doesn't support\n   multi-dimensional arrays.\n4) Column 'd': type circle is mapped to string(max). No appropriate Spanner\n   type.\n\nNote\n1) Some columns will consume more storage in Spanner e.g. for column 'b', source\n   DB type int4 is mapped to Spanner type int64.\n\n----------------------------\nTable default_value\n----------------------------\nSchema conversion: POOR (many columns did not map cleanly).\nData conversion: NONE (no data rows found).\n\nWarning\n1) Some columns have default values which Spanner does not support e.g. column\n   'b'.\n\n----------------------------\nTable excellent_schema\n----------------------------\nSchema conversion: EXCELLENT (all columns mapped cleanly).\nData conversion: NONE (no data rows found).\n\n----------------------------\nTable foreign_key\n----------------------------\nSchema conversion: POOR (many columns did not map cleanly).\nData conversion: NONE (no data rows found).\n\nWarning\n1) Column 'a' uses foreign keys which Spanner does not support.\n\n----------------------------\nTable no_pk\n----------------------------\nSchema conversion: GOOD (all columns mapped cleanly, but missing primary key).\nData conversion: POOR (60% of 5000 rows written to Spanner).\n\nWarning\n1) Column 'synth_id' was added because this table didn't have a primary key.\n   Spanner requires a primary key for every table.\n\nNote\n1) Some columns will consume more storage in Spanner e.g. for column 'b', source\n   DB type int4 is mapped to Spanner type int64.\n\n----------------------------\nUnexpected Conditions\n----------------------------\nFor debugging only. This section provides details of unexpected conditions\nencountered as we processed the pg_dump data. In particular, the AST node\nrepresentation used by the lfittl/pg_query_go library used for parsing\npg_dump output is highly permissive: almost any construct can appear at\nany node in the AST tree. The list details all unexpected nodes and\nconditions.\n  --------------------------------------\n   count  condition\n  --------------------------------------\n       5  Testing unexpected messages\n\n"
        	            	actual  : "----------------------------\nSummary of Conversion\n----------------------------\nSchema conversion: OK (some columns did not map cleanly + some missing primary keys).\nData conversion: POOR (66% of 6000 rows written to Spanner).\n\nThe remainder of this report provides stats on the pg_dump statements processed,\nfollowed by a table-by-table listing of schema and data conversion details. For\nbackground on the schema and data conversion process used, and explanations of\nthe terms and notes used in this report, see HarbourBridge's README.\n\n----------------------------\nStatements Processed\n----------------------------\nAnalysis of statements in pg_dump output, broken down by statement type.\n  schema: statements successfully processed for Spanner schema information.\n    data: statements successfully processed for data.\n    skip: statements not relevant for Spanner schema or data.\n   error: statements that could not be processed.\n  --------------------------------------\n  schema   data   skip  error  statement\n  --------------------------------------\n       5      0      0      0  CreateStmt\nSee github.com/lfittl/pg_query_go/nodes for definitions of statement types\n(lfittl/pg_query_go is the library we use for parsing pg_dump output).\n\n----------------------------\nTable bad_schema\n----------------------------\nSchema conversion: POOR (many columns did not map cleanly + missing primary key).\nData conversion: OK (94% of 1000 rows written to Spanner).\n\nWarnings\n1) Column 'synth_id' was added because this table didn't have a primary key.\n   Spanner requires a primary key for every table.\n2) Column 'a': type numeric is mapped to float64. Spanner does not support\n   numeric. This type mapping could lose precision and is not recommended for\n   production use.\n3) Column 'c': type int4[4][2] is mapped to string(max). Spanner doesn't support\n   multi-dimensional arrays.\n4) Column 'd': type circle is mapped to string(max). No appropriate Spanner\n   type.\n\nNote\n1) Some columns will consume more storage in Spanner e.g. for column 'b', source\n   DB type int4 is mapped to Spanner type int64.\n\n----------------------------\nTable default_value\n----------------------------\nSchema conversion: POOR (many columns did not map cleanly).\nData conversion: NONE (no data rows found).\n\nWarning\n1) Some columns have default values which Spanner does not support e.g. column\n   'b'.\n\n----------------------------\nTable excellent_schema\n----------------------------\nSchema conversion: EXCELLENT (all columns mapped cleanly).\nData conversion: NONE (no data rows found).\n\n----------------------------\nTable foreign_key\n----------------------------\nSchema conversion: POOR (many columns did not map cleanly).\nData conversion: NONE (no data rows found).\n\nWarning\n1) Column 'a' uses foreign keys which HarbourBridge does not support yet.\n\n----------------------------\nTable no_pk\n----------------------------\nSchema conversion: GOOD (all columns mapped cleanly, but missing primary key).\nData conversion: POOR (60% of 5000 rows written to Spanner).\n\nWarning\n1) Column 'synth_id' was added because this table didn't have a primary key.\n   Spanner requires a primary key for every table.\n\nNote\n1) Some columns will consume more storage in Spanner e.g. for column 'b', source\n   DB type int4 is mapped to Spanner type int64.\n\n----------------------------\nUnexpected Conditions\n----------------------------\nFor debugging only. This section provides details of unexpected conditions\nencountered as we processed the pg_dump data. In particular, the AST node\nrepresentation used by the lfittl/pg_query_go library used for parsing\npg_dump output is highly permissive: almost any construct can appear at\nany node in the AST tree. The list details all unexpected nodes and\nconditions.\n  --------------------------------------\n   count  condition\n  --------------------------------------\n       5  Testing unexpected messages\n\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -70,3 +70,3 @@
        	            	 Warning
        	            	-1) Column 'a' uses foreign keys which Spanner does not support.
        	            	+1) Column 'a' uses foreign keys which HarbourBridge does not support yet.
        	            	 
        	Test:       	TestReport
```
nevinheintze: Done!
Thanks for catching.



Patch:
diff --git a/internal/report.go b/internal/report.go
index ed5ff43..accc56a 100644
--- a/internal/report.go
+++ b/internal/report.go
@@ -189,7 +189,7 @@ func buildTableReportBody(conv *Conv, srcTable string, issues map[string][]Schem
 				case DefaultValue:
 					l = append(l, fmt.Sprintf("%s e.g. column '%s'", issueDB[i].brief, srcCol))
 				case ForeignKey:
-					l = append(l, fmt.Sprintf("Column '%s' uses foreign keys which Spanner does not support", srcCol))
+					l = append(l, fmt.Sprintf("Column '%s' uses foreign keys which HarbourBridge does not support yet", srcCol))
 				case AutoIncrement:
 					l = append(l, fmt.Sprintf("Column '%s' is an autoincrement column. %s", srcCol, issueDB[i].brief))
 				case Timestamp:


Test patch:
diff --git a/mysql/report_test.go b/mysql/report_test.go
index 3a12d98..c4c4abc 100644
--- a/mysql/report_test.go
+++ b/mysql/report_test.go
@@ -133,7 +133,7 @@ Schema conversion: POOR (many columns did not map cleanly).
 Data conversion: NONE (no data rows found).
 
 Warning
-1) Column 'a' uses foreign keys which Spanner does not support.
+1) Column 'a' uses foreign keys which HarbourBridge does not support yet.
 
 ----------------------------
 Table no_pk
diff --git a/postgres/report_test.go b/postgres/report_test.go
index bf15a7d..c818ef0 100644
--- a/postgres/report_test.go
+++ b/postgres/report_test.go
@@ -135,7 +135,7 @@ Schema conversion: POOR (many columns did not map cleanly).
 Data conversion: NONE (no data rows found).
 
 Warning
-1) Column 'a' uses foreign keys which Spanner does not support.
+1) Column 'a' uses foreign keys which HarbourBridge does not support yet.
 
 ----------------------------
 Table no_pk