soundcloud/large-hadron-migrator — Issue #11: table parser does not parse columns with only name and type. this leads to data loss over these column types during the copy phase/
Author: purzelrakete
Created: 2012-02-14 08:14:25+00:00
State: open

Lhm::Table::Parser is used to parse the strucutre of origin and destination tables in a migration. The resulting intersection determines which columns are copied in the chunking phase.

Lhm::Table::Parser currently parses the output of show create table. An error in the regular expression for columns causes the following table definition to parse incorrectly:

CREATE TABLE `users` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`reference` int(11) DEFAULT NULL,
`username` varchar(255) DEFAULT NULL,
`group` varchar(255) DEFAULT NULL,
`created_at` datetime DEFAULT NULL,
`comment` varchar(20) DEFAULT NULL,
`description` text,
PRIMARY KEY (`id`),
UNIQUE KEY `index_users_on_reference` (`reference`),
KEY `index_users_on_username_and_created_at` (`username`,`created_at`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8

Parsing this ddl will fail to add the `description` column to the parsed table.

A more robust Table parser using information_schema and no regular expressions fixes this problem.