madlib/madlib — pull request #180 by aojwang (2012-07-20 10:19:28+00:00)
Title: Enhance the encoding method

Jira: MADLIB-592

In MADlib 0.4 version, as we encode the training table column by column, the scalability is not good and the encoding time is very long for some datasets with a large number of columns, such as internet_ads (it has about 1600 columns). Now, we encode all the columns of training table in parallel.

The steps of the encoding method for training.
```javascript
1. Breakup the training table(n columns and m rows) to a new table with fixed columns (id, fid, fval, is_cont, class) and n * m rows. We call this new table as breakup table.

2. Based on the breakup table, generating the Key-Value (KV) (We only have two KV tables: One is for class and the other is for features)

3. Generate the encoded table by joining the breakup table with the KV table for features.
```

The steps of the encoding method for classification
```javascript
1. Transform the classification table to the breakup table.

2. Generate the encoded table by joining the breakup table with the KV table for features, which is produced during training.
```