Repository: weecology/retriever

Issue:
Unable to install nla into mysql
retriever throw an error when installing nla dataset into mysql
```
$ retriever install mysql nla
=> Installing nla
Creating database nla...

Inserting data from nla2007_basin_landuse_metrics_20061022.csv...
Installing nla.basin_landuse_metrics_2007: 100%|█████████████████████████████████████████████████████████████████████████████████████████| 1157/1157 [00:00<00:00, 1794.17rows/s]
Inserting data from nla2007_profile_20091008.csv...
Installing nla.profile_2007: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 12670/12670 [00:02<00:00, 5400.99rows/s]
Inserting data from nla2007_visualassessment_20091015.csv...
Installing nla.visualassessment_2007: 100%|██████████████████████████████████████████████████████████████████████████████████████████████| 1252/1252 [00:00<00:00, 1163.85rows/s]
Couldn't create table ((1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'repeat TEXT, site_type TEXT, lake_samp TEXT, tnt TEXT, lon_dd DOUBLE, lat_dd DOU' at line 1")). Trying to continue anyway.
Inserting data from nla2007_sampledlakeinformation_20091113.csv...
Installing nla.sampledlakeinformation_2007:  77%|████████████████████████████████████████████████████████████████████▍                    | 962/1252 [00:00<00:00, 2315.29rows/s]INSERT INTO nla.sampledlakeinformation_2007 (site_id, visit_no, sampled, date_col, repeat, site_type, lake_samp, tnt, lon_dd, lat_dd, albers_x, albers_y, fld_lon_dd, fld_lat_dd, fld_src, fld_flag, st, state_name, cntyname, epa_reg, nhdname, lakename, area_cat7, neslake, neslake_id, stratum, panel, dsgn_cat, mdcaty, wgt, wgt_nla, adjwgt_cat, urban, wsa_eco3, wsa_eco9, eco_lev_3, eco_l3_nam, nut_reg, nutreg_name, eco_nuta, lake_origin, eco3_x_origin, ref_cluster, refclus_name, rt_nla, ref_nutr, area_ha, size_class, lakearea, lakeperim, sld, depth_x, depthmax, elev_pt, huc_2, huc_8, reachcode, com_id, index_samp, status_ver, status_fld, status_dsk, perm_water, non_saline, srfc_area, meter_deep, open_water, aquacultur, disposal, sewage, evaporate, phys_acces, flag_info, comment_info, sampled_info, sampled_profile, sampled_secchi, sampled_assess, sampled_phab, indxsamp_phab, sampled_chem, indxsamp_chem, sampled_chla, indxsamp_chla, sampled_zoop, indxsamp_zoop, indxsamp_phyt, sampled_core, indxsamp_core, sampled_inf, indxsamp_inf, sampled_ente, indxsamp_ente, sampled_micr, indxsamp_micr, sampled_sdhg, indxsamp_sdhg, visit_id, fid_1) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)
(1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'repeat, site_type, lake_samp, tnt, lon_dd, lat_dd, albers_x, albers_y, fld_lon_d' at line 1")
Installing nla.sampledlakeinformation_2007:  80%|███████████████████████████████████████████████████████████████████████                  | 999/1252 [00:00<00:00, 1500.01rows/s]
Done!
```

Looks like a table named `nla2007_visualassessment_20091015` is the reason for this


Discussion:
Thanks for opening up the issue @adhaamehab!

MySQL has some reserved keywords that can't be used while making tables. One of those words is "repeat" which happens to be a column name in the table nla2007_visualassessment_20091015. 

Changing the column name to something closely related allowed me to add the table into mysql! 
Nice detective work @zhangcandrew!

We can fix this by adding this column name and an appropriate replacement to `replace_dict` in `clean_column_name`:
https://github.com/weecology/retriever/blob/76fcc677e4545a8fbe85be2cab6dd3cc90b2e4b9/retriever/lib/table.py#L153

@adhaamehab - are you interested in testing and submitting a PR for this solution?
@zhangcandrew  Thanks.
@ethanwhite  I'll open a PR of this and fix it.

Resolving patch:
diff --git a/retriever/lib/table.py b/retriever/lib/table.py
--- a/retriever/lib/table.py
+++ b/retriever/lib/table.py
@@ -165,7 +165,8 @@ def clean_column_name(self, column_name):
             "table": "tables",
             "update": "updates",
             "date": "record_date",
-            "index": "indices"
+            "index": "indices",
+            "repeat": "repeats"
         }
         for x in (")", "\n", "\r", '"', "'"):
             replace_dict[x] = ''


Test patch: