Update README.md
Update README.md
Merge branch 'master' of github.com:frankV/pypubcrawl
local repo added to public github repo/profile. will need to change things up a bit to keep this secure. jester awaits an ssh key... will want to pass directory path as a sys.arg in the near future.
removed more unecessary files
Update README.md
database insertions are working <- problem was error in how I was tryign to load the table from db with sqlalchemy. currently table is dropped and then created... will rework this in near future. TO ADD: 1.sys args (verbose, help, pass in /usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin, etc.) 2.encode pickle filenames, store in a sub-dir 3.make db integration more efficient (dont drop and create everytime) 4.create db schema and setup foriegn keys. Almost there
database insertions are working <- problem was error in how I was tryign to load the table from db with sqlalchemy. currently table is dropped and then created... will rework this in near future. TO ADD: 1.sys args (verbose, help, pass in /usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin, etc.) 2.encode pickle filenames, store in a sub-dir 3.make db integration more efficient (dont drop and create everytime) 4.create db schema and setup foriegn keys. Almost there
database insertions are working <- problem was error in how I was tryign to load the table from db with sqlalchemy. currently table is dropped and then created... will rework this in near future. TO ADD: 1.sys args (verbose, help, pass in path to crawl directory, etc.) 2.encode pickle filenames, store in a sub-dir 3.make db integration more efficient (dont drop and create everytime) 4.create db schema and setup foriegn keys. Almost there
database insertions are working <- problem was error in how I was try…
…ign to load the table from db with sqlalchemy. currently table is dropped and then created... will rework this in near future. TO ADD: 1.sys args (verbose, help, pass in path to crawled directory, errors, etc.) 2.encode pickle filenames, store in a sub-dir 3.make db integration more efficient (dont drop and create everytime) 4.create db schema and setup foriegn keys. Almost there
boss might read this, removed word "drunkenly"
changes to dir_crawler, control comes from command line parameters, path must be passed in now
added requirements.txt for when running from within python virtualenv
Merge branch 'master' of github.com:frankV/pythonpubcrawl

merging on jester.coaps, need to resync files to reflect update of project on dev.coaps
ignore .p pickle extensions
added comments regarding dependencies
added comments regarding dependencies and usage
test
changed locations of certain files
add pickler./py, will use later