[WIP]: shorten url using dynamoDB

I am using my own amzon account at the moment.

So you will have trouble testing.

According to my calculations, it should be quite cheap if we decide to go this way
(less than 5 dollars per month)
http://calculator.s3.amazonaws.com/index.html

Table size:
1’396 MB
Number of rows at the moment:
2’577’960
Number of new rows per day (as of yesterday)
8’121
8’150*365 = 2’974’750 new rows per year

Read/write per seconds (mostly over 12 hours)
8’121/60*60*12 (sec) = 0.18 write per seconds (less than one)


Code is pretty straightforward at the moment, it's just a demo.


Useful links:
http://boto.readthedocs.org/en/latest/getting_started.html
https://github.com/boto/boto
https://boto.readthedocs.org/en/latest/dynamodb2_tut.html
https://boto.readthedocs.org/en/latest/ref/dynamodb2.html

they should not... but a quick log analytic can help here...@clausen?


On Tue, Jun 24, 2014 at 1:11 PM, loicgasser <notifications@github.com>
wrote:

> In chsdi/scripts/pgsql2dynamodb.py:
>
> > +        except:
> > +            parser.print_help()
> > +            sys.exit(1)
> > +
> > +    def parse_url_params(url):
> > +        ''' Malformed parts of the query string
> > +        are dropped automatically
> > +        '''
> > +        # Drop urls containing the old techincal addresses
> > +        if '.bgdi.admin.ch' in url:
> > +            return None
> > +        return dict(urlparse.parse_qsl(
> > +            urlparse.urlparse(url).query
> > +        ))
> > +
> > +    def drop_re2_params(params):
>
> Do you think people still have some of those?
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/geoadmin/mf-chsdi3/pull/563/files#r14123550>.
>