[WIP]: shorten url using dynamoDB

I am using my own amzon account at the moment.

So you will have trouble testing.

According to my calculations, it should be quite cheap if we decide to go this way
(less than 5 dollars per month)
http://calculator.s3.amazonaws.com/index.html

Table size:
1’396 MB
Number of rows at the moment:
2’577’960
Number of new rows per day (as of yesterday)
8’121
8’150*365 = 2’974’750 new rows per year

Read/write per seconds (mostly over 12 hours)
8’121/60*60*12 (sec) = 0.18 write per seconds (less than one)


Code is pretty straightforward at the moment, it's just a demo.


Useful links:
http://boto.readthedocs.org/en/latest/getting_started.html
https://github.com/boto/boto
https://boto.readthedocs.org/en/latest/dynamodb2_tut.html
https://boto.readthedocs.org/en/latest/ref/dynamodb2.html

For the old technical url, shouldn't we change them to the new one ?


On Fri, Jun 20, 2014 at 11:24 AM, loicgasser <notifications@github.com>
wrote:

> In chsdi/scripts/pgsql2dynamodb.py:
>
> > +        except:
> > +            parser.print_help()
> > +            sys.exit(1)
> > +
> > +    def parse_url_params(url):
> > +        ''' Malformed parts of the query string
> > +        are dropped automatically
> > +        '''
> > +        # Drop urls containing the old techincal addresses
> > +        if '.bgdi.admin.ch' in url:
> > +            return None
> > +        return dict(urlparse.parse_qsl(
> > +            urlparse.urlparse(url).query
> > +        ))
> > +
> > +    def drop_re2_params(params):
>
> Good point, I'll add also the ones that have been dropped (techincal
> addresses/malformed urls)
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/geoadmin/mf-chsdi3/pull/563/files#r14012374>.
>



-- 
Twitter: http://twitter.com/cedricmoullet
Linked In: http://www.linkedin.com/in/cedricmoullet

<http://twitter.com/cedricmoullet>