Package: crimestat3000
crimestat3000A tool for automated parsing of Russian crime statistics reports (form 4-ЕГС) fromcrimestat.ru. All you need to know is which section of report you need, which sheets and columns. (Beware: these tend to change over the years so make sure to check for that and if needed separate you parsing process into several parts with different configurations.)There's no need to download files manually --crimestat3000will take care of that without generating temporary files. But if you happen to have the files locally you can pass the path to their location tolocal_dirargument to slightly increase processing speed.Important: a 4-ЕГС report mostly shows cumulative sums since the beginning of the year. By defaultcrimestat3000turns them into monthly values -- one can switch it off by settingcumsumargument toTrue.You can also optionally specify the level of detail you need. Some sheets contain information on a previously mentioned article's specific part or paragraph -- you can drop those or keep those or just start with parsing all the sheets there are to decide knowingly later.Finally you can setshorten_descrargument toTrueto turn column names likeСтрока 12: умышленное причинение легкого вреда здоровью, совершенное по мотивам политической, идеологической, расовой, национальной или религиозной ненависти или вражды либо по мотивам ненависти или вражды в отношении какой-либо социальной группы п. «б» ч. 2 ст. 115 УК РФto115_ч2_б. It is neat -- but keep in mind that you should use shortener only if you are interested just in the sheets dedicated to some specific article or an article's part/paragraph. If no article is mentioned the shortner will return the sheets number with "no articles mentioned" comment instead of a proper column name: e.g.Строка 3: небольшой и средней тяжестиturns intoСтрока 3: no articles mentioned.To installcrimestat3000use pip:pip install crimestat3000Here's an example call:import crimestat3000 as cs

kwargs = {
    'first_month': '01-2016',
    'last_month' : '12-2016',
    'section'    : 2,

    # optional arguments                                  
    # ==================                                  ========
    # 'sheets'       : {'all', a list of sheets}          # defaults to 'all'

    # 'keep'         : {'all', 'articles', 'articles+'}   # 'all'       -- get all sheets (default).
                                                          # 'articles'  -- all sheets with an article mentioned in description,
                                                          #                but not the sheets with specific article part or paragraph:
                                                          #                i.e. it will get you 228 data but not 228.1.
                                                          # 'articles+' -- all sheets with anything specific mentioned.

    'columns'      : ['C', 'E'],                          # defaults to 'C', usually the sheet's total.
                                                          # Include only the value columns in your list -- 
                                                          # regions column is always included automaticly. 
    'shorten_descr': True                                 # defaults to False
    # 'local_dir'    : {None, path to a local directory}  # defaults to None
    # 'cumsum'       : {True, False}                      # defaults to False
}

table_2016 = cs.parse.period(**kwargs)(OLE2 inconsistencywarnings may pop up sometimes -- don't worry about that: it happens while reading an .xls file content into pandas because some of the files at crimestat.ru are a little bit malformed -- but it doesn't affect anything.)