Package: ppss-pyramidutils
OverviewA collection of simple utils for common operations
This includes configuration item from .ini file, a filemanger for hadling file uploads, some utilities for SQLAlchemy models and a CSV reader/writer unicode-safe for python 2.7Configuration from .iniThe class ppss_pyramidutils.Utils offers convenience options to get data from the main ini file, allowing default values.
The easiest way to use it is to subclass it with your conf class and override themyconfproperty:fromppss_pyramidutilsimportUtilsasIniUtilsclassMyClass(IniUtils):myconf=['param1','param2']Then you can read ini file calling the class methodconfig:MyClass.config(settings)This method accetps other optional parameters:prefixanddefaultval.Ifprefixis not passed, lowered case class name is used instead (i.e.: myclass).
Ths config method use  all values inmyconfproperty and read in the ini file the parameter:prefix.valueIn this example it reads myclass.param1 and myclass.param2.
If a key in missing in the ini file, the corresponding class property will assume the valuedefaultval.myconfcan be a tuple/list instead of a string. In this case the first value is the string, the second value is the default value only for that key.A full example can work as follow:#ini filemyprefix.param1=12myprefix.param2=AsimpleStringfromppss_pyramidutilsimportUtilsasIniUtilsclassMyClass(IniUtils):myconf=['param1','param2',('param3',"a missing value with a default"),"param4"]MyClass.config(settings,"myprefix",defaultval="default")MyClass.param1# '12'MyClass.param2# 'A simple String'MyClass.param3# 'a missing value with a default'MyClass.param4# 'param4'Because they are just class values, you can simply include your class anywhere in your code and access the class property required.
This will allow to read and set default for each value in a centralized place and to access it whenever and wherever needed.filemanger (for upload and other purposes)ppss_pyramidutils.FileManagerextendsppss_pyramidutils.Uitlssettingmyconfattribute myconf = ['savepath','tmppath']
You can include the class in the ini of the pyramid app and call theconfigmethod against it to properly config the class.FileManager offer the following class methods:saveToTmp(cls,requestfile)->file_path: takes as argument the file object from the form, replace some chars in the filename and save it to a temp location (specified through ini file). This method returns the file path of the temporary file.moveToDestination(cls,source,filename,subfolder="")->file_path: this method takes a file path (tipically the return value of pthe revious saveToTmp call) and moves it in target folder (as of ini file), with the givenfilename. It can put it in a subfolder ifsubfolderis specified. This will create the folder if required. Returns the complete path of the file.deleteFile(cls,file)-> None : delete a file with the pathfileIt also has two commodity class methods:_sanitizeFilename(cls,filename, whitelist=valid_filename_chars, replace=' ')->sanitizedfilename: replaces all occurency of each char ofreplacestring with a "_", that removes all not allowed char (allowed chat by default are: -, _, ., (, ),string.ascii_lettersandstring.digits). The method returns the sanitized file name. It is called automatically by saveToTmp to prevent attempts of injections in file system.slugify(cls,filename)->sluggifiedfilename: This method tries to convert an arbitrary file name in a sluggied string.CSV reader/writer for python 2.7Python 2.7 CSV reader and writer fail to address many unicode problems.
You can simply useppss_pyramidutils.UnicodeReaderandppss_pyramidutils.UnicodeWriterfor this purpose.BothUnicodeReaderandUnicodeWriter.__init____ methods accept this parameters:f: the already opened file to read.dialect=csv.excel: the csv dialect to use.encoding="utf-8-sig": encoding to use.All other keyword arguments are passed on to the CSV reader (check the module in standard Python 2.7 documentation)For conveninece and use as importer/export of CSV formatted data, two more classes are defined.Importerclass can be initialized with this parameters:fn: File name of the source. It will be opened and passed tomapping=None: allow to remap column names. If present the mapping param must be a dictionary-like object, containing all CSV column names as key, and the mapped names as valuesdelimiter=",": the delimiter for the CSV inputheadertransform=None: a callable that receive each column name and may transform it. Usually used to do some sanitization on the names (ie: lower and remove/substitute forbiden chars)Creating theImporterobject actually triggers the file opening and reading. To get the resulting rows as a list of dictionaries, you can use the methodgetRows.Exporterclass takes this parameters when initialized:evcollection: list-like object, where each item is a dictionary-like (see getter param) object.titles: ordered list-like object, containing column namesparserows= True: if set, allow pre-processing of the input (evcollection), usinggetteranddatetimeformatparamesters.delimiter= ',': the delimiter for the CSV outputgetter= None: a callable to override the dictionary like way to get values. If set the getter will be called for each column with the item and name of the column (ie: val = getter(ev,k)). Only used ifparserowsis True.datetimeformat= "%Y-%m-%d" : formatter for datetime.datetime and datetime.date objects. Only used ifparserowsis True.This method set the propertyretfileto atempfile.NamedTemporaryFile(set to writemode in Python 2)the methodwriteAllaccept only adelimiterparam and actualy writes all rows in theretfileproperty (by default atempfile.NamedTemporaryFileinstance)Modelbase utilitiesppss_pyramidutils.ModelCommonParentattach some commodity methods to derived SQLAlchemy classes, as class methods, like:all(cls,DBSession): returns all elements of the given classbyId(cls,id,DBSession): returns the object withid= id. COC: the class must have a id column.byField(cls,field,value,DBSession): like byId, on an arbitratyfieldbyFields(cls,fields,DBSession): accept a list of filters as tuples of (columnname, value) and returns all items that matchesdelete(cls,element,DBSession): delete the elementdeleteAll(cls,elements,DBSession): delete all elements in elements list-like parameter.__v1.6.1
added skip rows to jsonwalker__v1.6.0
added cryptoutils and jsonutils__v1.5.4.0
added custom header row to importer
__v1.5.3.2
imported names in base init
__v1.5.3.1
fix conf readerv1.5.2New background thread in loop interfaceini reader updated and refactoredv1.5added/refactored functions to have dbsessions for off-request activitiesv1.5added utility for background jobsv1.4.4.5fixed encoding of file in csv importer (now forced to utf-8)create tmp dir if not existsv1.4.4.4addedd order by and order direction to modelbase commodity methodsv1.4.4.3fix in utf8csv change six unicode method to str method for py3 compatibilityv1.4.4.2fix in modelbasestrmethod for py3 compatibility1.4.3.1added check on int/float before encoding for UTF-8 writer