Package: pnu-adsv
InstallationDepending on if you want only this tool, the full set of PNU tools, or PNU plus a selection of additional third-parties tools, use one of these commands:pip installpnu-adsvpip installPNUpip installpytnixADSV(1)NAMEadsv - Analyze delimiter-separated values filesSYNOPSISadsv[-d|--delimiter CHAR]
[-e|--encoding STRING]
[-f|--fields LIST]
[-F|--flatten]
[-h|--hide INT]
[-m|--min INT]
[-M|--max INT]
[-t|--top INT]
[--debug]
[--help|-?]
[--version]
[--]
filename
[...]DESCRIPTIONTheadsvutility analyzesdelimiter-separated valuesfiles, such asComma-Separated Values .csvorTab-Separated Values .tsvfiles, and either prints information about their structure and the data in each of their fields, or prints a selection of fields in the order requested.The information gathered are:for the file:the character set encodingtheCSV dialect(characters used for delimiting, quoting, escaping or lines terminating. Plus the use or not of double quoting)the presence or not of a headers linethe number of lines and fieldsfor each field:its number and headerthe number of distinct valuesthe values type (strings, integers, floating numbers, complex numbers, date and time (whatever their format))the values by descending countthe values range by ascending order using the detected type (useful for numbers and dates)When analyzing a DSV dataset, this allows for a quick and automated way of getting global information about the contents, and explore any oddities...There are options:to control and limit what is printed (-h|--hide,-m|--min,-M|--maxand-t|--top),to avoid (or correct) the detection of the character set encoding and delimiter (-d|--delimiter,-e|--encoding):the character set detection can take a long time with big files, so if you know that the file is in "Windows-1252" or "utf-8" encoding, it's quicker to say it...If you use the-f|--fieldsoption, you'll skip printing the file analysis, and instead print the selected fields in the order requested, using the detected delimiting, quoting and escaping characters.If you encounter multi-lines fields and want to "flatten" them to single lines, you can use the-F|--flattenoption for that.OPTIONSOptionsUse-d|--delimiter CHARSpecify delimiter to be CHAR-e|--encoding STRINGSpecify charset encoding to be STRING (because detecting encoding can take a long time!)-f|--fields LISTExtract LISTed fields values in given order (ex: 6,2-4,1 with fields numbered from 1)-F|--flattenMake multi-lines fields single line-h|--hide INTHide the display of distinct values above INT % (default is 20%)-m|--min INTOnly display distinct values whose count >= INT (default is to display all distinct values)-M|--max INTOnly display INT lines of distinct values (default is to display all distinct values, within the hide limit)-t|--top INTOnly display the top/bottom INT lines of values (default is to display the 5 bottom and top lines)--debugEnable debug mode--help|-?Print usage and a short help message and exit--versionPrint version and exit--Options processing terminatorENVIRONMENTThe ADSV_DEBUG environment variable can also be set to any value to enable debug mode.EXIT STATUSTheadsvutility exits 0 on success, and >0 if an error occurs.SEE ALSOcut(1),file(1)STANDARDSTheadsvutility is not a standard UNIX command.This implementation tries to follow thePEP 8style guide forPythoncode.The DSV dialects that can be handled are those compatible withRFC 4180: Common Format and MIME Type for Comma-Separated Values (CSV) Files.PORTABILITYTested OK under Windows.HISTORYThis implementation was made for thePNU project.I do this kind of analysis with each dataset I have to work with.
Last time I did that, I decided that it was about time to fully automate the process, especially as I was working with fields containing multi-lines values...LICENSEIt is available under the3-clause BSD license.AUTHORSHubert TournierCAVEATSUsing "Sep=X" as a first line in order to set the X character as a delimiter is not supported.There is no support either for potential commented lines inside the data (for example, with/etc/passwdfiles under Unix), but it's not part of any recognized DSV dialect anyway.