[Android Apps and Games] hhenault:
eXtended Sort/Merge for Android

Advanced Use​

Those wanting more may directly invoke the hxsm command in a terminal emulator

For help, just enter 'hxsm' at the command prompt :

usage :

hxsm -c, --check -q|-v, --quiet|--verbose -h, --help

-m, --merge |--copy[|--sort]

-k, --key=all | start,len[,order(A|D),[format(B|C|I|P|Z|Y)] ]

-r, --recfm=F|V|M

-z|-l, --lrecl=nnnn

-i, --infile=in_id1[,recfm=x[,lrecl=nnn] --infile=in_id2....

-o, --outfile=out_id[,recfm=x[,lrecl=nnn]]

--outrec=(in_pos1,len1,out_pos1,type1[in_pos2,...])

-f, --field-separator=TAB|BAR|COMMA|COLUMN|SEMICOLUMN|c|0Xhh

-uk|-ur, --unique-key|--unique-record

--include=start,len,op,val[AND|OR,start,len,op,val...]

--exclude=start,len,op,val[AND|OR,start,len,op,val...]

-t --sortwork=dir1,dir2,.. --sortwork=dir3,dir4,..

-y, --storage=nnnK|M|G

--keep-order

-f --field-separator=TAB|COMMA|COLUMN|SEMICOLUMN|c|0Xhh

--record-separator=c|0xhh

--collating-sequence=ascii|ebcdic

--skip-head=nnn

--throw-empty-records

-E --ignore-ioerror [ = ignore short lines ]

--norun

Example for the job given as an example in post #1 :

hxsm --verbose --recfm=V --lrecl=250 --key=1,10 --input=/sdcard/tmp/50mt.txt --output=/sdcard/tmp/6=50mt-sorted.txt/FONT]

or

hxsm -v -rV -l250 -k 1,10 -i/sdcard/tmp/50mt.txt -o/sdcard/tmp/6=50mt-sorted.txt/FONT]

Explanation for some usefull options

-k : sort/merge keys

You may specify several keys

For each key, specify the start column (byte number starting at 1), the key length in bytes, and eventualy the letter 'D' for descending order.

Furthermore, for each key, you may also have 'specific' fields format, most inherited from IBM main frames, like 'Packed' or 'Zoned' decimal fields, or just 'Numeric'.

example :

hxsm ... -k 14,3,D,P --key=1,10,A,Z -k35,40 ... --recfm=F --lrecl=100

This will sort the file on a global key including:

a packed, signed decimal field (5 digits + sign), descending order

a zoned, signed, decimal field (10 digits, including sign)

an alphanumeric field (40 bytes).

Note that in that case, the file cannot be a text file, because possible x'0D' inside the packed field : it should be specified as 'Binary Fixed' (--recfm=F)

-i : input file

you can have several input files, provided that the sort key are at the same place in all files

example :

hxsm ... -i file1 --infile=file2 -ifile3 ...

--skip-head=nnn : throws the nnn fist lines/records

-m : merge several input files already sorted on the same key

example :

hxsm -m -o resulting_file -i file1 --infile=file2 -ifile3 ...

-t : sort-work directories

It may improve perpormances, provided you have more than 1 sdcard (a fast one, class 10), or a fast usb stick connected

If this is the case, just specify a workibg directory on the corresponding mount point :

example :

hxsm ... -t /mnt/sdcard/external_sd/tmp (and check you've done a chmod 777 on it ...)

Input/Output

if the -i option is omitted, then the program will read from stdin

if the -o option is omitted, then the program will write onto stdout

Filtering

You may filter the lines or records you want to be written onto the output :

-uk : unique key

if several lines/records have the same key, then only one will be written

-ur : unique record

if several lines/records are strictly identical, then only one will be written

--include=start,len,op,val ...

--exclude=start,len,op,val ...

takes in, or omits, lines/records whose field(s) responds to some criterias

Examples:

hxsm ... --include=12,3,EQ,C'ABC' ...

this will keep only records where there is 'ABC' in cols. 12-14

hxsm ... --include=12,3,EQ,C'ABC',OR,12,13,EQ,c'ABD' ...

this will keep only records where there is 'ABC' or 'ABD' in cols. 12-14

hxsm ... --exclude=15,1,EQ,C'Z' --include=12,3,EQ,XC'ABC' ...

this will throw all records having a 'Z' in col. 15,

then, in the records left, keep only those having 'ABC' in cols 12-14

Sorting 'CSV-like' files

If the input file(s) has no fixed keys location, but rather varying length fields,

then the fields are "SOMETHING SEPARATED" (i.e. column, semi-column, tabulation ...)

In this case, just specify the 'SOMETHING' with the -f (--field-separator=...) option.

The program will considered that all lines are made of fields separated by 'SOMETHING',

and that each length given in the '--key=' option is a MAXIMUM length.

For 'SOMETHING', you may specify :

--field-separator=TAB (0x09) or BAR (0x7C) or COLUMN (0x3A) or SEMI[-]COLUMN (0x3B)

or any hexadecimal value 0x00 .. 0XFF

Example :

hxsm ... -f TAB -k 3,5,D --key=1,10,A -k 5,40 -k2,9,A,N ...

This will sort the file on a global key including:

an alphanumeric key in field #3, max length 5 chars, descending order

an alphanumeric key in field #1, max length 10 chars

an alphanumeric key in field #5, max length 40 chars

a numeric key in field #2, max length 9 digits

All fields being separated by a TAB char (0x09).