Package: feanor-csv
Feanor CSVFeanor is an artisan of CSV files. It can generate complex CSV files or file bundles for examples and tests.Note:Feanor is currently in development. All releases prior to1.0.0should be considered alpha releases
and both the command line interface and the library API might change significantly between releases.
Release1.0.0will provide a stable API and stable command line interface for the1.xseries.Usage$ feanor --help
usage: feanor [-h] [--no-header] [-L LIBRARY] [-C GLOBAL_CONFIGURATION]
              [-r RANDOM_MODULE] [-s RANDOM_SEED] [--version]
              (-n N | -b N | --stream-mode STREAM_MODE)
              {expr,cmdline} ...

optional arguments:
  -h, --help            show this help message and exit
  --no-header           Do not add header to the output.
  -L LIBRARY, --library LIBRARY
                        The library to use.
  -C GLOBAL_CONFIGURATION, --global-configuration GLOBAL_CONFIGURATION
                        The global configuration for arbitraries.
  -r RANDOM_MODULE, --random-module RANDOM_MODULE
                        The random module to be used to generate random data.
  -s RANDOM_SEED, --random-seed RANDOM_SEED
                        The random seed to use for this run.
  --version             show program's version number and exit
  -n N, --num-rows N    The number of rows of the produced CSV
  -b N, --num-bytes N   The approximate number of bytes of the produced CSV
  --stream-mode STREAM_MODE

Schema definition:
  {expr,cmdline}        Commands to define a CSV schema.Checking the version:$ feanor --version
feanor 0.5.0Arbitrary typesEach arbitrary is assigned an "arbitrary type", which describes how to generate the values.
The syntax of the arbitrary type is the following:# <ARBITRARY_NAME> [ CONFIG ]WhereARBITRARY_NAMEmust match\w+andCONFIGis a pythondictliteral.For example the built-inintarbitrary type can be used in the following ways:%intor%int{}: default configuration%int{"min": 10}: do not generate numbers smaller than10(inclusive).%int{"max": 10}: do not generate numbers bigger than10(inclusive).%int{"min": 10, "max":1000}: generate numbers between10and1000(both inclusive).Feanor DSL ExpressionsValues are defined by a simple DSL that allows you to combine multiple arbitraries in different ways and they
allow to express complex logic for your data generation.Arbitrary definitionsAn arbitrary definition is simply its type and follows the syntax#<NAME>[CONFIG]as explained before.AssignmentsYou can assign a name to a certain expression with the syntax(<expr>)=<NAME>.ReferencesYou can refer to the values of an expression to which you assigned a name by using the syntax@<NAME>.ConcatenationYou can concatenate multiple values using the syntax<expr_1> . <expr_2>or<expr_1> · <expr_2>.ChoiceYou can define an expression that can randomly take a value by using the choice operator|using the
syntax<expr_1> | <expr_2>.The value of such expression will take the value ofexpr_1for50%of the time and the value ofexpr_2the other times. You can specify the chances with the syntax:expr_1 <0.3|0.7> expr_2.
In this case the expression will evaluate toexpr_1only30%of the time and toexpr_2the remaining70%of the time. You may omit one of the two numbers, henceexpr_1 <0.3|> expr_2is equivalent to the last
expression.If the sum of the left and right weight add up to a value smaller than1then the remaining portion
is the chance of the value to be empty. For exampleexpr_1 <0.25|0.25> expr_2defines
an expression that in25%of the time evaluates toexpr_1,25%of the time evaluates toexpr_2and50%of the time evaluates toNone(i.e. empty)MergeYou can define an expression that can merge values of two different expressions using the+operator.For example%int + %floatis an expression that evaluates to the sum of a random integer and a random float.ExamplesNOTE:the following examples all specify the option-s 0. This is used solely for reproducibility reason.
The common use cases for Feanor do not need to specify a random seed and in fact doing so often defeats the purpose of the tool.Using thecmdlinesubcommandGenerate 10 rows with random integers:$ feanor -s 0 -n 10 cmdline -c a '%int' -c b '%int'
a,b
885440,403958
794772,933488
441001,42450
271493,536110
509532,424604
962838,821872
870163,318046
499748,375441
611720,934973
952225,229053Generate about 1 kilobyte of rows with 2 random integers in them and write result to/tmp/out.csv:$ feanor -s 0 -b 1024 cmdline -c a '%int' -c b '%int'  /tmp/out.csv
$ head /tmp/out.csv 
a,b
885440,403958
794772,933488
441001,42450
271493,536110
509532,424604
962838,821872
870163,318046
499748,375441
611720,934973Generate 10 rows with random integers, the first column between0and10, the second column between0and1000:$ feanor -s 0 -n 10 cmdline -c a '%int{"min":0, "max":10}' -c b '%int{"min": 0, "max":1000}'
a,b
6,776
6,41
4,988
8,497
6,940
4,991
7,366
9,913
3,516
2,288Generate 10 rows with random integers and their sum:$ feanor -s 0 -n 10 cmdline -c a '%int' -c b '%int' -c c '@a+@b'
a,b,c
885440,403958,1289398
794772,933488,1728260
441001,42450,483451
271493,536110,807603
509532,424604,934136
962838,821872,1784710
870163,318046,1188209
499748,375441,875189
611720,934973,1546693
952225,229053,1181278Using theexprsubcommandGenerate 10 rows with random integers:$ feanor -s 0 -n 10 expr -c a,b '%int·%int'
a,b
885440,403958
794772,933488
441001,42450
271493,536110
509532,424604
962838,821872
870163,318046
499748,375441
611720,934973
952225,229053Generate about 1 kilobyte of rows with 2 random integers in them and write result to/tmp/out.csv:$ feanor -s 0 -b 1024 expr -c a,b /tmp/out.csv '%int·%int'
$ head /tmp/out.csv 
a,b
885440,403958
794772,933488
441001,42450
271493,536110
509532,424604
962838,821872
870163,318046
499748,375441
611720,934973Generate 10 rows with random integers, the first column between0and10, the second column between0and1000:$ feanor -s 0 -n 10 expr -c a,b '%int{"min":0, "max":10}·%int{"min": 0, "max":1000}'
a,b
6,776
6,41
4,988
8,497
6,940
4,991
7,366
9,913
3,516
2,288Generate 10 rows with random integers and their sum:$ feanor -s 0 -n 10 expr -c a,b,c '(%int)=a·(%int)=b·(@a+@b)'
a,b,c
885440,403958,1289398
794772,933488,1728260
441001,42450,483451
271493,536110,807603
509532,424604,934136
962838,821872,1784710
870163,318046,1188209
499748,375441,875189
611720,934973,1546693
952225,229053,1181278or also:$ feanor -s 0 -n 10 expr -c a,b,c 'let a:=%int b:=%int in @a·@b·(@a+@b)'
a,b,c
885440,403958,1289398
794772,933488,1728260
441001,42450,483451
271493,536110,807603
509532,424604,934136
962838,821872,1784710
870163,318046,1188209
499748,375441,875189
611720,934973,1546693
952225,229053,1181278