Package: experiment-server
OverviewThis is a python application that allows you to create/maintain/manage study configurations away from your implementations.experiment-serverhas several different interfaces (see below) to allow using it in a range of different scenarios. I've used with python, js andUnity projects. Seewikifor examples.SetupRequirementsPython 3.8+InstallationInstall it directly into an activated virtual environment:$ pip install experiment-serveror add it to yourPoetryproject:$ poetry add experiment-serverUsageConfiguration of an experimentThe configuration id defined in atomlfile. See example.tomlbelow for how the configuration can be defined.# The `configuration` table contains settings of the study/experiment itself[configuration]# The `order` is an array of block names or an array of array of block names.order=[["conditionA","conditionB","conditionA","conditionB"]]# The `groups` and `within_groups` are optional keys that allows you to define how the# conditions specified in `order` will be managed. `groups` would dictate how the top# level array of `order` will be handled. `within_groups` would dictate how the conditions# in the nested arrays (if specified) would be managed. These keys can have one# of the following values.# - "latin_square": Apply latin square to balance the values.# - "randomize": For each participant randomize the order of the values in the array.# - "as_is": Use the order of the values as specified.# When not specified, the default value is "as_is" for both keys.groups="latin_square"within_groups="randomize"# The random seed to use for any randomization. Default seed is 0. The seed will be# the value of random_seed + participant_indexrandom_seed=0# The subtable `variabels` are values that can be used anywhere when defining the blocks.# Any variable can be used by appending "$" before the variable name in the blocks. See# below for an exmaple of how variables can be used[configuration.variables]TRIALS_PER_ITEM=3# Blocks are defined as an array of tables. Each block must contain `name` and the# subtable `config`. Optionally, a block can also specify `extends`, whish is a `name` of# another block. See below for more explanation on how `extends` works# Block: Condition A[[blocks]]name="conditionA"# The `config` subtable can have any key-values. Note that `name` and `participant_index`# will be added to the `config` when this file is being processed. Hence, those keys# will be overwritten if used in this subtable.[blocks.config]trialsPerItem="$TRIALS_PER_ITEM"param1=1# The value can also be a function call. A function call is represented as a table# The following function call will be replaced with a call to# [random.choices](https://docs.python.org/3/library/random.html#random.choices)# See `# Function calls` in README for more information.param2={function_name="choices",args={population=[1,2,3],k=2}}param3={function_name="choices",args=[[1,2,3]],params={unique=true}}# Block: Condition B[[blocks]]name="conditionB"extends="conditionA"# Since "conditionB" is extending "conditionA", the keys in the `config` subtable of# the block "conditionA" not defined in the `config` subtable of "conditionB" will be copied# to the `config` subtable of "conditionB". In this example, `param1`, `param2` and# `trialsPerItem` will be copied over here.[blocks.config]param3=[2]Seetoml specfor more information on the format of a toml file.The above config file, after being processed, would result in the following list of blocks for participant number 1:[{"name":"conditionB","extends":"conditionA","config":{"param3":[2],"trialsPerItem":3,"param1":1,"param2":[1,2],"participant_index":1,"name":"conditionB"}},{"name":"conditionA","config":{"trialsPerItem":3,"param1":1,"param2":[2,2],"param3":[3],"participant_index":1,"name":"conditionA"}},{"name":"conditionA","config":{"trialsPerItem":3,"param1":1,"param2":[1,1],"param3":[2],"participant_index":1,"name":"conditionA"}},{"name":"conditionB","extends":"conditionA","config":{"param3":[2],"trialsPerItem":3,"param1":1,"param2":[3,1],"participant_index":1,"name":"conditionB"}}]Verify configA config file can be validated by running:$experiment-serververify-config-filesample_config.tomlThis will show how the expanded config looks like for the first 5 participant.Loading experiment through serverAfter installation, the server can used as:$experiment-serverrunsample_config.tomlSee more options with--helpA simple web interface can be accessed at/or/indexThe server exposes the following REST API:[GET]/api/items-count: The total number of blocks. Returns an integer[GET]/api/active: Test if the server is working. Returns boolean[GET]/api/config: Return theconfigsubtable in the configuration file of the current block as a json object. Note thatmove-to-nexthas to be called atleast once before this can be called.[GET]/api/block-id: Returns the current block id[GET]/api/status-string: Returns the staus as a string[GET]/api/global-data: Returns a json object, with the following keys:"participant_index": the participant index"config_length": same value/items-count[GET]/api/all-configs: Return allconfigs of all the blocks as a list, ordered based on theorderfor the configured participant.[POST]/api/move-to-next: Sets the current block to the next block in the list of blocks. Returns a json object, with the key "names", which is the name of the current block after moving. If there are no more blocks, the value of "names" will be "end".[POST]/api/move-to-block/:block_id: Set the block at indexblock_idin the list of blocks as the current block.[POST]/api/shutdown: Shutdown the server.[POST]/api/change-participant-index/:participant_index: Set the participant_index to valueparticipant_index. Note that this will set the sate back to the initial state as if the server was freshly stared.For a python application,experiment_server.Clientcan be used to access configs from the server. Also, the server can be launched programatically usingexperiment_server.server_processwhich returns aProcessobject. Note that the server will reload the config and reset the state to the initial state when the config file loaded is modified.Loading experiment through APIA configuration can be loaded and managed by importingexperiment_server.Experiment.Generate expanded config filesA config file (i.e..tomlfile), can be expanded to json files with the following command$experiment-servergenerate-config-jsonsample_config.toml--participant-range5The above will generate the expanded configs for participant indices 1 to 5 as json files. See more options with--helpFunction calls in configA function call in the config is represented by a table, with the following keysfunction_name: Should be one of the names in the supported functions list below.args: The arguments to be passed to the function represented byfunction_name. This can be a list or a table/dict. They should unpack with*or**respectively when called with the corresponding function.(optional)params: function specific configurations to apply with the function calls.(optional)id: A unique identifier to group function calls.A table that has keys other than the above keys would not be treated as a function call. Any function calls in different places of the config with the sameidwould be treated as a single group. Tables without anidare grouped based on their key-value pairs. Groups are used to identify how some parameters effect the results (e.g.,uniqueforchoices). Function calls can also be inconfigurations.variabels. Note that all function calls are made after theextendsare resolved and variables fromconfigurations.variabelsare replaced.Supported functionschoices: Callsrandom.choices.paramscan be a table/dictionary which can have the keyunique. The value ofuniquemust betrueorfalse. By defaultuniqueisfalse. If it'strue, within a group of function calls, no value from the population passed torandom.choicesis repeated for a given participant.Example function callsparam={function_name="choices",args=[[1,2,3,4]],params={unique=true}}param={foo="test",bar={function_name="choices",args={population=["w","x","y","z"],k=1}}}For more on theexperiemnt-serverand how it can be used see thewikiWishlist (todo list?)Serve multiple participants at the same time.Improved docsAdd the option of using dict values in orderImprove cli help docs