Package: expflow
ExpflowDescriptionExpflow is a Python library that controls the flow and handles the data of psychological experiments. While it isn't a
full-fledged experiment builder likePsychoPy, it may be used with other tools to create
robust experiments with minimal coding.InstallationExpflow is available on PyPI and can be installed in the typical way withpip:pipinstallexpflowTutorialOverviewExpflow has strong opinions about how psychological experiments should be structered and coded, and performs extensive
automatic bookkeeping and numerous validation checks under the hood to ensure that the user's code is consistent with
these opinions.Expflow defines an experiment as a sequence of trials presented to a participant. Typically, trials are identical except
for a small number of critical details, such as the stimulus. The participant makes a response on each trial, after
which the trial ends. The next trial begins after the previous one. After the last trial, the experiment is finished.LoggingExpflow makes extensive use of theloggingmodule. The first few times
you use expflow, I recommend setting up basic logging at the most verbose level at beginning of your experiment script,
which will allow you to see exactly what expflow does. If blazing fast performance is not a major concern — which
typically it isn't, because psychological experiment are often simple things and modern computers are powerful — you
could leave verbose logging switched on all the time (that's what I do).importlogginglogging.basicConfig(level=logging.DEBUG)logging.debug("Hello, world!")The rest of this tutorial will now print many log message to the console.Importing expflowImport expflow as follows.importexpflowDirectoriesTemporary directorySee the bit about creating a temporary directory in the log output?Expflow needs somewhere to store data right away, and the default behaviour is to use a temporary directory. This is
suitable for testing, but a bad idea for real work, because the data you generate will be lost when the program ends.
Expflow will bug you with warnings if you don't set a permanent data directory. For example, try this:expflow.get_expflow_dir()# produces a warningPermanent directoryTo set a permanent directory, use theset_expflow_dirfunction. You can pass any valid writable path to a directory. A
good choice is a dedicated subdirectory of your home directory. For convenience, expflow provides this as a constant
calledexpflow.USER_DIR.expflow.set_expflow_dir(expflow.USER_DIR)# set a direxpflow.get_expflow_dir()# prints something like `/Users/username/Expflow`SubdirectoriesThe functionset_expflow_dircreates several subdirectories if they don't already exist. These are used to store
different types of data. Currently, participant data are stored in theparticipantssubdirectory and experiment data
are stored in theexperimentsubdirectory. The others will be used by future versions of expflow.Cleaning upBefore we go on, let's take a moment delete some files that may exist in your user directory if you ran this tutorial
previously.foriinrange(1,10):(expflow.USER_DIR/"Participants"/f"example_p{i}.json").unlink(True)(expflow.USER_DIR/"Experiments"/f"example_p{i}.example_e1.json").unlink(True)ParticipantsCreating participantsA participant is a person who takes part in an experiment. In expflow, a participant is represented by aParticipantobject. You should create an instance of this class for each participant in an experiment.p=expflow.Participant("example_p1")# creates a new participantParticipant objects aredataclasses. The single required argument
becomes a field calledparticipant_id, which must be a unique string for each participant. Other optional fields you
can set are:dob: Date of birth. Should be adateobject.age: Age. Should be anintorfloat. Doesn't make sense to use this ifdobis specified.gender: Participant gender.language: Participant language.comments: Any comments about the participant.group: Participant group.There are other fields as well. You can see them all like so:importdataclassesforfieldindataclasses.fields(p):logging.info(f"{field.name}:{getattr(p,field.name)}")However,don't go changing the values of the other fields willy-nilly!Expflow manages themautomatically and uses them for bookkeeping. Generally, I recommend setting participant fields at object instantiation
and, except perhaps for thecommentsfield, never changing them.Saving and loadingWhen we created our participant object, it automatically saved aJSONrepresentation of itself to a file in theparticipantssubdirectory, whose path is given by thepathfield. This
will always happen every time we create a participant object and there is no way to stop it, by design. A participant
object will also save itself before garbage collection.delp# delete the participantThis "autosaving" feature allows expflow to enforceGolden Rule #1: A given participant can't be created twice. An
exception will be raised if you try to create a new participant with the sameparticipant_idas an existing
participant, even if the older participant was not created during the current Python session.try:p=expflow.Participant("example_p1")# even though we deleted `p`!raiseRuntimeError("You won't see this message")exceptexpflow.ParticipantExistsErroraser:logging.error(er)Sometimes you may need to load a participant, for example to add something to theircommentsfield. You can do so
using theloadclass method, which requires theparticipant_id.p=expflow.Participant.load("example_p1")p.comments+="- Here's a comment\n"p.comments+="- Here's another\n"delp# autosave on deletionYou can overtly save the participant with thesavemethod, but you shouldn't need to. If you want to quickly get at
the participant data for testing or debugging purposes, you can use theto_dictandto_jsonmethods. But otherwise,
participant objects will save on garbage collection.CompressionBy default, JSON representations of participants are uncompressed, but you can usegzipcompression instead by setting the optional fieldcompressiontoTruewhen creating a new participant or setting the
global variableexpflow.using_compressiontoTrueto turn on compresson by default. Compressed files usually much
smaller, but not human readable, and have the extension.json.gzinstead of.json.Experiments and trialsCreating experimentsExperiments are represented by theExperimentdataclass. You must create an instance of this class each time a new
participant is about to run an experiment. Experiment objects have two required fields:participant_idandexperiment_id.e=expflow.Experiment("example_p1","example_e1")Thecombinationof these identifiers must be unique. In other words, a single participant can perform multiple
different experiments, and multiple different participants can perform the same experiment, butGolden Rule #2: A
given participant can't perform a given experiment twice.. You can load an experiment object with an existing
combination of identifiers.deletry:e=expflow.Experiment("example_p1","example_e1")raiseRuntimeError("You won't see this message")exceptexpflow.ExperimentExistsErroraser:logging.error(er)e=expflow.Experiment.load("example_p1","example_e1")Under the hood, this is enforced by saving on creation and garbage collection.You also can't create an experiment if the participant doesn't exist.try:_=expflow.Experiment("example_p2","example_e1")raiseRuntimeError("You won't see this message")exceptexpflow.ParticipantDoesNotExistErroraser:logging.error(er)Finally, experiments have an optional user-specified field calledtrials, which is discussed in the next section. You
can set this on instantiation or later via special methods.Creating trialsExperiments contain trials. Trials are represented by instance of theTrialdataclass, but unlike participant and
experiment dataclasses, they can't be saved or loaded individually (maybe in a future version). Expflow doesn't insist
on trials having unique identifiers or on required fields, either.There are a number of optional fields:stimulusresponsetrial_numberblock_numberconditionpracticeIt should be obvious what each of these is supposed to represent.Let's create a list of trials.trials=[expflow.Trial(trial_number=i)foriinrange(3)]Appending trials to an experimentInside an experiment object, trials are stored in field calledtrials, which is a list ofTrialobjects. The
experiment we created currently has an emptytrialsfield.assertlen(e.trials)==0This is a good time to bring upGolden Rule #3: Run experiments by iterating experiment objects. Accordingly,eis iterable and its__len__method returns the number of trials it contains.assertlen(e)==0We can append the trials we created to the experiment using theappend_trialsmethod.e.append_trials(trials)You can append to or otherwise directly modifye.trialsbut please don't, because theappend_trialsmethod does
extra things like checks you are actually appendingTrialinstances and saves the experiment object.After appending, the experiment has three trials.assertlen(e)==3Experiments save themselves after a trial is appended, so if we delete the experiment and reload it, we will see that
the appended trials are there.delee=expflow.Experiment.load("example_p1","example_e1")assertlen(e)==3Other manipulations to the trial listCurrently, there are onlyappend_trials,append_trial, andinsert_trialmethods, but more may be added in future
versions of expflow.Experiment flowThis section will describe the core features that allows expflow to control experiment flow. First, it is important to
know about two variables inside experiment objects: the trial index and the status.Trial indexExperiments have a field calledtrial_index, which is automatically managed. Its value isNoneon instantiation, and
becomes an integer when the experiment runs. Predictably, this is used to index the experiment's current position in
the trial list. User's shouldn't set this field themselves.StatusesExperiments (and trials) have a specialstatusproperty (mirrored by thecurrent_statusfield). Users shouldn't set
this property or its field themselves, but they can read it or test its value withis_*boolean properties.There are only six possible statuses and only certain status transitions are possible. The possible statuses, their
meanings, and acceptable transitions are given in the table below.StatusDescriptionAcceptable transitions"pending"Trial is scheduled to run laterrunning, skipped"running"Trial is running right nowfinished, timed_out, paused"paused"Trial is temporarily pausedrunning"timed_out"Trial went on too long and has ended-"finished"Trial ended as expected-"skipped"Trial will not run-This is designed to conisistent with the common usage of the words. On instantiation, the status of an experiment is
always"pending". This is natural, because experiments are created before they are run. Pending experiments can be
changed to"running"or"skipped". Running experiments can be changed to"finished"if they were completed normally,"paused"if they were paused, or"timed_out"if they were timed out. Paused experiments must be unpaused (i.e., set back to"running") before they can be"finished"."finished","timed_out", and"skipped"are terminal statuses. Hopefully this all makes intuitive sense.Individual trials have a status property that behaves in exactly the same way.As we shall see, experiment and trial statuses are managed automatically as we run an experiment. In fact, this is the
major trick expflow employs to ensure proper flow.Running experimentsAs per Golden Rule #3, to run an experiment, you iterate over the experiment object, such as in aforloop.defshow_stimulus(stimulus):"""Replace this with something that presents stimuli."""passdefget_response():"""Replace this with something that collects responses."""return"response"fortrialine:# do experimental stuff here, for example ...show_stimulus(trial.stimulus)trial.response=get_response()# ... end of experimental stuffifnote.is_running:breakThis isalmosta completely normal Pythonforloop. I say almost because it contains anifstatement that will
prematurely break the loop if the experiment status is no longer set to"running". This is necessary to catch pauses,
skips, and time outs (discussed later).If your experiment is embedded within a larger program and it is not convenient or possible to use theforsyntax,
you could usenext(e)instead, but just remember to catch theStopIterationexception.Saving dataNotice that in our toy experiment above, theresponsefield of the current trial was set to the participant's response
on that trial. How do we make sure those responses are recorded?Eachtrialin the loop was a reference to the experiment's current trial (also available viaself.current_trial).
Therefore, because we settrial.responseto"response", participant responses are available after iteration (i.e.,
after the experiment has finished).fortrialine.trials:# remember Golden Rule #3asserttrial.response=="response"Furthermore, experiments save themselves after each iteration and status change, so we have alsoserialisedthe data
as well. Let's delete the experiment object and reload it.delee=expflow.Experiment.load("example_p1","example_e1")fortrialine.trials:# Golden Rule #3asserttrial.response=="response"The responses were recorded! This is important — it means that expflow automatically stores data across Python sessions
with no extra effort on the part of the user.PausingSuppose an experiment is too long to be completed all in one session: we need to pause it and resume it later. The
following code simulates a pause halfway through an experiment.p2=expflow.Participant("example_p2")trials=[expflow.Trial(trial_number=i)foriinrange(10)]e3=expflow.Experiment("example_p2","example_e1",trials=trials)fortrialine:ife.trial_index>5:show_stimulus(trial.stimulus)trial.response=get_response()else:e.pause()ifnote.is_running:breakasserte.is_pausedasserte.current_trial.is_pausedYou can resume a paused experiment by iterating over it again.fortrialine:asserte.trial_index>5show_stimulus(trial.stimulus)trial.response=get_response()ifnote.is_running:breakTiming outIndividual trials or entire experiments may have time limits. If so, you can use thetrial.time_outandexperiment.time_outmethods to time out a trial or experiment, respectively. Timed out trials and experiments cannot
be resumed (unlike paused trials).SkippingSometimes an experiment may skip upcoming trials, or an experiment may be skipped enitrely (if it is part of a batch of
experiments, for example). You can use thetrial.skipandexperiment.skipmethods to achieve this. You can't skip a
trial or experiment that was already started, nor can you ever
start a skipped trial or experiment.Experiments don't always present every trial to every participant. Sometimes an experiment may have a stopping rule,
where some or all remaining trials are skipped due to poor performance, or trial-specific or whole-experiment time
limits. The statuses"skipped"and"timed_out"— and their respecitve methods,skipandtimeout(ortime_out) — exist to deal with these circumstances. You can skip or time out individual trials or entire experiments. Skipped or timed out experiments/trials cannot be rerun afterwards under any circumstances.DurationExperiments and trials have adurationfield that is calculated when the experiment/trial is finished or timed out.
It represents the total time taken to complete the experiment/trial, minus any time spent in a paused state, in seconds.Crash recoveryExpflow has a rudimentary crash-recovery feature. It doesn't work all the time, but it could save your bacon once in a
while.Consider this example:p3=expflow.Participant("example_p3")trials=[expflow.Trial(trial_number=i)foriinrange(3)]e2=expflow.Experiment("example_p3","example_e1",trials=trials)fortrialine3:show_stimulus(trial.stimulus)trial.response=get_response()break# <- crash!dele3# <- crash!e3=expflow.Experiment.load("example_p3","example_e1")asserte3.is_pausedasserte3.current_trial.is_pausedHere, we have simulated a computer crash by breaking the trial loop and deleting our experiment object, which is
approximately what would happen if you encountered an unexpected exception during your experiment. On garbage
collection, an experiment object will change its status from running to
paused if necessary, and save itself. Therefore, when this experiment is loaded again, it behaves as if it were paused
at the point of garbage collection.