Package: reviewer2
reviewer #2This tool starts a simple image server that lets you quickly flip through image files from a local directory using your web browser.
Also, it optionally shows a customizable form where you can take notes or answer questions about each image or set of images.Example uses:manual curation / review of sequencing data visualization images such as those generated byREViewerfor short tandem repeat locimachine learning training set creationreviewing a pile of photosFeatures:simple way to flip through many local image files using your web browsercrawls a top-level directory to find .png, .jpeg, or .svg image filesweb interface: home page lists all imagesweb interface: image pages show the image, an optional customizable form where you can take notes or answer questions about the image, next/previous page links, and optional other customizable info for contextuse subdirectories to group images. Any images found in the same subdirectory will be shown on the same pageInstall:python3 -m pip install reviewer2Run:python3 -m reviewer2    # start server for all images in the current directory and subdirectoriesBelow are more examples. Run with--helpto see all available options and their descriptions.python3 -m reviewer2 -x temp -x keyword2  /path/dir-with-images    #  -x are keyword(s) of paths to skip and /path/dir-with-images is the top level dir to search instead of the current dir

python3 -m reviewer2 -t /path/user_responses.xls    #  change where user responses get saved (default: reviewer2_form_responses.tsv)

python3 -m reviewer2 -m /path/metadata.tsv          #  provide a metadata tableAfter the server is running, open your web browser tohttp://localhost:8080to start reviewing images.Options:metadata table (-m)It's often useful to add extra info to the image pages to help with review - such as image descriptions, quality scores, etc.
To enable this, there are several ways to specify arbitrary key-value pairs to add to specific image pages.
The 1st way is to put a file calledreviewer2_metadata.jsonnext to the image(s). All keys and values from this file
will appear on that image page. The 2nd way is to use-mto pass in a metadata table
(.tsvor.xls) with aPathcolumn + arbitrary other columns. If thePathvalue matches the relative directory containing
the image(s), entries from that row will be added to this image page.Since the keys and values are treated as html, they can be used to add more complex info - such as
colors, text formatting, <img ..> tags with images from other web pages, iframes containing entire sections of external pages, etc.responses table (-t)As you fill in the forms at the top of the image pages, the responses are written to this table. If you later restart reviewer2 with the same-t, it will reload previous responses. You can also optionally use this table to provide additional columns to display - sometimes this can be more convenient than using-m.Default:reviewer2_form_responses.tsvcustom form schema (--form-schema-json)If you'd like to use non-default questions in the image page forms, you can specify the path or url of a .json file containing a custom form schema. For examples of the expected format seemain/form_schema_examplesconfig file (~/.reviewer2_config)Most settings that can be provided on the command line can also be set via this YAML config file instead. For example:~/.reviewer2_configform-schema-json: /path/to/my-schema.json
hide-metadata-on-home-page: true
host: 127.0.0.1
port: 8080For more details, run:python3 -m reviewer2 --helpComparing reviews:If 2 or more people review the same images, the reviews can be compared using thecompare_form_response_tablesscript.For example:python3 -m compare_form_response_tables egor_reviewer2_form_responses.tsv  ben_reviewer2_form_responses.tsv -o combined_responses.tsv -s1 egor -s2 benThis script will print concordance stats, and output acombined_responses.tsvtable that contains one image per row as well as each person's review of the image.To see the full list of args and descriptions, runpython3 -m compare_form_response_tables --helpDevelopment:To create a local dev instance, rungit clone git@github.com:broadinstitute/reviewer2.git
cd reviewer2

# start server in dev mode so it reloads code on change
python3 -m reviewer2 /path/dir-with-images --dev-mode