BACKGROUND
multiplexed bead arrays allow researchers to perform immunoassays that test tens or even hundreds of analytes against each sample in each plate well  <cit> .
at present, many  <cit>   of these arrays leverage luminex® xmap® technology, so we refer to them here as “luminex” assays.
these include both commercially available assay kits and custom assays, such as the binding antibody multiplex assay  for human immunodeficiency virus  <dig>  developed by the tomaras laboratory at duke university  <cit> .
such multiplexed assays can speed experimental efforts, increase lab efficiency and consume smaller amounts of sample material than ordinary enzyme-linked immunosorbent assays   <cit> .
in an elisa, each analyte must be tested with a separate aliquot of sample in a separate well.
although luminex assays can offer experimental advantages, they can also pose challenges in analysis  <cit>  and quality control  <cit> , particularly given the greater complexity and dimensionality of such assays than ordinary elisas.
the continued evolution of analysis and quality control techniques, the limits of current tools and the increasingly important role these assays play in certain fields of biomedical research all make enhanced software tools desirable for management and analysis of luminex immunoassay data.
the field of vaccine immunology provides an example of the growing importance of luminex assays and the need for better software support.
in this field, samples from vaccine trial participants can be too scarce to assay against large numbers of analytes using ordinary elisa techniques  <cit> .
to maximize the insights gained from vaccine trials, researchers have started to rely on multiplexed luminex methods  <cit> .
recent follow-up studies for the alvac-aidsvax trial, the first vaccine trial ever to demonstrate some degree of vaccine efficacy against hiv- <dig>  <cit> , heavily used luminex assays.
several of these studies used luminex assays to examine the binding of plasma immunoglobulins to panels of hiv- <dig> envelope proteins to determine immune correlates of vaccine efficacy  <cit> .
while completing alvac-aidsvax follow-up studies, several collaborating teams found that the existing software for analysis and quality control of research luminex immunoassays did not meet their needs in two areas.
first, labs found it necessary to move data between multiple software tools to fully process all experimental data, employ advanced analytical techniques, and perform quality control across runs and reagent/bead lots.
this was labor-intensive, introduced additional opportunities for error, and multiplied versions of data and analyses.
second, existing luminex tools did not enable smooth handoff of quality-controlled data from labs to central network statisticians and principal investigators, nor provide transparency into data excluded from analysis by the labs as part of quality control.
the inherent complexity of multiplexed immunoassays makes robust, transparent quality control techniques particularly vital to achieving reproducibility, reliability and comparability of such assays  <cit> .
custom immunoassays that use luminex technology, such as bama, can be highly reproducible; for example, bama has been validated for use in analyzing hiv- <dig> specific antibody responses in clinical trials  <cit> , data in preparation for publication by georgia tomaras].
nevertheless, variability in assay execution, analysis, and results is currently considered a key inhibitor of the reliable use of commercially available multiplexed assays in clinical applications, such as diagnostic biomarkers or surrogate endpoints for clinical trials  <cit> .
one recent study of variability in multiplexed cytokine assay results showed significant variation across every participating lab and/or material lot for every reagent kit and multiplex platform employed   <cit> .
large-scale efforts to develop assay “harmonization” guidelines  <cit>  emphasize the importance of facilitating and sharing quality techniques that can be used to overcome intra- and inter-assay variability and stabilize assay performance across labs.
growing efforts to develop reporting standards for immunoassay protocols and results  <cit>  further emphasize the importance of clearly tracking and reporting variables  that affect assay consistency.
in response to the limitations of existing software tools, the hiv vaccine trials network  and the statistical center for hiv/aids research & prevention  at the fred hutchinson cancer research center collaborated with labkey software to enhance the freely available, open source labkey server system  <cit>  to support management, analysis and quality control of luminex immunoassay data.
data may be produced by either customized or commercially available  assays.
the software may be particularly helpful to research organizations who seek greater control over analysis and quality control techniques; to consolidate, simplify and standardize calculation and monitoring procedures; and  to facilitate secure collaboration.
steps for a luminex immunoassay
to provide context and terminology for software discussions, we briefly sketch an abbreviated set of experimental, analysis and quality control steps that might be used as part of an immunoassay based on luminex xmap technology.
note that experimental steps, materials and terminology vary with the type of immunoassay; similarly, analysis and quality control techniques continue to be active areas of research  <cit> .
figure  <dig> shows how you might evaluate the binding of plasma antibodies from study participants  antibodies) to a panel of analytes  proteins ).
experimental steps:
conjugate each analyte with a different type of bead.
each bead type contains a mixture of red and infrared dyes whose ratio can produce a spectral signature that uniquely identifies the bead type .
mix bead/analyte complexes with a sample in each plate well.
sufficient numbers of each type of bead must be deposited in each well to satisfy thresholds for statistically significant results.
optionally, unbound  beads may be included in the bead mixture to allow subtraction of fluorescence from non-specific binding to beads.
the type of sample deposited in each well varies depending on the purpose of the well:  a titrated standard, whose concentration is known and used to calculate a reference curve  a titrated quality control, whose concentration is known and used to calculate a reference curve  an unknown, such as serum from a study participant  a positive or negative control  a background well, which contains only buffer and is used for subtracting that results from non-specific binding of the detector.
samples are usually replicated in several wells.
incubate, then rinse.
this removes excess sample.
add detector conjugated to r-phycoerythrin.
the detector and its attached fluorochrome later serve as reporter for the binding of sample material to the bead/analyte complex.
the detector is sometimes called the conjugate for the experiment.
incubate, then rinse.
this removes excess detector.
use two lasers to measure bead number and sample binding.
typically, flow cytometry and two lasers are used to make single-bead measurements.
for each bead, one laser excites the bead’s dye, allowing identification of the bead type  from the red/infrared ratio of light, as well as detection of beads of incorrect size .
a second laser excites the detector-bound fluorochrome, allowing measurement of sample binding.
for each type of analyte in each well, the instrument reports the median fluorescence intensity  for the bound detector for all beads coupled with that analyte.
on a per-run basis, analysis and quality control of results may include: normalization to account for background fluorescence, exclusion of suspect data, calculation of dose–response curves for titrated standards, the use of these curves to interpolate estimated concentrations of unknowns, and other steps.
performing quality control across many assay runs may involve: calculating representative metrics for each standard curve for each run, plotting these metrics and expected ranges on levey-jennings  <cit>  charts to determine outliers, excluding data from analysis based on quality metrics, and identifying trends that require further investigation.
implementation
architecture
labkey server is a web application implemented in java.
it runs on the apache tomcat web server and stores its data in a relational database, either postgresql or microsoft sql server.
labkey server has been tested on computers running microsoft windows and most unix variants, including linux, macintosh osx and solaris.
labkey server’s tool for luminex is packaged in a java-based module that encapsulates user interface elements and calculation logic for designing, processing and displaying structured assays of various kinds.
the tool can leverage a customizable transform script that performs luminex-specific analysis calculations during import of assay data.
if included, this script is also re-run automatically when data from a titration are excluded from analysis because the exclusion affects the analysis results.
the default luminex transform script is written in r, but many other programming languages could be used .
details of labkey server’s architecture and assay module are covered elsewhere  <cit> .
labkey server v <dig> , available in december  <dig>  is the 25th official, public release of the platform since  <dig>
configuration of scripting
to access all features of the labkey server tool for luminex, the r scripting environment must be installed and configured on the server.
to leverage the default luminex transform script , one must also install the script, its utility script , the ruminex r package  <cit> , and required r packages .
see additional file  <dig> to obtain the first three; the other r packages are available from the comprehensive r archive network   <cit> .
RESULTS
overview of the lumimex tool
when a researcher imports luminex run data into a labkey server, the researcher is prompted to enter relevant metadata about the batch of runs, the run itself, included analytes and relevant titrations.
metadata include information required for calculations  and experimental characteristics that can be used for tracking purposes .
the system can be configured to use an r script to automatically analyze raw data and metadata to determine dose–response curves for titrations, estimate concentrations using these curves, identify outliers, determine quality control metrics for curves and plot curves.
unreliable experimental data can be flagged for exclusion from analyses.
when a transform script is configured, the system automatically provides levey-jennings quality-control charts and allows users to define expected ranges using baseline sets of runs.
assay results can be integrated with other data types, such as specimen information and clinical data for study participants.
the luminex tool displays all source data, quality control exclusions, calculated values and curves in a secure, interactive, web-based interface.
options for leveraging the tool
labkey server’s tool for luminex can be accessed in two ways:
• by installing and configuring a labkey server instance.
this option allows you to administer and customize a private labkey server for your lab or organization.
the “availability and requirements” section of this paper explains how to obtain installers and/or source code.
• by logging into to the atlas science portal  <cit> .
this option is available to qualified researchers at many of the consortia participating in the global hiv enterprise  <cit> .
to inquire about access, contact atlas@scharp.org.
documentation, tutorials, live demo, and context-sensitive help
full documentation and tutorials for setting up, configuring and using labkey server and its luminex assay tool are available at http://www.labkey.org.
this documentation is updated regularly to match the currently released version of labkey server.
the basic and advanced luminex tutorials  <cit>  provide detailed walk-throughs of luminex setup and scenarios.
they also include a troubleshooting guide  <cit>  that helps users interpret and fix error messages from luminex transform scripts, plus address problems with script-calculated values, such as curve fits.
the tutorials are accompanied by live demos  <cit>  that allow visitors to interact with luminex features that do not require editor-level permissions.
for each page in the luminex workflow, the software provides a context-sensitive help link in the upper right corner that leads to a relevant tutorial or documentation page.
assay file formats
labkey server understands microsoft excel® files output by bio-plex manager™  <cit>  for each luminex assay plate run.
bio-plex manager is software that runs several types of luminex devices.
the labkey server tool for luminex has been tested with the results of 96-well plate assays.
it is expected to be compatible with newer, larger plates, such as those with  <dig> wells.
each run file includes a separate worksheet for each type of bead .
each sheet displays a header with run metadata, one or two data tables, and a footer with calculated parameters and flags.
data tables report results provided by the instrument, either on a per-well basis  or on a per-sample basis .
if summary data are reported, each row supplies an average for all sample replicates.
some files contain both raw and summary data tables.
each row in a data table reports the sample type , sample identifier, well location , sample description , fi , fi minus mean background well fi, coefficient of variance, expected concentration , dilution, bead count , and other columns.
usage scenario for the tool
to analyze and perform quality control on luminex data, a lab might follow these steps:
<dig>  set up an assay project.
after setting up a labkey server, an administrator creates an assay-type project on the server with appropriate permissions for user access.
it provides a central location for managing descriptions of luminex assays and associated run data.
<dig>  create an assay design.
an assay design  <cit>  both describes the content expected for a particular type of assay run file and provides a framework for importing many runs and their associated metadata in a standardized manner.
an administrator can either select the default luminex assay design, or import a luminex assay design archive that is tailored to support the default transform script.
both designs can be further customized, as shown in figure  <dig>  when usage of a transform script is desired, the path to this script is entered in the assay design.
this screenshot shows a portion of an assay design and illustrates how customization of such a design can facilitate collection of run-specific data from the user during data import.
here, the isotype run metadata field  has been defined as a lookup to a pre-defined list .
this list, also called isotype, populates a drop-down list of data entry options that are presented to the user during run import .
the isotype field has been further customized to default initially to iga , then after that to the last entered value.
<dig>  import data and enter metadata.
after an administrator has created an assay design in the assay project, users can import run data to the server.
as part of this process, the assay design guides the collection of appropriate metadata for the assay run, as shown in figure  <dig>  certain metadata are used to control luminex-specific data processing.
for example, information collected may determine processing steps for inputs to calculations , identify the standard used for calculating interpolated concentrations, or associate analytes with metadata .
other metadata are used to annotate the run, run analytes, and/or batch of runs.
during import, the transform script automatically performs calculations on the data and the system flags certain data for quality control .
several files may be imported together as part of one run, allowing standards in one file to be associated with results in another.
<dig>  explore results.
curve fits and related results are calculated automatically by the transform script for each run during the import process.
results include curve fit parameters, estimated concentrations interpolated from calculated curve fits for standard titrations, and metrics calculated from curve fits, including ec <dig> , auc  and highmfi .
calculated results and data from run files are displayed together in grid views.
the system provides curve visualizations based on both 4- and 5-parameter logistic regressions , as shown in figure  <dig>  users can explore their data through sorting, filtering and customizing grid views; using built-in visualization and analysis tools; or building custom sql queries or visualizations  <cit> .
for example, labkey server’s built-in r scripting environment allows statisticians to define r scripts that produce visualizations and associate them with certain kinds of results.
<dig>  perform within-run quality control.
questionable results can be excluded from analysis on a per-analyte or per-replicate-group basis.
if a replicate group of wells for a titration is excluded, the transform script is re-run to recalculate curves and associated results.
figure  <dig> shows how the system applies red highlighting to all excluded data.
<dig>  perform cross-run quality control.
the system automatically plots levey-jennings charts for four metrics of performance  for both standards and quality controls chosen for each analyte.
users can select a suite of baseline runs  to establish expected ranges for analyte-specific standard metrics for all runs associated with the same lot of experimental materials, as shown in figure  <dig>  these ranges  are displayed on the levey-jennings plot for each metric for the analyte standard, as shown in figure  <dig>  for each run associated with the same lot of materials, quality control metrics that fall outside of expected ranges are flagged for review, as shown in figure  <dig>  standard curves for selected runs can be displayed together to help identify inconsistencies in curves across runs, as shown in figure  <dig>
this example shows differences in expected ec <dig> ranges for two groups of runs, each of which is associated with a different lot of experimental materials.
the example also shows a trend over time in ec <dig>  which might suggest decay in experimental conditions or materials.
such decay would require further investigation.
it might require exclusion of results from certain days or material lots, replacement of old reagents, investigation of material supply consistency, or other such measures.
early identification of such a trend can allow a lab to take corrective measures before a quality metric falls outside of acceptable bounds and assay runs must be rejected.
outliers visible in this plot are flagged for review, as shown in figure  <dig>  charts can be exported to pdf using the pdf icon.
<dig>  share quality-controlled data with collaborators.
after a lab has completed quality control for a set of runs, a lab data manager copies finalized data into a shared, study-type project.
a study-type project serves as an integration and analysis hub for many types of related data for a study   <cit> .
collaborators with appropriate permissions can use the server’s web portal to view and analyze this data.
administrators can set up custom views or sql queries that allow different parties  to see the data in their own preferred formats.
role-based, inheritable and customizable permission settings  <cit>  support controlled, secure sharing within a lab, between labs, between organizations, and/or with the general public.
collaborating scientists can use the shared data portal to bring different perspectives to results, explore alternative analysis approaches, and collaborate efficiently on publication.
<dig>  publicly share data and analyses as part of a publication.
upon publication of results in a scientific journal, specified subsets of anonymized, releasable data can be publicly shared on a labkey server web portal alongside contextual information and tools, including quality control information and analytical scripts.
providing public, interactive access to publication-related data allows a wider pool of scientists to explore the results and investigate alternative analytical approaches.
interactive release of data, quality control information, and analytical tools also supports study reproducibility  <cit> .
calculations
calculations performed on luminex data fall into three categories: automatic calculations performed by the server, automatic calculations performed by the transform script, and optional calculations performed by the script.
calculations by the script only occur if the script is associated with the assay design.
calculations by the system and the script are performed during data import.
additionally, if data for a replicate group of wells in a titration are excluded, the transform script and its calculations are re-run.
automatic calculations and flagging performed by the server.
labkey server provides automatic calculation of coefficients of variation  and provides flagging for outliers of various kinds to facilitate human review.
for quality control of multiplexed immunoassays, researchers typically pre-select acceptable thresholds of variation across replicates and use these thresholds to flag, review, and exclude dubious data in a standardized fashion.
cvs are used to measure variability for different kinds of replicates  and identify questionable outliers or patterns that may be artifacts of experimental conditions  <cit> .
outliers can bias top asymptotes, bottom asymptotes, and slope parameters  <cit> .
cvs for both intra- and inter-assay measures may also be monitored over time to identify changes in experimental conditions that may affect the accuracy and comparability of results  <cit> .
when the imported run file does not include summary data, labkey server calculates the cv of the fis of each replicate group .
cvs are stored with each well-level row and measure variability across replicates.
they are displayed as percents .
labkey server adds a quality control flag to each data row whose reported fi is greater than  <dig> and whose %cv is greater than  <dig>  or  <dig> .
fi >  <dig> is used as a threshold to save review time; for our collaborators, only samples that are potentially “positive”  warrant further investigation.
columns flagged through this process are highlighted in red.
labkey server can automatically provide additional flagging and reporting of outliers.
assay design fields can be configured with arbitrary conditional formatting settings that automatically color code values when viewed in the user interface or in excel file exports.
a user can set up saved, custom views that automatically filter results to show only data rows that exceed user-set thresholds for user-selected measurements.
furthermore, as mentioned above, the system’s analyte- and replicate-specific exclusion options can be used to systematically add custom flags and exclusions, which the server displays in summary reports.
automatic calculations performed by the default transform script.
the script automatically applies a conversion function to normalized fis to account for negative values potentially introduced by the removal of blank bead and/or background fluorescence.
currently, the script calculates: converted fi = max +  <dig>
when calculating 5pl curve fits, the default transform script uses the ruminex r package  <cit> , which in turn uses the drc r package  <cit> .
when calculating 4pl curve fits, the script uses the drc r package directly.
users can edit the transform script to use alternative strategies for performing 5pl and 4pl curve fits  <cit> , including using custom seed and optimization functions.
users can also add new calculations to the transform script.
optional calculations performed by the default transform script.
the default transform script enables users to opt to:
• normalize fi by subtracting blank bead and/or background well fi before calculating curves or interpolating unknowns.
this subtracts fluorescence signal that is due to non-specific binding to the beads.
• log transform normalized fis before calculation of curve fits, which can help stabilize variance extremes  <cit> .
• apply weights to the squared errors used in goodness of fit optimization calculations.
weighting can help account for lower variance at lower fi .
this improves curve fits, particularly at the low range, thus expanding the useful range of data.
using the transform script, these weights can be arbitrary
• calculate "positivity" by determining whether the value for each analyte has increased relative to a baseline value and also exceeds an analyte-specific threshold.
this is helpful, for example, when you wish to determine whether measurements for an individual in a trial have increased relative to baseline, pre-trial values.
both the baseline and threshold for each analyte are entered during run import.
additional system features and supported technologies
additional features of labkey server are likely to be particularly helpful to users of the luminex tool.
these include: a fine-grained security model; auditing; options for exporting data in a variety of formats; file upload and sharing; and client apis  for building custom interfaces and workflows using a variety of common languages   <cit> .
labkey server also allows researchers to integrate assay results  with related study data  <cit> .
specimen identifiers in run data files can be mapped automatically to study-specific specimen identifiers .
this allows linkage and integration of specimen, experimental and clinical data.
the system also supports assays and techniques commonly used by immunologists, including elisas, neutralizing antibody assays  <cit> , flow cytometry  <cit> , proteomics  <cit> , and next-generation sequencing  <cit> .
furthermore, labkey server also supports the description, management and analysis of novel assay types  <cit> .
current usage
the full set of features described here only became available in april  <dig> with the release of labkey server  <dig> , so adoption of the improved luminex assay tool has been promising.
as of january  <dig>  the atlas science portal maintained by scharp, hvtn and their collaborators contained a total of  <dig>  luminex runs contributed by at least three separate labs.
discussion
overview
among software for luminex analysis and quality control, labkey server stands out most distinctively in its support for customization, traceable quality control, and secure data sharing.
these capabilities can help labs develop and adopt the new methods; simplify and standardize workflows; and securely, smoothly share information with collaborators.
labkey server is also notably unique among luminex-specific tools for being both open source and freely available .
current, notable limitations include a lack of support for running luminex instruments and a requirement that input run files conform to the excel file format output by bio-plex instrument software.
alternatives
beyond labkey server, only a few software tools provide luminex-specific support, and all of those tools provide only limited customizability.
a much wider range of software tools provide significant customizability alongside general-purpose statistical features, but those tools do not inherently support luminex-specific workflows.
table  <dig> compares the capabilities of labkey server with six representative tools: three systems tailored for luminex  and three general-purpose statistical tools .
to highlight tradeoffs, features are roughly ordered from greatest luminex specificity to greatest scope for customization.
this table compares labkey server with a representative sample of software used for analysis and quality control of research luminex immunoassays.
this list aims to show how labkey server provides advantages over other existing software tools, not to compare all capabilities relevant to luminex analysis.
documentation for many tools was incomplete, so the table provides only a reasonable inference of feature availability.
this table does not aim to address tools for scientific data sharing  <cit> .
furthermore, the table only includes luminex-specific software designed for research assays, not software designed to support standardized diagnostic tests.
research assays typically require greater scope for cutting-edge analysis than manufacturer-defined diagnostic tests.
advantages
labkey server’s tool for management and analysis of luminex immunoassay results can help labs and research organizations to:
efficiently perform complex analyses of high-content data.
the nature of multiplexed immunoassays results in a significant amount of information per experimental run.
the tool can help teams efficiently manage and quickly analyze the complex, multidimensional data produced by such assays.
facilitate networked research through smooth, secure sharing of luminex results.
a central, shared data portal/repository can prove particularly useful to research networks and consortia who employ central data management and statistics groups to execute study-wide and/or cross-study analyses based on data pooled from collaborating labs  <cit> .
as shown in figure  <dig>  labs can use such a portal to supply statisticians with data and quality control information in a secure fashion.
central data managers can use the system’s reporting tools to efficiently monitor and troubleshoot study progress.
principal investigators  can gain transparency into study progress, analyses and data, as well as explore alternative analysis approaches and collaborate on publications.
using a central repository can also help prevent loss of valuable data from individual computers.
after study publication, a subset of releasable, anonymized study results can be made publicly available on the organization’s data portal by changing permission settings.
retain raw luminex data, quality controlled data, and analyses to make them available for later re-analysis.
labkey server stores raw run data from the instrument, quality control information, calculated values, final datasets, visualizations, and the original run data files.
data records that have been flagged for quality control can be excluded from analysis without deletion.
when data is stored in a shared repository, statisticians, labs and principal investigators can examine or export copies of the data as it looked at different stages of processing .
consolidate, simplify and standardize lab workflows by using one primary software tool for luminex analyses.
eliminating the labor-intensive steps necessary to move data between tools for different calculations can increase efficiency, foster standardization and reduce opportunities for error.
if the software is deployed on a shared portal, users have access to the same version of the tool and the same analysis algorithms without the need to install updates on a multiplicity of machines  <cit> .
customize calculations and visualizations.
labs can adopt new analysis and quality control techniques without adopting additional software tools or complicating workflows.
they can use scripting and custom assay designs to incorporate the latest algorithms , visualizations  and processing steps  while still providing users with a friendly, graphical user interface.
labs can test out new analytical techniques and transform scripts without disrupting mainline workflows by establishing separate assay designs for exploratory analytics.
track quality control across runs and reagent/bead lots using levey-jennings plots and outlier flagging.
both labkey server and existing luminex-specific software packages enable both automatic and manual identification of outliers within runs.
however, only labkey server provides extensive support for quality control tracking across runs and material lots using levey-jennings plots.
gain flexibility in plate layouts and titration analyses.
labkey server allows a user to associate multiple standards with each analyte, automatically perform curve fitting for all titrations , and select a standard on a different plate for use in interpolation of concentrations of unknowns.
bio-plex manager does not currently allow any of these options without rerunning analyses.
other software supports only some of these options.
leverage labkey server platform features.
the base system provides extensive support for data analysis, integration, visualization, reporting and workflow interface development.
it also supports other assays and techniques commonly used by immunologists.
use open-source, freely available tools.
at present, labkey server is the only open source tool with a graphical user interface that provides substantial support for luminex-specific scenarios.
the r scripting environment is also open source, as are a wide variety of r statistical analysis packages.
using open source tools can provide advantages in transparency, cost and extensibility.
facilitate assay harmonization across organizations.
overcoming variability and reproducibility issues for commercially available, multiplexed immunoassays for cytokine measurement is considered key to broadening the viable applications of such assays  <cit> .
to harmonize assay results, researchers identify experimental and analytical variables that strongly influence assay performance and use them to develop quality control guidelines that improve assay reproducibility across labs, material lots, organizations, etc.
identification of such performance variables requires sharing not just final results, but also protocol records, quality control information, data processing methodologies, and  analytical tools across collaborating organizations.
to our knowledge, labkey server has not yet been employed for assay harmonization at a scale worthy of note; nevertheless, the system seems particularly well-suited given its support for sharing just this kind of information and tools.
an installation of labkey server  has already been used for wide-scale proficiency testing  of a neutralizing antibody assay  <cit> .
limitations
current limitations on the use of labkey server for analysis of luminex experiments include:
no support for running a luminex instrument or designing luminex plate layouts.
software such as luminex xponent, bio-plex manager or milliplex analyst™  <cit>  is currently required for running instruments that leverage luminex xmap technology.
limited luminex data file format compatibility.
currently, the labkey server tool for luminex only understands data files in the excel format output by the bio-plex manager software.
however, the system can still infer the structure  of other tables of data, which allows users to quickly import other kinds of tabular data files, such as metadata files.
custom sql queries can join imported tables of metadata to other kinds of luminex data for display as custom data grids  <cit> .
familiarity with r recommended for extensive customization of analyses.
customization of certain aspects of data processing and analysis  can be made through the user interface.
however, making extensive changes to the default transform script  requires some knowledge of the r programming language.
at the same time, transform scripts can be written in most programming languages , not just r.
certain interfaces reflect terminology specific to certain kinds of immunoassays.
the levey-jennings charting interface uses terminology consistent with the kind of immunoassays shown in figure  <dig>  for such experiments, analytes are antigens and the antibodies being measured are immunoglobulin isotypes.
the interface can still be useful for other types of experiments as long as this terminology is understood.
future enhancements
labkey server undergoes ongoing development, culminating in 3– <dig> releases per year, so features and user interface refinements are regularly added to the platform.
features that may interest luminex users have already been added during the period this paper has been under review.
these improvements are available in labkey server v <dig>   and include: multi-curve titration plots for overlaying and comparing results from several runs , the ability to run arbitrary numbers of transform scripts sequentially during data import, enhanced heuristics for identifying filter options for columns in data grids, and tools for anonymizing data for public release.
one ongoing focus of development is support for ancillary studies  <cit> , particularly “freezer studies” that leverage valuable, stored specimens to gain novel insights through new experiments, such as luminex immunoassays.
CONCLUSIONS
labkey server’s support for luminex data analysis and quality control allows labs to consolidate, customize and standardize data analysis workflows, which can eliminate labor-intensive and error-prone steps.
using labkey server as a data repository and portal can enable teams to securely share data, analyses, quality control information, and related documents.
these capabilities can be particularly helpful to research organizations whose success depends on smooth collaboration between labs, central data managers, statistical teams and principal investigators.
the open source license  <cit>  for labkey server and its luminex tool allow other researchers to freely leverage, customize and improve this software.
availability and requirements
labkey server source code and compiled binaries
the labkey server open source software is freely available for download at http://www.labkey.org under the terms of the apache license  <dig>   <cit> .
this site also provides documentation, tutorials and demos for users and developers, along with instructions for developers who wish to contribute code to the project through the labkey server subversion repository.
compiled binaries for windows, unix, linux or macintosh installation are available for free through labkey software at http://www.labkey.com.
a graphical installer is available for computers running windows xp or later.
it includes the labkey server web application; the apache tomcat web server, v <dig> .35; the sun java runtime environment, v <dig> .0-29; the postgresql database server, v <dig> .1; and additional third-party components.
note that the installer does not include r. to access all features of the labkey server tool for luminex, the r scripting environment must be installed and configured on the server.
• project name: labkey server
• project home page:http://www.labkey.org
• operating system: platform independent
• programming languages: java, javascript, r, perl, python, sas, etc.
• other requirements, as of labkey server v <dig> : apache tomcat  <dig> x; sun java runtime environment  <dig> or 7; and either postgresql  or microsoft sql server .
check the project site for latest requirements of the most recent release.
• license: apache license  <dig>   <cit> .
access to the atlas science portal
access to atlas  <cit>  is available to qualified researchers at many of the consortia participating in the global hiv enterprise  <cit> .
to inquire about access, contact atlas@scharp.org.
abbreviations
4pl:  <dig> parameter logistic regression; 5pl:  <dig> parameter logistic regression; %cv: coefficient of variation expressed as a percent; aids: acquired immune deficiency syndrome; auc: area under the curve; bama: binding antibody multiplex assay; ec50: effective concentration at 50% of the difference between the asymptotes of a dose–response curve; elisa: enzyme-linked immunosorbent assay; highmfi: highest mean fluorescence intensity; hiv-1: human immunodeficiency virus type 1; hvtn: the hiv vaccine trials network; fi: median fluorescence intensity; id: identifier; iga: immunoglobulin a; scharp: the statistical center for hiv/aids research & prevention at the fred hutchinson cancer research center; qc: quality control.
competing interests
je, cn, ekn, evn, and bp are employees of labkey software, a software consulting company that provides development, customization, and support for labkey server.
labkey server is open source and freely available, so these authors do not have a direct financial interest in the software itself.
authors’ contributions
all authors reviewed and approved this manuscript.
je and cn designed and implemented the luminex assay tool.
je also provided project management.
ekn conceptualized, researched and wrote the paper.
ekn also contributed tutorials, demos, documentation, and testing.
sgs gathered requirements and provided project management and feature design.
evn contributed automated and manual testing.
nly and vca provided usage scenarios, workflow expertise, and assistance in prioritizing laboratory requirements.
ljh contributed expertise on analysis and data quality, as well as testing.
mb provided data coordination and information on workflows.
yf wrote the ruminex r package.
gdt saw the need for the tool and directed the lab work.
bp contributed project management.
je, cn, ekn, sgs, nly, ljh, gdt and bp contributed paper edits.
supplementary material
additional file 1
zip archive containing the transform script, ruminex package, assay design archive, and supporting files.
this .zip archive can be unzipped to obtain the following files:
• labkey transform script for luminex: labkey_luminex_transform_script.r.
• utility script used by the transform script: youtil.r.
• ruminex r package, v <dig> .9: ruminex_ <dig> .
<dig> zip.
• luminex assay design archive tailored for the transform script: luminex assay  <dig> xar.
• labkey server list archive useful for populating fields in the assay design: luminex_listarchive.lists.zip.
these files have been tested against labkey server v <dig> , released in december  <dig>  for future releases of labkey server, we recommend obtaining updated versions of these files.
the most current versions are provided as part of the advanced tutorial for luminex  <cit> .
labkey server tutorials and documentation available at http://www.labkey.org are updated regularly to match the currently released version of labkey server.
click here for file
acknowledgements
we are indebted to thomas donn of scharp for his assistance in testing, daryl morris of scharp for help with ruminex curve fits, and kim carson of brighteyeweb for improving figures  <dig> and  <dig>  we thank dr. marcella sarzotti-kelsoe of duke university for quality assurance oversight and collaboration on validating the binding antibody multiplex assays .
we also thank matthew bellew, adam rauch, mark igra, peter hussey, ren lis, karl lum, brian connolly, kevin krouse, trey chadick, steve hanson, nick arnold, kim carson, kristin fitzsimmons, alan vezina, ben bimber, dave bradlee, dax hawkins, rylan bauermeister, ben hackett, and avital sadot for their ongoing work on the labkey server platform.
this work was supported by grants from the bill and melinda gates foundation to the collaboration for aids vaccine discovery , grants from the national institute of health  to the hiv vaccine trials network  , and the duke university center for aids research  grant .

 multiplexed bead arrays allow researchers to perform immunoassays that test tens or even hundreds of analytes against each sample in each plate well. at present, many of these arrays leverage luminex® xmap® technology, so we refer to them here as “luminex” assays. these include both commercially available assay kits and custom assays, such as the binding antibody multiplex assay for human immunodeficiency virus developed by the tomaras laboratory at duke university. such multiplexed assays can speed experimental efforts, increase lab efficiency and consume smaller amounts of sample material than ordinary enzyme-linked immunosorbent assays. in an elisa, each analyte must be tested with a separate aliquot of sample in a separate well. although luminex assays can offer experimental advantages, they can also pose challenges in analysis and quality control, particularly given the greater complexity and dimensionality of such assays than ordinary elisas. the continued evolution of analysis and quality control techniques, the limits of current tools and the increasingly important role these assays play in certain fields of biomedical research all make enhanced software tools desirable for management and analysis of luminex immunoassay data. the field of vaccine immunology provides an example of the growing importance of luminex assays and the need for better software support. in this field, samples from vaccine trial participants can be too scarce to assay against large numbers of analytes using ordinary elisa techniques. to maximize the insights gained from vaccine trials, researchers have started to rely on multiplexed luminex methods. recent follow-up studies for the alvac-aidsvax trial, the first vaccine trial ever to demonstrate some degree of vaccine efficacy against hiv-, heavily used luminex assays. several of these studies used luminex assays to examine the binding of plasma immunoglobulins to panels of hiv- envelope proteins to determine immune correlates of vaccine efficacy. while completing alvac-aidsvax follow-up studies, several collaborating teams found that the existing software for analysis and quality control of research luminex immunoassays did not meet their needs in two areas. first, labs found it necessary to move data between multiple software tools to fully process all experimental data, employ advanced analytical techniques, and perform quality control across runs and reagent/bead lots. this was labor-intensive, introduced additional opportunities for error, and multiplied versions of data and analyses. second, existing luminex tools did not enable smooth handoff of quality-controlled data from labs to central network statisticians and principal investigators, nor provide transparency into data excluded from analysis by the labs as part of quality control. the inherent complexity of multiplexed immunoassays makes robust, transparent quality control techniques particularly vital to achieving reproducibility, reliability and comparability of such assays. custom immunoassays that use luminex technology, such as bama, can be highly reproducible; for example, bama has been validated for use in analyzing hiv- specific antibody responses in clinical trials, data in preparation for publication by georgia tomaras]. nevertheless, variability in assay execution, analysis, and results is currently considered a key inhibitor of the reliable use of commercially available multiplexed assays in clinical applications, such as diagnostic biomarkers or surrogate endpoints for clinical trials. one recent study of variability in multiplexed cytokine assay results showed significant variation across every participating lab and/or material lot for every reagent kit and multiplex platform employed. large-scale efforts to develop assay “harmonization” guidelines emphasize the importance of facilitating and sharing quality techniques that can be used to overcome intra- and inter-assay variability and stabilize assay performance across labs. growing efforts to develop reporting standards for immunoassay protocols and results further emphasize the importance of clearly tracking and reporting variables that affect assay consistency. in response to the limitations of existing software tools, the hiv vaccine trials network and the statistical center for hiv/aids research & prevention at the fred hutchinson cancer research center collaborated with labkey software to enhance the freely available, open source labkey server system to support management, analysis and quality control of luminex immunoassay data. data may be produced by either customized or commercially available assays. the software may be particularly helpful to research organizations who seek greater control over analysis and quality control techniques; to consolidate, simplify and standardize calculation and monitoring procedures; and to facilitate secure collaboration. steps for a luminex immunoassay to provide context and terminology for software discussions, we briefly sketch an abbreviated set of experimental, analysis and quality control steps that might be used as part of an immunoassay based on luminex xmap technology. note that experimental steps, materials and terminology vary with the type of immunoassay; similarly, analysis and quality control techniques continue to be active areas of research. figure  shows how you might evaluate the binding of plasma antibodies from study participants antibodies) to a panel of analytes proteins ). experimental steps: conjugate each analyte with a different type of bead. each bead type contains a mixture of red and infrared dyes whose ratio can produce a spectral signature that uniquely identifies the bead type. mix bead/analyte complexes with a sample in each plate well. sufficient numbers of each type of bead must be deposited in each well to satisfy thresholds for statistically significant results. optionally, unbound beads may be included in the bead mixture to allow subtraction of fluorescence from non-specific binding to beads. the type of sample deposited in each well varies depending on the purpose of the well: a titrated standard, whose concentration is known and used to calculate a reference curve a titrated quality control, whose concentration is known and used to calculate a reference curve an unknown, such as serum from a study participant a positive or negative control a background well, which contains only buffer and is used for subtracting that results from non-specific binding of the detector. samples are usually replicated in several wells. incubate, then rinse. this removes excess sample. add detector conjugated to r-phycoerythrin. the detector and its attached fluorochrome later serve as reporter for the binding of sample material to the bead/analyte complex. the detector is sometimes called the conjugate for the experiment. incubate, then rinse. this removes excess detector. use two lasers to measure bead number and sample binding. typically, flow cytometry and two lasers are used to make single-bead measurements. for each bead, one laser excites the bead’s dye, allowing identification of the bead type from the red/infrared ratio of light, as well as detection of beads of incorrect size. a second laser excites the detector-bound fluorochrome, allowing measurement of sample binding. for each type of analyte in each well, the instrument reports the median fluorescence intensity for the bound detector for all beads coupled with that analyte. on a per-run basis, analysis and quality control of results may include: normalization to account for background fluorescence, exclusion of suspect data, calculation of dose–response curves for titrated standards, the use of these curves to interpolate estimated concentrations of unknowns, and other steps. performing quality control across many assay runs may involve: calculating representative metrics for each standard curve for each run, plotting these metrics and expected ranges on levey-jennings charts to determine outliers, excluding data from analysis based on quality metrics, and identifying trends that require further investigation. implementation architecture labkey server is a web application implemented in java. it runs on the apache tomcat web server and stores its data in a relational database, either postgresql or microsoft sql server. labkey server has been tested on computers running microsoft windows and most unix variants, including linux, macintosh osx and solaris. labkey server’s tool for luminex is packaged in a java-based module that encapsulates user interface elements and calculation logic for designing, processing and displaying structured assays of various kinds. the tool can leverage a customizable transform script that performs luminex-specific analysis calculations during import of assay data. if included, this script is also re-run automatically when data from a titration are excluded from analysis because the exclusion affects the analysis results. the default luminex transform script is written in r, but many other programming languages could be used. details of labkey server’s architecture and assay module are covered elsewhere. labkey server v, available in december is the 25th official, public release of the platform since configuration of scripting to access all features of the labkey server tool for luminex, the r scripting environment must be installed and configured on the server. to leverage the default luminex transform script, one must also install the script, its utility script, the ruminex r package, and required r packages. see additional file to obtain the first three; the other r packages are available from the comprehensive r archive network.  overview of the lumimex tool when a researcher imports luminex run data into a labkey server, the researcher is prompted to enter relevant metadata about the batch of runs, the run itself, included analytes and relevant titrations. metadata include information required for calculations and experimental characteristics that can be used for tracking purposes. the system can be configured to use an r script to automatically analyze raw data and metadata to determine dose–response curves for titrations, estimate concentrations using these curves, identify outliers, determine quality control metrics for curves and plot curves. unreliable experimental data can be flagged for exclusion from analyses. when a transform script is configured, the system automatically provides levey-jennings quality-control charts and allows users to define expected ranges using baseline sets of runs. assay results can be integrated with other data types, such as specimen information and clinical data for study participants. the luminex tool displays all source data, quality control exclusions, calculated values and curves in a secure, interactive, web-based interface. options for leveraging the tool labkey server’s tool for luminex can be accessed in two ways: • by installing and configuring a labkey server instance. this option allows you to administer and customize a private labkey server for your lab or organization. the “availability and requirements” section of this paper explains how to obtain installers and/or source code. • by logging into to the atlas science portal. this option is available to qualified researchers at many of the consortia participating in the global hiv enterprise. to inquire about access, contact atlas@scharp.org. documentation, tutorials, live demo, and context-sensitive help full documentation and tutorials for setting up, configuring and using labkey server and its luminex assay tool are available at http://www.labkey.org. this documentation is updated regularly to match the currently released version of labkey server. the basic and advanced luminex tutorials provide detailed walk-throughs of luminex setup and scenarios. they also include a troubleshooting guide that helps users interpret and fix error messages from luminex transform scripts, plus address problems with script-calculated values, such as curve fits. the tutorials are accompanied by live demos that allow visitors to interact with luminex features that do not require editor-level permissions. for each page in the luminex workflow, the software provides a context-sensitive help link in the upper right corner that leads to a relevant tutorial or documentation page. assay file formats labkey server understands microsoft excel® files output by bio-plex manager™ for each luminex assay plate run. bio-plex manager is software that runs several types of luminex devices. the labkey server tool for luminex has been tested with the results of 96-well plate assays. it is expected to be compatible with newer, larger plates, such as those with wells. each run file includes a separate worksheet for each type of bead. each sheet displays a header with run metadata, one or two data tables, and a footer with calculated parameters and flags. data tables report results provided by the instrument, either on a per-well basis or on a per-sample basis. if summary data are reported, each row supplies an average for all sample replicates. some files contain both raw and summary data tables. each row in a data table reports the sample type, sample identifier, well location, sample description, fi, fi minus mean background well fi, coefficient of variance, expected concentration, dilution, bead count, and other columns. usage scenario for the tool to analyze and perform quality control on luminex data, a lab might follow these steps: set up an assay project. after setting up a labkey server, an administrator creates an assay-type project on the server with appropriate permissions for user access. it provides a central location for managing descriptions of luminex assays and associated run data. create an assay design. an assay design both describes the content expected for a particular type of assay run file and provides a framework for importing many runs and their associated metadata in a standardized manner. an administrator can either select the default luminex assay design, or import a luminex assay design archive that is tailored to support the default transform script. both designs can be further customized, as shown in figure  when usage of a transform script is desired, the path to this script is entered in the assay design. this screenshot shows a portion of an assay design and illustrates how customization of such a design can facilitate collection of run-specific data from the user during data import. here, the isotype run metadata field has been defined as a lookup to a pre-defined list. this list, also called isotype, populates a drop-down list of data entry options that are presented to the user during run import. the isotype field has been further customized to default initially to iga, then after that to the last entered value. import data and enter metadata. after an administrator has created an assay design in the assay project, users can import run data to the server. as part of this process, the assay design guides the collection of appropriate metadata for the assay run, as shown in figure  certain metadata are used to control luminex-specific data processing. for example, information collected may determine processing steps for inputs to calculations, identify the standard used for calculating interpolated concentrations, or associate analytes with metadata. other metadata are used to annotate the run, run analytes, and/or batch of runs. during import, the transform script automatically performs calculations on the data and the system flags certain data for quality control. several files may be imported together as part of one run, allowing standards in one file to be associated with results in another. explore results. curve fits and related results are calculated automatically by the transform script for each run during the import process. results include curve fit parameters, estimated concentrations interpolated from calculated curve fits for standard titrations, and metrics calculated from curve fits, including ec, auc and highmfi. calculated results and data from run files are displayed together in grid views. the system provides curve visualizations based on both 4- and 5-parameter logistic regressions, as shown in figure  users can explore their data through sorting, filtering and customizing grid views; using built-in visualization and analysis tools; or building custom sql queries or visualizations. for example, labkey server’s built-in r scripting environment allows statisticians to define r scripts that produce visualizations and associate them with certain kinds of results. perform within-run quality control. questionable results can be excluded from analysis on a per-analyte or per-replicate-group basis. if a replicate group of wells for a titration is excluded, the transform script is re-run to recalculate curves and associated results. figure  shows how the system applies red highlighting to all excluded data. perform cross-run quality control. the system automatically plots levey-jennings charts for four metrics of performance for both standards and quality controls chosen for each analyte. users can select a suite of baseline runs to establish expected ranges for analyte-specific standard metrics for all runs associated with the same lot of experimental materials, as shown in figure  these ranges are displayed on the levey-jennings plot for each metric for the analyte standard, as shown in figure  for each run associated with the same lot of materials, quality control metrics that fall outside of expected ranges are flagged for review, as shown in figure  standard curves for selected runs can be displayed together to help identify inconsistencies in curves across runs, as shown in figure  this example shows differences in expected ec ranges for two groups of runs, each of which is associated with a different lot of experimental materials. the example also shows a trend over time in ec which might suggest decay in experimental conditions or materials. such decay would require further investigation. it might require exclusion of results from certain days or material lots, replacement of old reagents, investigation of material supply consistency, or other such measures. early identification of such a trend can allow a lab to take corrective measures before a quality metric falls outside of acceptable bounds and assay runs must be rejected. outliers visible in this plot are flagged for review, as shown in figure  charts can be exported to pdf using the pdf icon. share quality-controlled data with collaborators. after a lab has completed quality control for a set of runs, a lab data manager copies finalized data into a shared, study-type project. a study-type project serves as an integration and analysis hub for many types of related data for a study. collaborators with appropriate permissions can use the server’s web portal to view and analyze this data. administrators can set up custom views or sql queries that allow different parties to see the data in their own preferred formats. role-based, inheritable and customizable permission settings support controlled, secure sharing within a lab, between labs, between organizations, and/or with the general public. collaborating scientists can use the shared data portal to bring different perspectives to results, explore alternative analysis approaches, and collaborate efficiently on publication. publicly share data and analyses as part of a publication. upon publication of results in a scientific journal, specified subsets of anonymized, releasable data can be publicly shared on a labkey server web portal alongside contextual information and tools, including quality control information and analytical scripts. providing public, interactive access to publication-related data allows a wider pool of scientists to explore the results and investigate alternative analytical approaches. interactive release of data, quality control information, and analytical tools also supports study reproducibility. calculations calculations performed on luminex data fall into three categories: automatic calculations performed by the server, automatic calculations performed by the transform script, and optional calculations performed by the script. calculations by the script only occur if the script is associated with the assay design. calculations by the system and the script are performed during data import. additionally, if data for a replicate group of wells in a titration are excluded, the transform script and its calculations are re-run. automatic calculations and flagging performed by the server. labkey server provides automatic calculation of coefficients of variation and provides flagging for outliers of various kinds to facilitate human review. for quality control of multiplexed immunoassays, researchers typically pre-select acceptable thresholds of variation across replicates and use these thresholds to flag, review, and exclude dubious data in a standardized fashion. cvs are used to measure variability for different kinds of replicates and identify questionable outliers or patterns that may be artifacts of experimental conditions. outliers can bias top asymptotes, bottom asymptotes, and slope parameters. cvs for both intra- and inter-assay measures may also be monitored over time to identify changes in experimental conditions that may affect the accuracy and comparability of results. when the imported run file does not include summary data, labkey server calculates the cv of the fis of each replicate group. cvs are stored with each well-level row and measure variability across replicates. they are displayed as percents. labkey server adds a quality control flag to each data row whose reported fi is greater than and whose %cv is greater than or. fi > is used as a threshold to save review time; for our collaborators, only samples that are potentially “positive” warrant further investigation. columns flagged through this process are highlighted in red. labkey server can automatically provide additional flagging and reporting of outliers. assay design fields can be configured with arbitrary conditional formatting settings that automatically color code values when viewed in the user interface or in excel file exports. a user can set up saved, custom views that automatically filter results to show only data rows that exceed user-set thresholds for user-selected measurements. furthermore, as mentioned above, the system’s analyte- and replicate-specific exclusion options can be used to systematically add custom flags and exclusions, which the server displays in summary reports. automatic calculations performed by the default transform script. the script automatically applies a conversion function to normalized fis to account for negative values potentially introduced by the removal of blank bead and/or background fluorescence. currently, the script calculates: converted fi = max + when calculating 5pl curve fits, the default transform script uses the ruminex r package, which in turn uses the drc r package. when calculating 4pl curve fits, the script uses the drc r package directly. users can edit the transform script to use alternative strategies for performing 5pl and 4pl curve fits, including using custom seed and optimization functions. users can also add new calculations to the transform script. optional calculations performed by the default transform script. the default transform script enables users to opt to: • normalize fi by subtracting blank bead and/or background well fi before calculating curves or interpolating unknowns. this subtracts fluorescence signal that is due to non-specific binding to the beads. • log transform normalized fis before calculation of curve fits, which can help stabilize variance extremes. • apply weights to the squared errors used in goodness of fit optimization calculations. weighting can help account for lower variance at lower fi. this improves curve fits, particularly at the low range, thus expanding the useful range of data. using the transform script, these weights can be arbitrary • calculate "positivity" by determining whether the value for each analyte has increased relative to a baseline value and also exceeds an analyte-specific threshold. this is helpful, for example, when you wish to determine whether measurements for an individual in a trial have increased relative to baseline, pre-trial values. both the baseline and threshold for each analyte are entered during run import. additional system features and supported technologies additional features of labkey server are likely to be particularly helpful to users of the luminex tool. these include: a fine-grained security model; auditing; options for exporting data in a variety of formats; file upload and sharing; and client apis for building custom interfaces and workflows using a variety of common languages. labkey server also allows researchers to integrate assay results with related study data. specimen identifiers in run data files can be mapped automatically to study-specific specimen identifiers. this allows linkage and integration of specimen, experimental and clinical data. the system also supports assays and techniques commonly used by immunologists, including elisas, neutralizing antibody assays, flow cytometry, proteomics, and next-generation sequencing. furthermore, labkey server also supports the description, management and analysis of novel assay types. current usage the full set of features described here only became available in april with the release of labkey server, so adoption of the improved luminex assay tool has been promising. as of january the atlas science portal maintained by scharp, hvtn and their collaborators contained a total of luminex runs contributed by at least three separate labs. discussion overview among software for luminex analysis and quality control, labkey server stands out most distinctively in its support for customization, traceable quality control, and secure data sharing. these capabilities can help labs develop and adopt the new methods; simplify and standardize workflows; and securely, smoothly share information with collaborators. labkey server is also notably unique among luminex-specific tools for being both open source and freely available. current, notable limitations include a lack of support for running luminex instruments and a requirement that input run files conform to the excel file format output by bio-plex instrument software. alternatives beyond labkey server, only a few software tools provide luminex-specific support, and all of those tools provide only limited customizability. a much wider range of software tools provide significant customizability alongside general-purpose statistical features, but those tools do not inherently support luminex-specific workflows. table  compares the capabilities of labkey server with six representative tools: three systems tailored for luminex and three general-purpose statistical tools. to highlight tradeoffs, features are roughly ordered from greatest luminex specificity to greatest scope for customization. this table compares labkey server with a representative sample of software used for analysis and quality control of research luminex immunoassays. this list aims to show how labkey server provides advantages over other existing software tools, not to compare all capabilities relevant to luminex analysis. documentation for many tools was incomplete, so the table provides only a reasonable inference of feature availability. this table does not aim to address tools for scientific data sharing. furthermore, the table only includes luminex-specific software designed for research assays, not software designed to support standardized diagnostic tests. research assays typically require greater scope for cutting-edge analysis than manufacturer-defined diagnostic tests. advantages labkey server’s tool for management and analysis of luminex immunoassay results can help labs and research organizations to: efficiently perform complex analyses of high-content data. the nature of multiplexed immunoassays results in a significant amount of information per experimental run. the tool can help teams efficiently manage and quickly analyze the complex, multidimensional data produced by such assays. facilitate networked research through smooth, secure sharing of luminex results. a central, shared data portal/repository can prove particularly useful to research networks and consortia who employ central data management and statistics groups to execute study-wide and/or cross-study analyses based on data pooled from collaborating labs. as shown in figure  labs can use such a portal to supply statisticians with data and quality control information in a secure fashion. central data managers can use the system’s reporting tools to efficiently monitor and troubleshoot study progress. principal investigators can gain transparency into study progress, analyses and data, as well as explore alternative analysis approaches and collaborate on publications. using a central repository can also help prevent loss of valuable data from individual computers. after study publication, a subset of releasable, anonymized study results can be made publicly available on the organization’s data portal by changing permission settings. retain raw luminex data, quality controlled data, and analyses to make them available for later re-analysis. labkey server stores raw run data from the instrument, quality control information, calculated values, final datasets, visualizations, and the original run data files. data records that have been flagged for quality control can be excluded from analysis without deletion. when data is stored in a shared repository, statisticians, labs and principal investigators can examine or export copies of the data as it looked at different stages of processing. consolidate, simplify and standardize lab workflows by using one primary software tool for luminex analyses. eliminating the labor-intensive steps necessary to move data between tools for different calculations can increase efficiency, foster standardization and reduce opportunities for error. if the software is deployed on a shared portal, users have access to the same version of the tool and the same analysis algorithms without the need to install updates on a multiplicity of machines. customize calculations and visualizations. labs can adopt new analysis and quality control techniques without adopting additional software tools or complicating workflows. they can use scripting and custom assay designs to incorporate the latest algorithms, visualizations and processing steps while still providing users with a friendly, graphical user interface. labs can test out new analytical techniques and transform scripts without disrupting mainline workflows by establishing separate assay designs for exploratory analytics. track quality control across runs and reagent/bead lots using levey-jennings plots and outlier flagging. both labkey server and existing luminex-specific software packages enable both automatic and manual identification of outliers within runs. however, only labkey server provides extensive support for quality control tracking across runs and material lots using levey-jennings plots. gain flexibility in plate layouts and titration analyses. labkey server allows a user to associate multiple standards with each analyte, automatically perform curve fitting for all titrations, and select a standard on a different plate for use in interpolation of concentrations of unknowns. bio-plex manager does not currently allow any of these options without rerunning analyses. other software supports only some of these options. leverage labkey server platform features. the base system provides extensive support for data analysis, integration, visualization, reporting and workflow interface development. it also supports other assays and techniques commonly used by immunologists. use open-source, freely available tools. at present, labkey server is the only open source tool with a graphical user interface that provides substantial support for luminex-specific scenarios. the r scripting environment is also open source, as are a wide variety of r statistical analysis packages. using open source tools can provide advantages in transparency, cost and extensibility. facilitate assay harmonization across organizations. overcoming variability and reproducibility issues for commercially available, multiplexed immunoassays for cytokine measurement is considered key to broadening the viable applications of such assays. to harmonize assay results, researchers identify experimental and analytical variables that strongly influence assay performance and use them to develop quality control guidelines that improve assay reproducibility across labs, material lots, organizations, etc. identification of such performance variables requires sharing not just final results, but also protocol records, quality control information, data processing methodologies, and analytical tools across collaborating organizations. to our knowledge, labkey server has not yet been employed for assay harmonization at a scale worthy of note; nevertheless, the system seems particularly well-suited given its support for sharing just this kind of information and tools. an installation of labkey server has already been used for wide-scale proficiency testing of a neutralizing antibody assay. limitations current limitations on the use of labkey server for analysis of luminex experiments include: no support for running a luminex instrument or designing luminex plate layouts. software such as luminex xponent, bio-plex manager or milliplex analyst™ is currently required for running instruments that leverage luminex xmap technology. limited luminex data file format compatibility. currently, the labkey server tool for luminex only understands data files in the excel format output by the bio-plex manager software. however, the system can still infer the structure of other tables of data, which allows users to quickly import other kinds of tabular data files, such as metadata files. custom sql queries can join imported tables of metadata to other kinds of luminex data for display as custom data grids. familiarity with r recommended for extensive customization of analyses. customization of certain aspects of data processing and analysis can be made through the user interface. however, making extensive changes to the default transform script requires some knowledge of the r programming language. at the same time, transform scripts can be written in most programming languages, not just r. certain interfaces reflect terminology specific to certain kinds of immunoassays. the levey-jennings charting interface uses terminology consistent with the kind of immunoassays shown in figure  for such experiments, analytes are antigens and the antibodies being measured are immunoglobulin isotypes. the interface can still be useful for other types of experiments as long as this terminology is understood. future enhancements labkey server undergoes ongoing development, culminating in 3– releases per year, so features and user interface refinements are regularly added to the platform. features that may interest luminex users have already been added during the period this paper has been under review. these improvements are available in labkey server v and include: multi-curve titration plots for overlaying and comparing results from several runs, the ability to run arbitrary numbers of transform scripts sequentially during data import, enhanced heuristics for identifying filter options for columns in data grids, and tools for anonymizing data for public release. one ongoing focus of development is support for ancillary studies, particularly “freezer studies” that leverage valuable, stored specimens to gain novel insights through new experiments, such as luminex immunoassays.  labkey server’s support for luminex data analysis and quality control allows labs to consolidate, customize and standardize data analysis workflows, which can eliminate labor-intensive and error-prone steps. using labkey server as a data repository and portal can enable teams to securely share data, analyses, quality control information, and related documents. these capabilities can be particularly helpful to research organizations whose success depends on smooth collaboration between labs, central data managers, statistical teams and principal investigators. the open source license for labkey server and its luminex tool allow other researchers to freely leverage, customize and improve this software. availability and requirements labkey server source code and compiled binaries the labkey server open source software is freely available for download at http://www.labkey.org under the terms of the apache license. this site also provides documentation, tutorials and demos for users and developers, along with instructions for developers who wish to contribute code to the project through the labkey server subversion repository. compiled binaries for windows, unix, linux or macintosh installation are available for free through labkey software at http://www.labkey.com. a graphical installer is available for computers running windows xp or later. it includes the labkey server web application; the apache tomcat web server, v.35; the sun java runtime environment, v.0-29; the postgresql database server, v.1; and additional third-party components. note that the installer does not include r. to access all features of the labkey server tool for luminex, the r scripting environment must be installed and configured on the server. • project name: labkey server • project home page:http://www.labkey.org • operating system: platform independent • programming languages: java, javascript, r, perl, python, sas, etc. • other requirements, as of labkey server v: apache tomcat x; sun java runtime environment or 7; and either postgresql or microsoft sql server. check the project site for latest requirements of the most recent release. • license: apache license. access to the atlas science portal access to atlas is available to qualified researchers at many of the consortia participating in the global hiv enterprise. to inquire about access, contact atlas@scharp.org. abbreviations 4pl: parameter logistic regression; 5pl: parameter logistic regression; %cv: coefficient of variation expressed as a percent; aids: acquired immune deficiency syndrome; auc: area under the curve; bama: binding antibody multiplex assay; ec50: effective concentration at 50% of the difference between the asymptotes of a dose–response curve; elisa: enzyme-linked immunosorbent assay; highmfi: highest mean fluorescence intensity; hiv-1: human immunodeficiency virus type 1; hvtn: the hiv vaccine trials network; fi: median fluorescence intensity; id: identifier; iga: immunoglobulin a; scharp: the statistical center for hiv/aids research & prevention at the fred hutchinson cancer research center; qc: quality control. competing interests je, cn, ekn, evn, and bp are employees of labkey software, a software consulting company that provides development, customization, and support for labkey server. labkey server is open source and freely available, so these authors do not have a direct financial interest in the software itself. authors’ contributions all authors reviewed and approved this manuscript. je and cn designed and implemented the luminex assay tool. je also provided project management. ekn conceptualized, researched and wrote the paper. ekn also contributed tutorials, demos, documentation, and testing. sgs gathered requirements and provided project management and feature design. evn contributed automated and manual testing. nly and vca provided usage scenarios, workflow expertise, and assistance in prioritizing laboratory requirements. ljh contributed expertise on analysis and data quality, as well as testing. mb provided data coordination and information on workflows. yf wrote the ruminex r package. gdt saw the need for the tool and directed the lab work. bp contributed project management. je, cn, ekn, sgs, nly, ljh, gdt and bp contributed paper edits. supplementary material additional file 1 zip archive containing the transform script, ruminex package, assay design archive, and supporting files. this.zip archive can be unzipped to obtain the following files: • labkey transform script for luminex: labkey_luminex_transform_script.r. • utility script used by the transform script: youtil.r. • ruminex r package, v.9: ruminex_. zip. • luminex assay design archive tailored for the transform script: luminex assay xar. • labkey server list archive useful for populating fields in the assay design: luminex_listarchive.lists.zip. these files have been tested against labkey server v, released in december for future releases of labkey server, we recommend obtaining updated versions of these files. the most current versions are provided as part of the advanced tutorial for luminex. labkey server tutorials and documentation available at http://www.labkey.org are updated regularly to match the currently released version of labkey server. click here for file acknowledgements we are indebted to thomas donn of scharp for his assistance in testing, daryl morris of scharp for help with ruminex curve fits, and kim carson of brighteyeweb for improving figures  and we thank dr. marcella sarzotti-kelsoe of duke university for quality assurance oversight and collaboration on validating the binding antibody multiplex assays. we also thank matthew bellew, adam rauch, mark igra, peter hussey, ren lis, karl lum, brian connolly, kevin krouse, trey chadick, steve hanson, nick arnold, kim carson, kristin fitzsimmons, alan vezina, ben bimber, dave bradlee, dax hawkins, rylan bauermeister, ben hackett, and avital sadot for their ongoing work on the labkey server platform. this work was supported by grants from the bill and melinda gates foundation to the collaboration for aids vaccine discovery, grants from the national institute of health to the hiv vaccine trials network, and the duke university center for aids research grant.
the hiv vaccine trials network and the statistical center for hiv/aids research and prevention collaborated with labkey software to enhance the open source labkey server platform to facilitate workflows for multiplexed bead assays. unlike other tools tailored for luminex immunoassays, labkey server allows labs to customize their luminex analyses using scripting while still presenting users with a single, graphical interface for processing and analyzing data. the labkey server system also stands out among luminex tools for enabling smooth, secure transfer of data, quality control information, and analyses between collaborators. labkey server and its luminex features are freely available as open source software at http://www.labkey.com under the apache license. immunoassays that employ multiplexed bead arrays produce high information content per sample. such assays are now frequently used to evaluate humoral responses in clinical trials. integrated software is needed for the analysis, quality control, and secure sharing of the high volume of data produced by such multiplexed assays. this system now supports the management, analysis, quality control, and secure sharing of data from multiplexed immunoassays that leverage luminex xmap® technology. newly added features enable labs to: import run data from spreadsheets output by bio-plex manager™ software; customize data processing, curve fits, and algorithms through scripts written in common languages, such as r; select script-defined calculation options through a graphical user interface; collect custom metadata for each titration, analyte, run and batch of runs; calculate dose–response curves for titrations; interpolate unknown concentrations from curves for titrated standards; flag run data for exclusion from analysis; track quality control metrics across runs using levey-jennings plots; and automatically flag outliers based on expected values.
 immunoassays that employ multiplexed bead arrays produce high information content per sample. such assays are now frequently used to evaluate humoral responses in clinical trials. integrated software is needed for the analysis, quality control, and secure sharing of the high volume of data produced by such multiplexed assays. software that facilitates data exchange and provides flexibility to perform customized analyses could increase scientists’ capacity to use these immunoassays to evaluate human clinical trials.  the hiv vaccine trials network and the statistical center for hiv/aids research and prevention collaborated with labkey software to enhance the open source labkey server platform to facilitate workflows for multiplexed bead assays. this system now supports the management, analysis, quality control, and secure sharing of data from multiplexed immunoassays that leverage luminex xmap® technology. these assays may be custom or kit-based. newly added features enable labs to: import run data from spreadsheets output by bio-plex manager™ software; customize data processing, curve fits, and algorithms through scripts written in common languages, such as r; select script-defined calculation options through a graphical user interface; collect custom metadata for each titration, analyte, run and batch of runs; calculate dose–response curves for titrations; interpolate unknown concentrations from curves for titrated standards; flag run data for exclusion from analysis; track quality control metrics across runs using levey-jennings plots; and automatically flag outliers based on expected values. existing system features allow researchers to analyze, integrate, visualize, export and securely share their data, as well as to construct custom user interfaces and workflows.  unlike other tools tailored for luminex immunoassays, labkey server allows labs to customize their luminex analyses using scripting while still presenting users with a single, graphical interface for processing and analyzing data. the labkey server system also stands out among luminex tools for enabling smooth, secure transfer of data, quality control information, and analyses between collaborators. labkey server and its luminex features are freely available as open source software at http://www.labkey.com under the apache license. 