Patent Document ID: 9854029
Application ID: 14533023
Patent Flag: 1

Claim One:
1. A system comprising: a server in communication with a plurality of client devices, the server comprising a hardware processor, memory, and computer-executable instructions stored in the memory that, when executed by the hardware processor, cause the hardware processor to: establish communication sessions with the plurality of client devices, wherein at least a portion of the communication sessions comprise a session identifier; assign a control state to a first subset of the plurality of client devices and a treatment state to a second subset of the plurality of client devices, wherein at least one of the control state or the treatment state causes output of one or more features by a respective client device by executing native code on the respective client device, further wherein the native code is compiled to execute on a hardware processor of the respective client device; store assigned experiment states for the first subset and the second subset of the plurality of client devices in association with a respective session identifier; receive response data from one or more first client devices of the first subset and one or more second client devices of the second subset, wherein the response data comprises session identifiers associated with the one or more first client devices and the one or more second client devices and state identifiers indicative of the output of the control state or the treatment state by the one or more first client devices and the one or more second client devices; compare at least a subset of the state identifiers to at least a subset of the assigned experiment states to determine a result comprising: an allocation match; an allocation mismatch associated with assignment of the treatment state and response data associated with the control state; and an allocation mismatch associated with assignment of the control state and response data associated with the treatment state; generate statistical data, using the result, wherein the statistical data is indicative of allocation matches corresponding to the control state and allocation matches corresponding to the treatment state, wherein the statistical data is not indicative of the allocation mismatches associated with assignment of the treatment state and response data associated with the control state and the allocation mismatches associated with assignment of the control state and response data associated with the treatment state; and cause modification of an experiment parameter based at least in part on the allocation mismatches associated with assignment of the control state and response data associated with the treatment state.