A method executing on a server device comprising: receiving a web page request from a client device; in response to the web page request, selecting a web page from a plurality of web pages, the web page describing an application and including an offer to continue a user experience of the application on the client device via an installation of an application configuration, wherein the web page is selected based on one or more criteria for selecting at least one web page for a control group or a treatment group; receiving, via the selected web page, an installation request to install the application configuration on the client device; in response to the installation request, transmitting an application installer and a resource identifier to the client device; receiving a configuration request indicating an execution of the application installer at the client device, wherein the configuration request comprises the resource identifier; in response to the configuration request, selecting a configuration file from a plurality of configuration files based on the resource identifier, and transmitting the selected configuration file to the client device, wherein installation parameters of the configuration file causes the application installer to install the application configuration on the client device, and wherein the application configuration is configured to return telemetry data with the resource identifier, or an experiment identifier, and wherein the configuration file further comprises one or more experiment parameters for controlling functions within the application configuration; receiving the telemetry data, the telemetry data including a measurement indicating a date of use or a duration of use of the application configuration, the telemetry data comprising at least one of the resource identifier, or the experiment identifier; and analyzing the telemetry data to generate a usage report comprising user retention data of the application configuration based on the telemetry data; and adjusting the one or more criteria based on the telemetry data to generate updated criteria for selecting one or more web pages for the control group or the treatment group in response to additional web page requests. receiving a web page request from a client device; in response to the web page request, selecting a web page from a plurality of web pages, the web page describing an application and including an offer to continue a user experience of the application on the client device via an installation of an application configuration, wherein the web page is selected based on one or more criteria for selecting at least one web page for a control group or a treatment group; receiving, via the selected web page, an installation request to install the application configuration on the client device; in response to the installation request, transmitting an application installer and a resource identifier to the client device; receiving a configuration request indicating an execution of the application installer at the client device, wherein the configuration request comprises the resource identifier; in response to the configuration request, selecting a configuration file from a plurality of configuration files based on the resource identifier, and transmitting the selected configuration file to the client device, wherein installation parameters of the configuration file causes the application installer to install the application configuration on the client device, and wherein the application configuration is configured to return telemetry data with the resource identifier, or an experiment identifier, and wherein the configuration file further comprises one or more experiment parameters for controlling functions within the application configuration; receiving the telemetry data, the telemetry data including a measurement indicating a date of use or a duration of use of the application configuration, the telemetry data comprising at least one of the resource identifier, or the experiment identifier; and analyzing the telemetry data to generate a usage report comprising user retention data of the application configuration based on the telemetry data; and adjusting the one or more criteria based on the telemetry data to generate updated criteria for selecting one or more web pages for the control group or the treatment group in response to additional web page requests.