A method of performing testing across a plurality of smart devices, the method comprising: registering, by a testing device, the plurality of smart devices to be accessed for performing testing, wherein the plurality of smart devices are associated with a plurality of users; installing, by the testing device, a thin client on each of the plurality of smart devices in response to the registering, wherein the thin client enables communication of a smart device of the plurality of smart devices with the testing device, wherein the thin client periodically monitors the smart device to retrieve a plurality of parameters associated with the smart device that comprise location of the smart device, Operating System installed on the smart device, and storage space within the smart device; determining, by the testing device, at least one time-window for each of the plurality of smart devices, wherein the smart device is in an idle mode during the at least one time-window wherein the determining is based on one of: the time-window being defined by user or the usage pattern of the smart device determined by the thin client; gathering, by the testing device, testing criteria and a time duration for performing a testing operation, wherein the testing criteria includes specific requirements for one or more of the plurality of parameters retrieved for each of plurality of smart devices; dynamically creating, by the testing device, a test group comprising one or more smart devices from the plurality of smart devices, wherein each of the one or more smart devices in the test group satisfy the testing criteria, and wherein the at least one time-window of each smart device in the test group is within the time duration; installing, by the testing device, at least one application on each smart device within test group; executing a test script on each smart device within the test group in response to installing the at least one application; collecting results of executing the test script on each smart device in the test group; and performing a predefined analysis on the results collected from each smart device in the test group. registering, by a testing device, the plurality of smart devices to be accessed for performing testing, wherein the plurality of smart devices are associated with a plurality of users; installing, by the testing device, a thin client on each of the plurality of smart devices in response to the registering, wherein the thin client enables communication of a smart device of the plurality of smart devices with the testing device, wherein the thin client periodically monitors the smart device to retrieve a plurality of parameters associated with the smart device that comprise location of the smart device, Operating System installed on the smart device, and storage space within the smart device; determining, by the testing device, at least one time-window for each of the plurality of smart devices, wherein the smart device is in an idle mode during the at least one time-window wherein the determining is based on one of: the time-window being defined by user or the usage pattern of the smart device determined by the thin client; gathering, by the testing device, testing criteria and a time duration for performing a testing operation, wherein the testing criteria includes specific requirements for one or more of the plurality of parameters retrieved for each of plurality of smart devices; dynamically creating, by the testing device, a test group comprising one or more smart devices from the plurality of smart devices, wherein each of the one or more smart devices in the test group satisfy the testing criteria, and wherein the at least one time-window of each smart device in the test group is within the time duration; installing, by the testing device, at least one application on each smart device within test group; executing a test script on each smart device within the test group in response to installing the at least one application; collecting results of executing the test script on each smart device in the test group; and performing a predefined analysis on the results collected from each smart device in the test group.