A computer-implemented method for providing an active artificial intelligence agent that collects data automatically and simultaneously in an online manner, the method comprising: automatically obtaining, with a training framework, physical interaction data from a data collection module, wherein the physical interaction data comprises at least one of image-based data for images of the physical interaction data, images of an object in different positions, or images of an object being rotated, and wherein weak annotations are automatically generated from clustering physical interaction data first, and then refined during training without manual interaction, wherein weak annotations comprise bounding boxes or image-level labels; storing the physical interaction data in a plurality of databases (710,..., 720), wherein physical interaction data comprising images of a respective object is stored in a respective one of the plurality of databases (710,..., 720); utilizing the training framework having the physical interaction data to automatically train diverse sets of machine learning program units with physical interaction data from a respective one of the plurality of databases (710,..., 720) to simulate physical interactions between a user and an object, wherein the physical interactions comprise at least one of pushing, grasping, or rotating the object, and wherein each individual program unit has a different model based on the applied physical interaction data; training a master program unit through jointly approximating and modeling behaviors of the sets of individual program units; and applying physical interaction data into the master program unit to generate predicted physical interactions based on the training of the individual program units and the master program unit. automatically obtaining, with a training framework, physical interaction data from a data collection module, wherein the physical interaction data comprises at least one of image-based data for images of the physical interaction data, images of an object in different positions, or images of an object being rotated, and wherein weak annotations are automatically generated from clustering physical interaction data first, and then refined during training without manual interaction, wherein weak annotations comprise bounding boxes or image-level labels; storing the physical interaction data in a plurality of databases (710,..., 720), wherein physical interaction data comprising images of a respective object is stored in a respective one of the plurality of databases (710,..., 720); utilizing the training framework having the physical interaction data to automatically train diverse sets of machine learning program units with physical interaction data from a respective one of the plurality of databases (710,..., 720) to simulate physical interactions between a user and an object, wherein the physical interactions comprise at least one of pushing, grasping, or rotating the object, and wherein each individual program unit has a different model based on the applied physical interaction data; training a master program unit through jointly approximating and modeling behaviors of the sets of individual program units; and applying physical interaction data into the master program unit to generate predicted physical interactions based on the training of the individual program units and the master program unit.