A system (100, 800) for annotating a video frame generated by a video game executed by a first device operated by a player, the system comprising: at least one processor circuit; at least one memory that stores program code configured to be executed by the at least one processor circuit, the program code comprising: a video game model generator that is configured to generate a video game model that associates element tags with in-game elements of the video game, wherein the in-game elements are located within the video game; a video game overlay that is executed concurrently with the video game and includes: a receiver that is configured to receive a remote user input from one or more remotely located devices over a network; a parser that is configured to apply the video game model to parse the remote user input and extract therefrom an extracted element tag corresponding to the video game; an element recognizer configured to apply the video game model to identify an in-game element of the video game that corresponds to the extracted element tag; and an annotation renderer configured to annotate the video frame by superimposing the annotation onto real-time gameplay of the video game, based on the identified in-game element and a location of the identified in-game element to enable real-time interaction between one or more remote users and the player. at least one processor circuit; at least one memory that stores program code configured to be executed by the at least one processor circuit, the program code comprising: a video game model generator that is configured to generate a video game model that associates element tags with in-game elements of the video game, wherein the in-game elements are located within the video game; a video game overlay that is executed concurrently with the video game and includes: a receiver that is configured to receive a remote user input from one or more remotely located devices over a network; a parser that is configured to apply the video game model to parse the remote user input and extract therefrom an extracted element tag corresponding to the video game; an element recognizer configured to apply the video game model to identify an in-game element of the video game that corresponds to the extracted element tag; and an annotation renderer configured to annotate the video frame by superimposing the annotation onto real-time gameplay of the video game, based on the identified in-game element and a location of the identified in-game element to enable real-time interaction between one or more remote users and the player. a video game model generator that is configured to generate a video game model that associates element tags with in-game elements of the video game, wherein the in-game elements are located within the video game; a video game overlay that is executed concurrently with the video game and includes: a receiver that is configured to receive a remote user input from one or more remotely located devices over a network; a parser that is configured to apply the video game model to parse the remote user input and extract therefrom an extracted element tag corresponding to the video game; an element recognizer configured to apply the video game model to identify an in-game element of the video game that corresponds to the extracted element tag; and an annotation renderer configured to annotate the video frame by superimposing the annotation onto real-time gameplay of the video game, based on the identified in-game element and a location of the identified in-game element to enable real-time interaction between one or more remote users and the player. a receiver that is configured to receive a remote user input from one or more remotely located devices over a network; a parser that is configured to apply the video game model to parse the remote user input and extract therefrom an extracted element tag corresponding to the video game; an element recognizer configured to apply the video game model to identify an in-game element of the video game that corresponds to the extracted element tag; and an annotation renderer configured to annotate the video frame by superimposing the annotation onto real-time gameplay of the video game, based on the identified in-game element and a location of the identified in-game element to enable real-time interaction between one or more remote users and the player.