A computer-implemented method of modifying a video stream captured by a user device, comprising: receiving , by one or more processors, a video stream captured by a user device; identifying an object of interest in at least one frame of the video stream by determining pixels within the video stream corresponding to the object of interest; identifying thereafter a portion of the object of interest; determining a direction associated with the identified object by determining , as the direction, a direction of the identified portion for each frame of a set of previous frames of the video stream; combining the direction of the identified object for the set of previous frames to identify an aggregate direction of the identified object; and modifying the video stream to include a graphical element by positioning the graphical element within the at least one frame of the video stream based on the aggregate direction associated with the identified object; wherein the object of interest is a hand of the user and the portion of the object of interest is a finger of the user; and wherein the method comprises: dynamically increasing a number of frames in the set of previous frames upon determining that a distance of movement by the finger is sufficient to cause jitter, artifacts, or other unwanted signals, and/or dynamically decreasing the number of frames upon determining that the number of frames is greater than a minimum number so as to reduce resource consumption. receiving , by one or more processors, a video stream captured by a user device; identifying an object of interest in at least one frame of the video stream by determining pixels within the video stream corresponding to the object of interest; identifying thereafter a portion of the object of interest; determining a direction associated with the identified object by determining , as the direction, a direction of the identified portion for each frame of a set of previous frames of the video stream; combining the direction of the identified object for the set of previous frames to identify an aggregate direction of the identified object; and modifying the video stream to include a graphical element by positioning the graphical element within the at least one frame of the video stream based on the aggregate direction associated with the identified object; wherein the object of interest is a hand of the user and the portion of the object of interest is a finger of the user; and wherein the method comprises: dynamically increasing a number of frames in the set of previous frames upon determining that a distance of movement by the finger is sufficient to cause jitter, artifacts, or other unwanted signals, and/or dynamically decreasing the number of frames upon determining that the number of frames is greater than a minimum number so as to reduce resource consumption. dynamically increasing a number of frames in the set of previous frames upon determining that a distance of movement by the finger is sufficient to cause jitter, artifacts, or other unwanted signals, and/or dynamically decreasing the number of frames upon determining that the number of frames is greater than a minimum number so as to reduce resource consumption. <EOS>