That residue, like a seed, will eventually produce an effect at some point in the future, and effect in the form of pleasure or pain for the person who performed the act. Karma affects events both within a single lifetime and in future lifetimes. If a being has karma when it dies, samsara continues and leads to another birth. Nonvirtuous actions lead to negative karma, which may cause a rebirth full of suffering and pain in one of the hells. Likewise, virtuous actions can lead to rebirth as a god.
Human lives are somewhere in between, with a mix of pleasure and suffering.
Beings are not the only things that undergo constant rebirths. The entire Buddhist universe had no beginning. The universe evolves for innumerable eons and then is gradually destroyed, only to be recreated after a long period of vacuity.