Yellowstone season 1 even revealed that Jimmy's father and the rest of his family were either in prison or dead. As for Lloyd, his past is largely a mystery, but it's clear that he is not related to Jimmy in any way.