Regardless of the parents' marital status, grandparents can seek court-ordered time with their grandkids. If you're a paternal grandparent with an unmarried grandchild, your grandson must establish paternity before you can utilize your visitation rights.