There are many reasons why fathers lose custody of their children. One reason is that the father may not have been involved in the child's life as much as the mother. Another reason is that the father may have a history of violence or substance abuse. The court will also consider the best interests of the child when making a custody determination.