Williams started the first Baptist church in America , the First Baptist Church of Providence .
He is believed to have started the first or second Baptist church in the United States .