In an attempt to carry out his orders from both Davis and Gen. Joseph E. Johnston, Pemberton and his Army of Mississippi set out east to combine with Johnston's forces gathering around Jackson, while remaining in contact and covering Vicksburg. Another order from Johnston changing their proposed meeting location caused Pemberton to turn around, and when he did he accidentally collided with Grant's army at the Battle of Champion Hill on May 16 and suffered a major defeat. Pemberton retreated to the Big Black River, where he fought and suffered even more heavy losses on May 17. Pemberton resolved to defend Vicksburg and led his defeated men back into its defenses on May 18. In the process, he gave up the high ground on Hayne's Bluff which Sherman had failed to take in December. Johnston had advised him that if this ground should ever fall, Vicksburg would be untenable, and that he should escape with his army of 31,000, sacrificing the city. Pemberton refused to take this advice. He held firm for over six weeks, while soldiers and civilians were starved into submission. (Pemberton, well aware of his reputation as a Northerner by birth, was probably influenced by his fear of public condemnation as a traitor if he abandoned Vicksburg.)