 Silver's final recordings with the Jazz Messengers were in May 1956. Later that year, he left Blakey after one and a half years, in part because of the heroin use prevalent in the band, which Silver did not want to be involved in. Soon after leaving, Silver formed his own long-term quintet, after receiving offers of work from club owners who had heard his albums. The first line-up was Mobley (tenor saxophone), Farmer (trumpet), Watkins (bass), and Louis Hayes (drums). The quintet, with various line-ups, continued to record, helping Silver to build his reputation. He wrote almost all of the material they played, and, in concert, he "won over the crowds through his affable personality and all-action approach. He crouched over the piano as the sweat poured out, with his forelock brushing the keys and his feet pounding."
