 "There Goes My Baby" is a, "slinky" down-tempo R & B ballad, with infused hints of neo soul. It is composed in a "moderate groove" of eighty beats per minute, and is set in common time. It is written the key of A minor and Usher's vocals span from the low note of E4 to high note of C6. It follows the chord progression A – Fmaj7 – G. The song makes use of Usher's falsetto range, which was warmly received by critics. Edna Gunderson of USA Today called Usher's falsetto "charming." Jaime Gill of Yahoo! Music UK said that the song was a delight and that its falsetto tenderness was absent in the rest of the work. Tyler Lewis of PopMatters said the song featured "nice falsetto work" from the singer, while Leah Greenblatt of Entertainment Weekly coined the song a "falsetto-laced plea."
