You have heard their reproach, Yahweh, and all their devices against me,
"I am their song"  (Lamentations 3:63).  This line forces the conclusion that the enemies of these last verses were Jeremiah's own people, the Jews themselves, and not the Babylonians.  The Babylonian conquerors were friendly to Jeremiah (See Vol. II of my commentary on the major prophets, Jeremiah, pp 437-440).  The people who were singing taunt songs against the prophet were his own people.