Rhythm makes all writing more musical, not only poetry. I don't write a great deal of rhyming poetry, but I do like reading it. My main consideration is that the rhyme not sound forced. A good rhyme seems like it was just the most appropriate word to use at that point, and would have been appropriate whether it rhymed or not. The most common mistakes of beginning rhyme-poets are to twist a sentence around so that the right word ends up at the end, to use an archaic word to fit the rhyme, or to throw in a really strained connection in order to make a rhyme.
Some people seem born with a knack for rhyme, an instinctive ear.
Most of the rest of us must work at it if we want to write natural-sounding verse. Play with the dictionary, learning new words and finding rhymes for them. Read out loud and listen for the sound of words. Listen to all of the sound patterns in poetry, not just echoes at the end of words. I have more about rhyme posted at Kalliope Poetry Exercise Workshop if you want to play with it.