Consider if they are found between vowels, at the beginning of words, at the end of words, before or after specific segments or classes of segments. At syllable boundaries. Determine if the sounds are in complementary distribution. If they are, the sounds are most likely allophones of the same phoneme. To the native speaker of the language, they are perceived as the same sound.
Different environments condition the sound so that it is pronounced a bit differently.
Make sure that you are noting the environments correctly. If you don't have minimal pairs and you cannot find complementary distribution, you are probably not inspecting the data closely enough. Have you looked at the specific sounds in the environment rather than just whether they are consonants or vowels. If the sounds are in free variation, you also can conclude that they are allophones of the same phoneme. Go back through the data and determine what is the distribution of the sounds that you believe may be in the relationship.