It depends on what we mean by fair. If we mean that the person guilty of a crime is reliably identified and punished, perhaps it would be perfectly fair. But if we mean that the rules society has established in order to decide these things are scrupulously followed in deciding who is guilty, then, no it would not be fair. The tension between those positions runs throughout criminal law. Those alternatives, of course, are caricatures.
Reality lies somewhere in between.
This is not an attack on the Fourth Amendment. If you and I would object to being stopped or searched without probable cause, it should not happen to any one else either. Rather, it is a comment on the workings of the trial and appeal system. Most citizens are familiar with such things as the right to remain silent, to have counsel, and other parts of the Miranda warnings that police must give before they arrest a person. What they do not see is the much more pervasive gauntlet of traps and pitfalls that judges, lawyers, and juries must negotiate to avoid having a conviction overturned.