Right, I wasn't thinking about the wrong labels correlating to anything.  If the labels can only be 'apples', 'oranges', or 'both apples and oranges', ( not labeled 'broccoli' for example ) and they are definitely wrong, then it should only take one or two tries.  Taking an apple out of the 'apple' box means that it is the both box, and that the other boxes are 'apples' -> 'oranges', 'oranges' -> 'apples'.  If you draw an orange out of the apple box, then you must draw out of the orange box.  If you get an orange, it is the both box, if you get an apple, then the first box is the both box.