It's not bad luck, or blox fruits hating you. It's the fact that you aren't looking hard enough. And even if you did, you still have to be quick when it comes to finding the blue flower in Second Sea. And you need to look HARD, not just skim through every island without double checking.