Yes! While you might have seen pickled eggs hanging out on a dusty bar like me, those eggs were cooked and processed to be shelf stable. Not so with my recipe. Since they have a soft yolk, you should definitely keep them in the refrigerator, and don't keep them longer than about a week.