Yes, you should always wash fingerling potatoes to remove any dirt or unwanted debris. You can just wash them with your hands, or if they are extra dirty, you can use a gentle brush.