It could be possible if you’ve just stained the brick. But still, some chemical cleaner might be needed.