No, you cannot. You write that you returned to work after the holiday; therefore, you were not working (or rather: were not available & able to work) on the holiday or (as you write) the two workdays prior. There is no right in the law to paid holidays: employers don't have to offer them at all. If they choose to offer them, they may put restrictions or limitations on who get holiday pay. Those restrictions may legally include that you just have worked the days right before the holiday and/or have been able/eligible to work the holiday itself, and so that if you were out of work for any reason, you do not get holiday pay.