Seems like there's a bug in Chrome where padding-box doesn't behave. Border box is sufficient for this case and solved the problem.