replace int by size_t when referencing buffer size in a few places

darktable has various issues when working with huge image sizes. one of the problems
is fixed now but others still remain.
(cherry picked from commit 949ae8ae6ba4704ea330bf34279a593eaf491fba)