Fix integer overflow in MallocFrameBuffer()

Promote integers to uint64_t to avoid integer overflow issue during frame buffer allocation for very large screen sizes