Track buffered_bytes in select/epoll loops, not in selectables them selves