A lower average view duration may be observed if you produce a lot of YouTube Shorts, but since this is a characteristic of Shorts, it doesn't actually hurt your channel. In fact, YouTube Shorts can aid in drawing in and keeping viewers who might later subscribe to your channel.