if supplied, server.config will be passed to server

That would have been true for custom servers even without this PR. The solution you've proposed is inconsistent. Perhaps brunch's message should be made more generic for any custom server. 

Additionally,  the callback should be the last arg. I'm not yet convinced there is a good reason for passing brunch's instance of logger at all, but if we will, probably best as part of the opts like you had it originally.