I'm in a need to show implementation notes for APIs, so enabling Swashbuckle to use some xml descriptions in a way Swagger.NET does looks like the least ugly way. There's also a way to use custom attributes as in ServiceStack, yet it looks like an overkill here.
So what i ended up with is a substituted IDocumentationProvider used in a bit of a hacky way. I'm sorry about it, but the other options seemed even worse.