>There's a nice quote, "Being abstract is something profoundly different from being vague. The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise."

And under this definition, using AST level macros is vastly superior to text-transforming ones. Text-transforming macros don't allow the user to be precise. Or at least, to be precise, the user must know a number of rules that are arcane and not obvious. `#define min(X, Y) (X < Y ? X : Y)` feels like it should work, but it doesn't. Or well it does sometimes, but not always.

I'll come back to this in a second, but lets talk about sorting. You claim that if the user doesn't really care you can use a less sophisticated abstraction. But that's not at all true! In fact, its a violation of the Liskov Substitution Principle[0]. If I have an interface `Sort`, I shouldn't care whether it is of type Bubble, Tim, or Quick. But if I have a TimSort, I certainly may care if you swap it out with BubbleSort. In this case, the desirable property is speed. One way of putting this is that for any Sorts S and T, S can only be a Subtype of T if S is faster than T. This allows you to replace a T with an S, and you won't lose any desirable properties[1].

Another way to put this rule would be that when modifying an API, your changes shouldn't cost the user anything. Replacing Timsort with bubblesort costs the user speed. That's bad. It violates an assumption the user may have had.

And while you say that this doesn't make for a simpler conceptual end user experience, I disagree. If the system just is fast, you don't need to worry about performance. If `sort` is fast, you won't need to implement your own faster sorting function (or even have to try and figure out that that's what you need). Not having to write an efficient sorting algorithm certainly sounds simple to me!

Similarly, there is a cognitive cost to an API. And working with an API that is astonishing[2] has a cost. Its harder to reason about how it will interact with the rest of the system. An API that can guarantee that all macro results will be syntactically valid is simpler than one that requires that you manually do that bookkeeping. Same with a language that can guarantee that all memory accesses will be valid. There may be associated costs with these things (you can't string-concat syntactically invalid strs into valid ones, or you can't implement certain graph structures without jumping through hoops), so its not quite as clear cut as with sorts, but I think its pretty clear that AST-based macros are simper to interact with than Cs.

In fact, C's macros being so simple makes them more dangerous. Its easy to understand how they work internally, so a novice may feel like they won't be surprising (until...). But the simplicity of the implementation leads to footguns when they interact with other things. The abstraction C macros provide is conceptually simple but leaky, or to use your words, imprecise. You have to understand them more than you think to be able to interact with them safely.

AST based macros on the other hand aren't leaky. They're more difficult to conceptualize, but you don't really need to fully conceptualize, because they won't surprise you. You take in some expressions and modify them, and you'll get out what you expect.

Doing AST based transforms and substitutions instead of text-based ones significantly reduces the cognitive overhead. You stop having to worry about the edge cases where some transformation might happen in the wrong context, or not happen in the right one (as a simple example, applying substitutions via regex vs. via AST means that you no longer have to worry about where there were line breaks).

>I don't have anything against AST transformations. They are a good idea to implement if one can figure out relevant usecases and a usable API. But in most cases I guess personally I'm likely to prefer either that 1-line macro, or not to add tricky AST manipulation code to create heavy syntax magic for a one-off thing.

I'm a bit confused here. I'm not saying that the macros are harder to implement for the end user, in fact the opposite. But that they're more work for the language to implement. A text based macro like

DEFINE_handler(type, function_body) (void handle##(type)((type) input) { (function_body) };

isn't significantly easier to define than something like

DEFINE_HANDLER(type t, AST function_body) {
f = ast.Function()
f.name = "DEFINE" + t.name
f.body = function_body
return f
}

In fact, its arguably clearer what's going on in the second example.

[0]: https://en.wikipedia.org/wiki/Liskov_substitution_principle

[1]: Yes I realize there are other desirable properties of sorts, such as stability and inplaceness but I'm simplifying.

[2]: https://en.wikipedia.org/wiki/Principle_of_least_astonishmen...