Macros in Elixir are a code-generation facility that happens to be Turing-complete, nothing more. You should use it whenever code generation makes sense. Code generation comes with a set of well-known problems, for example the error messages and stacktraces need special attention. If you feel that the advantages of code generation outweight the disadvantages in your specific situation, you should use it, no matter if you're designing a framework or writing a one-off script.

One of the advantages of code generation is hiding the details of implementation, so that they can be changed more easily (because the description of the whole is shorter, and kept in one place). While you get the same benefits in Object Oriented languages via other means (interfaces and abstract classes, Template Class pattern), in Elixir you don't have much choice: you either implement a callback module (ie. behaviour; but then there are cross-module calls at runtime) or generate the code via macros (but then you lose the 1-to-1 correspondence between what's in the file and what's actually defined in the module). Of course, both can be (and frequently are, like in GenServer, Application, Supervisor, etc.) used at the same time.

Not using macros until some point in time in the development (after it "reaches that level of maturity") risks getting that point in time wrong. It almost always will be wrong, actually, and more often than not you'll find that it's too late: introducing macros after first spending time on optimizing the functional interface makes you just do the same work for the second time, at which point the improvement in "developer ux" may not be worth it or even noticeable. Erlang doesn't have (well, it does, but not as easily accessed) macros, and the libraries still get written in it - frameworks too - and there's little point in adding macro-based wrappers (beyond some simple default implementation injections) to them, because their functional interfaces are already optimized from the dev UX angle. So, to make the most out of macros, you probably shouldn't wait until "after you're done" with something, but you should introduce them as soon as you see a good use for them, while considering all the potential downsides and pitfals that come with their use.

Also of note, while you can compare macros to eg. Python (or Ruby, or Smalltalk) Metaclasses, there is a difference: Python offers a lot of mechanisms that you should prefer using over metaclasses if they would suffice. On the other hand, there's not really much you can use in Elixir, as it's a lot smaller and simpler language. So, while I don't think many Python programmers should be writing metaclasses (though all should know what they are!), I think almost all Elixir programmers should be able to use macros. The macro system in Elixir, BTW, is actually on a simpler side of Turing-complete macro systems, both defmacro from Lisp, syntax-case from Scheme, and imperative macros in Nim (and many others) are all much more complex and harder to use. So I think the Elixir designers wanted people to use macros and made an effort to make it safer and simpler. That's just my impression though, don't quote me on this :)