I have my own take on this: https://github.com/jrockway/opinionated-server

Some things we do differently, and some things we do the same. We give each RPC handler a logger, so you know where logs are coming from (and the trace ID associated with every message). We collect detailed prometheus metrics for every RPC, and send everything we know to Jaeger. We provide clients for outgoing RPCs that log and trace in the same way as incoming requests.

We pick concrete types for logging/tracing/metrics, which makes it easier to figure out what code is actually running. Having used the abstraction layers, I find that when they don't do what you want, it can take a long time to figure out why. If you have to switch from Prometheus to Statsd or something, the performance and configuration characteristics of your app change, so you'll probably want to manually revisit every metric. It's a straightforward refactoring that you will probably never do more than once, so I don't think it's worth making it "easy" out of the box.

Finally, we let you configure your app with either flags or environment variables. Generally, I like flags for running things while developing (go run main.go --foo=bar) and environment variables for production. Why pick one or the other when you can do both!

Extensive care has been taken to be able to log everything, including all payloads for bidirectional streams. (HTTP logging is a bit of a disaster; check out the code you have to write to be able to hook every type of http.ResponseWriter without the consumer noticing: https://github.com/prometheus/client_golang/blob/master/prom....)

Some features that are missing or sub-optimal:

* A long chain of interceptors for every request is very expensive. They should all be condensed into one wrapper that does "everything" explicitly. I don't think it's worth the debugging complexity to be able to compose a bunch of middleware at runtime; just write a function that wraps every request that does the stuff you want. If you want more stuff, type it in right there. Easy to read, easy for the computer to execute.

* There should be fine-grained control over logging at runtime. I think you really want to be able to pass in configuration for every sub-logger, rather than a global log level. For example, you may want to adjust log sampling rate at runtime for a particular component, or upgrade a particular component from INFO to DEBUG for temporary investigation. (Without these controls, people stop writing log statements because they're too spammy, and then you have absolutely nothing to help you with ongoing issues.)

* There is absolutely no support for mTLS, which is a crime in this day and age. This is probably the next thing that I'll add; you should be able to watch a volume for changes in TLS key/cert/ca material (so you can rotate keys frequently and without restarting), and require that incoming requests have a certificate trusted by the loaded CA. Outgoing requests should also be marked as "external" or "internal", and use the correct client cert for internal requests. (I somewhat promise to add this soon, because it's sorely needed.)

* I don't think applications should support any non-production features for development, like a different logging configuration. opinionated-server apps take a --pretty_logs argument that configures it to print plain text instead of JSON, but I don't use it anymore; I just pipe the logs through jlog (`go install github.com/jrockway/json-logs/cmd/jlog`) and the results are much prettier than what zap would do itself. And it's the same tool I use every day to look at production logs, so there are no surprises.

Ultimately, I wrote this because I like small pieces of software that do one thing, and found myself writing the same boilerplate over and over. Overall, this has been a success. I have written small utilities (github.com/jrockway/alertmanager-status, github.com/jrockway/nodedns, github.com/jrockway/alertmanager-discord, which I forked from someone else and converted to use opinionated-server), and larger applications (github.com/jrockway/jsso2, github.com/jrockway/ekglue), and overall not having to write the same monitoring/logging code over and over has proven quite productive. I have not achieved my goal of "1 hour to make a production application" quite yet, but opinionated-server is certainly helping.