Repository: elixir-ecto/ecto (elixir)

Issue:
Intermittent "invalid association" warnings at compile time
### Elixir version

1.15.6

### Database and Version

PostgreSQL 15.4

### Ecto Versions

3.10.3

### Database Adapter and Versions (postgrex, myxql, etc)

Postgrex

### Current behavior

For a few months now, we've been seeing the following warning show up in our logs intermittently when compiling or running `mix test`:

```warning: invalid association `family` in schema Core.UserDevices.UserDevice: associated module Core.Families.Family is not an Ecto schema
  lib/core/user_devices/user_device.ex:1: Core.UserDevices.UserDevice (module)```

Sometimes there are multiple warnings for a few schemas, sometimes none. The subject of their complaint is often this Family module, though it sometimes varies and can apply to any one of our schemas. I've traced the source of the message to the `after_compile_validation` functions in `association.ex` in Ecto, where it seems that for some reason, Elixir doesn't believe the Family module has `__schema__` exposed. It's definitely working as expected, though, and the app is functioning fine. So this warning seems to be a false alarm.

I assume that something strange is going on here with compilation order, but am unsure. Since it's intermittent, I don't have a simple code example that can reproduce the issue, but I saw [this past issue](https://github.com/crewfinance/ecto/commit/c66ab5f89ef348c7f520f427001c33d95a78bd2e) that seems very similar, as well as [this recent change](https://github.com/elixir-ecto/ecto/commit/31b7e2e5ea035974f1a23adc83d203b3656582ea#diff-175cbcdf996d9bf56858257b3de96731887a2a5e97d26ce474919488499121ccR2247) one call upstream from our issue and am crossing my fingers that others may have reported something similar recently, resulting in this change? Here's a simplified version of our Family schema that is showing the issue (though as I mentioned, any of our Ecto schemas at varying times will sometimes show up, seemingly depending on compile order).

```defmodule Core.Families.Family do
  @moduledoc """
  Representation of an entire family account.
  """

  # Pulls in the LetMe Ecto authorization library and some boilerplate for it.
  use Core.SecuredEctoSchema
  use Ecto.Schema

  import Ecto.Changeset
  import Ecto.Query

  alias __MODULE__
  alias Core.Accounts.Account
  alias Core.Addresses.Address
  alias Core.Cards.Card
  alias Core.Transfers.Transfer
  alias Core.UserDevices.UserDevice
  alias Core.Users.User

  @statuses [
    :activated,
    :deactivated,
    :failed
  ]

  schema "families" do
    field :uuid, Ecto.UUID, autogenerate: true

    field :name, :string
    field :status, Ecto.Enum, values: @statuses, default: :activated

    has_many :accounts, Account
    belongs_to :address, Address
    has_many :cards, Card
    has_many :devices, UserDevice
    has_many :transfers, Transfer
    has_many :users, User

    timestamps type: :utc_datetime_usec
  end

  @doc """
  Family changeset
  """
  def changeset(%Family{} = family, params \\ %{}) do
    cast(family, params, Family.__schema__(:fields))
  end

  @impl LetMe.Schema
  def scope(q, %User{family_id: family_id}, _opts \\ []) do
    where(q, id: ^family_id)
  end
end
```

The other schemas referencing (in the error above, UserDevice) simply have an alias to Core.Families.Family and then reference it with `belongs_to :family, Family`. Only things of note are that our schemas often use the LetMe library for authorization, and TypedEctoSchema to generate types for specs. Neither seem related since removing them from the schemas in the warning doesn’t change the warning, but worth mentioning just in case since I've never tried entirely pulling them from our codebase!

If this isn't enough information to confirm a bug since I'm unable to reproduce, feel free to close--I definitely recognize this isn't a support forum and I've come with some vague details! But if this is ringing any bells we'd love to help track down what's making Ecto mad! (And to get our logs happy again)!

### Expected behavior

Not seeing a warning for a thing that's a functioning Ecto schema!

Pull request:
Use after_verify callback and reduce compile-time work
Closes #4293.

Patch:
diff --git a/examples/friends/mix.lock b/examples/friends/mix.lock
index 043e5515..382de34d 100644
--- a/examples/friends/mix.lock
+++ b/examples/friends/mix.lock
@@ -1,12 +1,12 @@
 %{
   "connection": {:hex, :connection, "1.1.0", "ff2a49c4b75b6fb3e674bfc5536451607270aac754ffd1bdfe175abe4a6d7a68", [:mix], [], "hexpm", "722c1eb0a418fbe91ba7bd59a47e28008a189d47e37e0e7bb85585a016b2869c"},
-  "db_connection": {:hex, :db_connection, "2.6.0", "77d835c472b5b67fc4f29556dee74bf511bbafecdcaf98c27d27fa5918152086", [:mix], [{:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "c2f992d15725e721ec7fbc1189d4ecdb8afef76648c746a8e1cad35e3b8a35f3"},
-  "decimal": {:hex, :decimal, "2.1.1", "5611dca5d4b2c3dd497dec8f68751f1f1a54755e8ed2a966c2633cf885973ad6", [:mix], [], "hexpm", "53cfe5f497ed0e7771ae1a475575603d77425099ba5faef9394932b35020ffcc"},
+  "db_connection": {:hex, :db_connection, "2.7.0", "b99faa9291bb09892c7da373bb82cba59aefa9b36300f6145c5f201c7adf48ec", [:mix], [{:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "dcf08f31b2701f857dfc787fbad78223d61a32204f217f15e881dd93e4bdd3ff"},
+  "decimal": {:hex, :decimal, "2.2.0", "df3d06bb9517e302b1bd265c1e7f16cda51547ad9d99892049340841f3e15836", [:mix], [], "hexpm", "af8daf87384b51b7e611fb1a1f2c4d4876b65ef968fa8bd3adf44cff401c7f21"},
   "ecto": {:hex, :ecto, "2.0.1", "cf97a4d353e14af3d3cc3b4452cfbd18b3aeee1fb4075475efeccec3853444a9", [:mix], [{:poison, "~> 1.5 or ~> 2.0", [hex: :poison, optional: true]}, {:sbroker, "~> 1.0-beta", [hex: :sbroker, optional: true]}, {:mariaex, "~> 0.7.7", [hex: :mariaex, optional: true]}, {:postgrex, "~> 0.11.2", [hex: :postgrex, optional: true]}, {:db_connection, "~> 1.0-rc.2", [hex: :db_connection, optional: true]}, {:decimal, "~> 1.0", [hex: :decimal, optional: false]}, {:poolboy, "~> 1.5", [hex: :poolboy, optional: false]}]},
   "ecto_sql": {:hex, :ecto_sql, "3.11.3", "4eb7348ff8101fbc4e6bbc5a4404a24fecbe73a3372d16569526b0cf34ebc195", [:mix], [{:db_connection, "~> 2.4.1 or ~> 2.5", [hex: :db_connection, repo: "hexpm", optional: false]}, {:ecto, "~> 3.11.0", [hex: :ecto, repo: "hexpm", optional: false]}, {:myxql, "~> 0.6", [hex: :myxql, repo: "hexpm", optional: true]}, {:postgrex, "~> 0.16 or ~> 1.0", [hex: :postgrex, repo: "hexpm", optional: true]}, {:tds, "~> 2.1.1 or ~> 2.2", [hex: :tds, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4.0 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "e5f36e3d736b99c7fee3e631333b8394ade4bafe9d96d35669fca2d81c2be928"},
-  "jason": {:hex, :jason, "1.4.1", "af1504e35f629ddcdd6addb3513c3853991f694921b1b9368b0bd32beb9f1b63", [:mix], [{:decimal, "~> 1.0 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm", "fbb01ecdfd565b56261302f7e1fcc27c4fb8f32d56eab74db621fc154604a7a1"},
+  "jason": {:hex, :jason, "1.4.4", "b9226785a9aa77b6857ca22832cffa5d5011a667207eb2a0ad56adb5db443b8a", [:mix], [{:decimal, "~> 1.0 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm", "c5eb0cab91f094599f94d55bc63409236a8ec69a21a67814529e8d5f6cc90b3b"},
   "poison": {:hex, :poison, "4.0.1", "bcb755a16fac91cad79bfe9fc3585bb07b9331e50cfe3420a24bcc2d735709ae", [:mix], [], "hexpm"},
   "poolboy": {:hex, :poolboy, "1.5.1", "6b46163901cfd0a1b43d692657ed9d7e599853b3b21b95ae5ae0a777cf9b6ca8", [], [], "hexpm"},
-  "postgrex": {:hex, :postgrex, "0.18.0", "f34664101eaca11ff24481ed4c378492fed2ff416cd9b06c399e90f321867d7e", [:mix], [{:db_connection, "~> 2.1", [hex: :db_connection, repo: "hexpm", optional: false]}, {:decimal, "~> 1.5 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:table, "~> 0.1.0", [hex: :table, repo: "hexpm", optional: true]}], "hexpm", "a042989ba1bc1cca7383ebb9e461398e3f89f868c92ce6671feb7ef132a252d1"},
-  "telemetry": {:hex, :telemetry, "1.2.1", "68fdfe8d8f05a8428483a97d7aab2f268aaff24b49e0f599faa091f1d4e7f61c", [:rebar3], [], "hexpm", "dad9ce9d8effc621708f99eac538ef1cbe05d6a874dd741de2e689c47feafed5"},
+  "postgrex": {:hex, :postgrex, "0.19.3", "a0bda6e3bc75ec07fca5b0a89bffd242ca209a4822a9533e7d3e84ee80707e19", [:mix], [{:db_connection, "~> 2.1", [hex: :db_connection, repo: "hexpm", optional: false]}, {:decimal, "~> 1.5 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:table, "~> 0.1.0", [hex: :table, repo: "hexpm", optional: true]}], "hexpm", "d31c28053655b78f47f948c85bb1cf86a9c1f8ead346ba1aa0d0df017fa05b61"},
+  "telemetry": {:hex, :telemetry, "1.3.0", "fedebbae410d715cf8e7062c96a1ef32ec22e764197f70cda73d82778d61e7a2", [:rebar3], [], "hexpm", "7015fc8919dbe63764f4b4b87a95b7c0996bd539e0d499be6ec9d7f3875b79e6"},
 }
diff --git a/lib/ecto/association.ex b/lib/ecto/association.ex
index 88fa398b..d5f7a293 100644
--- a/lib/ecto/association.ex
+++ b/lib/ecto/association.ex
@@ -44,21 +44,6 @@ defmodule Ecto.Association do
 
   alias Ecto.Query.Builder.OrderBy
 
-  @doc """
-  Helper to check if a queryable is compiled.
-  """
-  def ensure_compiled(queryable, env) do
-    if not is_atom(queryable) or queryable in env.context_modules do
-      :skip
-    else
-      case Code.ensure_compiled(queryable) do
-        {:module, _} -> :compiled
-        {:error, :unavailable} -> :skip
-        {:error, _} -> :not_found
-      end
-    end
-  end
-
   @doc """
   Builds the association struct.
 
@@ -87,7 +72,7 @@ defmodule Ecto.Association do
   Useful for checking if associated modules exist without running
   into deadlocks.
   """
-  @callback after_compile_validation(t, Macro.Env.t()) :: :ok | {:error, String.t()}
+  @callback after_verify_validation(t) :: :ok | {:error, String.t()}
 
   @doc """
   Builds a struct for the given association.
@@ -771,14 +756,12 @@ defmodule Ecto.Association.Has do
   ]
 
   @impl true
-  def after_compile_validation(%{queryable: queryable, related_key: related_key}, env) do
-    compiled = Ecto.Association.ensure_compiled(queryable, env)
-
+  def after_verify_validation(%{queryable: queryable, related_key: related_key}) do
     cond do
-      compiled == :skip ->
+      not is_atom(queryable) ->
         :ok
 
-      compiled == :not_found ->
+      not Code.ensure_loaded?(queryable) ->
         {:error, "associated schema #{inspect(queryable)} does not exist"}
 
       not function_exported?(queryable, :__schema__, 2) ->
@@ -1040,7 +1023,7 @@ defmodule Ecto.Association.HasThrough do
   ]
 
   @impl true
-  def after_compile_validation(_, _) do
+  def after_verify_validation(_) do
     :ok
   end
 
@@ -1143,14 +1126,12 @@ defmodule Ecto.Association.BelongsTo do
   ]
 
   @impl true
-  def after_compile_validation(%{queryable: queryable, related_key: related_key}, env) do
-    compiled = Ecto.Association.ensure_compiled(queryable, env)
-
+  def after_verify_validation(%{queryable: queryable, related_key: related_key}) do
     cond do
-      compiled == :skip ->
+      not is_atom(queryable) ->
         :ok
 
-      compiled == :not_found ->
+      not Code.ensure_loaded?(queryable) ->
         {:error, "associated schema #{inspect(queryable)} does not exist"}
 
       not function_exported?(queryable, :__schema__, 2) ->
@@ -1346,24 +1327,21 @@ defmodule Ecto.Association.ManyToMany do
   ]
 
   @impl true
-  def after_compile_validation(%{queryable: queryable, join_through: join_through}, env) do
-    compiled = Ecto.Association.ensure_compiled(queryable, env)
-    join_compiled = Ecto.Association.ensure_compiled(join_through, env)
-
+  def after_verify_validation(%{queryable: queryable, join_through: join_through}) do
     cond do
-      compiled == :skip ->
+      not is_atom(queryable) ->
         :ok
 
-      compiled == :not_found ->
+      not Code.ensure_loaded?(queryable) ->
         {:error, "associated schema #{inspect(queryable)} does not exist"}
 
       not function_exported?(queryable, :__schema__, 2) ->
         {:error, "associated module #{inspect(queryable)} is not an Ecto schema"}
 
-      join_compiled == :skip ->
+      not is_atom(join_through) ->
         :ok
 
-      join_compiled == :not_found ->
+      not Code.ensure_loaded?(join_through) ->
         {:error, ":join_through schema #{inspect(join_through)} does not exist"}
 
       not function_exported?(join_through, :__schema__, 2) ->
diff --git a/lib/ecto/schema.ex b/lib/ecto/schema.ex
index 84269857..1c723622 100644
--- a/lib/ecto/schema.ex
+++ b/lib/ecto/schema.ex
@@ -124,7 +124,7 @@ defmodule Ecto.Schema do
   A field marked with `redact: true` will display a value of `**redacted**`
   when inspected in changes inside a `Ecto.Changeset` and be excluded from
   inspect on the schema unless the schema module is tagged with
-  the option `@ecto_derive_inspect_for_redacted_fields false`.
+  the option `@derive_inspect_for_redacted_fields false`.
 
   ## Schema attributes
 
@@ -161,6 +161,10 @@ defmodule Ecto.Schema do
     * `@derive` - the same as `@derive` available in `Kernel.defstruct/1`
       as the schema defines a struct behind the scenes;
 
+    * `@derive_inspect_for_redacted_fields false` - Ecto will automatically
+      derive the `Inspect` protocol if any redacted fields are set. This option
+      sets it to false;
+
     * `@field_source_mapper` - a function that receives the current field name
       and returns the mapping of this field name in the underlying source.
       In other words, it is a mechanism to automatically generate the `:source`
@@ -494,13 +498,6 @@ defmodule Ecto.Schema do
     quote do
       import Ecto.Schema, only: [schema: 2, embedded_schema: 1]
 
-      @primary_key nil
-      @timestamps_opts []
-      @foreign_key_type :id
-      @schema_prefix nil
-      @schema_context nil
-      @field_source_mapper fn x -> x end
-
       Module.register_attribute(__MODULE__, :ecto_primary_keys, accumulate: true)
       Module.register_attribute(__MODULE__, :ecto_fields, accumulate: true)
       Module.register_attribute(__MODULE__, :ecto_virtual_fields, accumulate: true)
@@ -512,8 +509,6 @@ defmodule Ecto.Schema do
       Module.register_attribute(__MODULE__, :ecto_autogenerate, accumulate: true)
       Module.register_attribute(__MODULE__, :ecto_autoupdate, accumulate: true)
       Module.register_attribute(__MODULE__, :ecto_redact_fields, accumulate: true)
-      Module.put_attribute(__MODULE__, :ecto_derive_inspect_for_redacted_fields, true)
-      Module.put_attribute(__MODULE__, :ecto_autogenerate_id, nil)
     end
   end
 
@@ -554,7 +549,7 @@ defmodule Ecto.Schema do
   when calling `c:Ecto.Repo.insert/2` or `c:Ecto.Repo.update/2`.
   """
   defmacro embedded_schema(do: block) do
-    schema(__CALLER__, nil, false, :binary_id, block)
+    schema(nil, false, :binary_id, block)
   end
 
   @doc """
@@ -565,64 +560,15 @@ defmodule Ecto.Schema do
   as value and can be manipulated with the `Ecto.put_meta/2` function.
   """
   defmacro schema(source, do: block) do
-    schema(__CALLER__, source, true, :id, block)
+    schema(source, true, :id, block)
   end
 
-  defp schema(caller, source, meta?, type, block) do
+  defp schema(source, meta?, type, block) do
     prelude =
       quote do
-        if line = Module.get_attribute(__MODULE__, :ecto_schema_defined) do
-          raise "schema already defined for #{inspect(__MODULE__)} on line #{line}"
-        end
-
-        @ecto_schema_defined unquote(caller.line)
-
-        @after_compile Ecto.Schema
-        Module.register_attribute(__MODULE__, :ecto_changeset_fields, accumulate: true)
-        Module.register_attribute(__MODULE__, :ecto_struct_fields, accumulate: true)
-
         meta? = unquote(meta?)
         source = unquote(source)
-        prefix = @schema_prefix
-        context = @schema_context
-
-        # Those module attributes are accessed only dynamically
-        # so we explicitly reference them here to avoid warnings.
-        _ = @foreign_key_type
-        _ = @timestamps_opts
-
-        if meta? do
-          unless is_binary(source) do
-            raise ArgumentError, "schema source must be a string, got: #{inspect(source)}"
-          end
-
-          meta = %Metadata{
-            state: :built,
-            source: source,
-            prefix: prefix,
-            context: context,
-            schema: __MODULE__
-          }
-
-          Module.put_attribute(__MODULE__, :ecto_struct_fields, {:__meta__, meta})
-        end
-
-        if @primary_key == nil do
-          @primary_key {:id, unquote(type), autogenerate: true}
-        end
-
-        primary_key_fields =
-          case @primary_key do
-            false ->
-              []
-
-            {name, type, opts} ->
-              Ecto.Schema.__field__(__MODULE__, name, type, [primary_key: true] ++ opts)
-              [name]
-
-            other ->
-              raise ArgumentError, "@primary_key must be false or {name, type, opts}"
-          end
+        prefix = Ecto.Schema.__schema__(__MODULE__, __ENV__.line, source, meta?, unquote(type))
 
         try do
           import Ecto.Schema
@@ -634,70 +580,13 @@ defmodule Ecto.Schema do
 
     postlude =
       quote unquote: false do
-        primary_key_fields = @ecto_primary_keys |> Enum.reverse()
-        autogenerate = @ecto_autogenerate |> Enum.reverse()
-        autoupdate = @ecto_autoupdate |> Enum.reverse()
-        fields = @ecto_fields |> Enum.reverse()
-        query_fields = @ecto_query_fields |> Enum.reverse()
-        virtual_fields = @ecto_virtual_fields |> Enum.reverse()
-        field_sources = @ecto_field_sources |> Enum.reverse()
-        assocs = @ecto_assocs |> Enum.reverse()
-        embeds = @ecto_embeds |> Enum.reverse()
-        redacted_fields = @ecto_redact_fields
-        loaded = Ecto.Schema.__loaded__(__MODULE__, @ecto_struct_fields)
-
-        if redacted_fields != [] and not List.keymember?(@derive, Inspect, 0) and
-             @ecto_derive_inspect_for_redacted_fields do
-          @derive {Inspect, except: @ecto_redact_fields}
-        end
-
-        defstruct Enum.reverse(@ecto_struct_fields)
+        {struct_fields, bags_of_clauses} = Ecto.Schema.__schema__(__MODULE__)
+        defstruct struct_fields
 
         def __changeset__ do
           %{unquote_splicing(Macro.escape(@ecto_changeset_fields))}
         end
 
-        def __schema__(:prefix), do: unquote(Macro.escape(prefix))
-        def __schema__(:source), do: unquote(source)
-        def __schema__(:fields), do: unquote(Enum.map(fields, &elem(&1, 0)))
-        def __schema__(:query_fields), do: unquote(Enum.map(query_fields, &elem(&1, 0)))
-        def __schema__(:primary_key), do: unquote(primary_key_fields)
-        def __schema__(:hash), do: unquote(:erlang.phash2({primary_key_fields, query_fields}))
-        def __schema__(:read_after_writes), do: unquote(Enum.reverse(@ecto_raw))
-        def __schema__(:autogenerate_id), do: unquote(Macro.escape(@ecto_autogenerate_id))
-        def __schema__(:autogenerate), do: unquote(Macro.escape(autogenerate))
-        def __schema__(:autoupdate), do: unquote(Macro.escape(autoupdate))
-        def __schema__(:loaded), do: unquote(Macro.escape(loaded))
-        def __schema__(:redact_fields), do: unquote(redacted_fields)
-        def __schema__(:virtual_fields), do: unquote(Enum.map(virtual_fields, &elem(&1, 0)))
-
-        def __schema__(:updatable_fields) do
-          unquote(
-            for {name, {_, writable}} <- fields, reduce: {[], []} do
-              {keep, drop} ->
-                case writable do
-                  :always -> {[name | keep], drop}
-                  _ -> {keep, [name | drop]}
-                end
-            end
-          )
-        end
-
-        def __schema__(:insertable_fields) do
-          unquote(
-            for {name, {_, writable}} <- fields, reduce: {[], []} do
-              {keep, drop} ->
-                case writable do
-                  :never -> {keep, [name | drop]}
-                  _ -> {[name | keep], drop}
-                end
-            end
-          )
-        end
-
-        def __schema__(:autogenerate_fields),
-          do: unquote(Enum.flat_map(autogenerate, &elem(&1, 0)))
-
         if meta? do
           def __schema__(:query) do
             %Ecto.Query{
@@ -709,15 +598,10 @@ defmodule Ecto.Schema do
           end
         end
 
-        for clauses <-
-              Ecto.Schema.__schema__(
-                fields,
-                field_sources,
-                assocs,
-                embeds,
-                virtual_fields
-              ),
-            {args, body} <- clauses do
+        def __schema__(:source), do: unquote(source)
+        def __schema__(:prefix), do: unquote(Macro.escape(prefix))
+
+        for clauses <- bags_of_clauses, {args, body} <- clauses do
           def __schema__(unquote_splicing(args)), do: unquote(body)
         end
 
@@ -826,7 +710,7 @@ defmodule Ecto.Schema do
   """
   defmacro timestamps(opts \\ []) do
     quote bind_quoted: binding() do
-      Ecto.Schema.__define_timestamps__(__MODULE__, Keyword.merge(@timestamps_opts, opts))
+      Ecto.Schema.__define_timestamps__(__MODULE__, opts)
     end
   end
 
@@ -2056,14 +1940,6 @@ defmodule Ecto.Schema do
     type.from_unix!(System.os_time(:microsecond), :microsecond)
   end
 
-  @doc false
-  def __loaded__(module, struct_fields) do
-    case Map.new([{:__struct__, module} | struct_fields]) do
-      %{__meta__: meta} = struct -> %{struct | __meta__: Map.put(meta, :state, :loaded)}
-      struct -> struct
-    end
-  end
-
   @doc false
   def __field__(mod, name, type, opts) do
     # Check the field type before we check options because it is
@@ -2095,7 +1971,9 @@ defmodule Ecto.Schema do
     if virtual? do
       Module.put_attribute(mod, :ecto_virtual_fields, {name, type})
     else
-      source = opts[:source] || Module.get_attribute(mod, :field_source_mapper).(name)
+      source =
+        opts[:source] ||
+          Module.get_attribute(mod, :field_source_mapper, &Function.identity/1).(name)
 
       if not is_atom(source) do
         raise ArgumentError,
@@ -2142,7 +2020,8 @@ defmodule Ecto.Schema do
   end
 
   @doc false
-  def __define_timestamps__(mod, timestamps) do
+  def __define_timestamps__(mod, opts) do
+    timestamps = Keyword.merge(Module.get_attribute(mod, :timestamps_opts, []), opts)
     type = Keyword.get(timestamps, :type, :naive_datetime)
     autogen = timestamps[:autogenerate] || {Ecto.Schema, :__timestamps__, [type]}
 
@@ -2221,7 +2100,7 @@ defmodule Ecto.Schema do
     opts = Keyword.put_new(opts, :foreign_key, :"#{name}_id")
 
     foreign_key_name = opts[:foreign_key]
-    foreign_key_type = opts[:type] || Module.get_attribute(mod, :foreign_key_type)
+    foreign_key_type = opts[:type] || Module.get_attribute(mod, :foreign_key_type, :id)
     foreign_key_type = check_field_type!(mod, name, foreign_key_type, opts)
     check_options!(foreign_key_type, opts, @valid_belongs_to_options, "belongs_to/3")
 
@@ -2320,23 +2199,22 @@ defmodule Ecto.Schema do
   ## Quoted callbacks
 
   @doc false
-  def __after_compile__(%{module: module} = env, _) do
+  def __after_verify__(module) do
     # If we are compiling code, we can validate associations now,
     # as the Elixir compiler will solve dependencies.
-    if Code.can_await_module_compilation?() do
-      for name <- module.__schema__(:associations) do
-        assoc = module.__schema__(:association, name)
-
-        case assoc.__struct__.after_compile_validation(assoc, env) do
-          :ok ->
-            :ok
-
-          {:error, message} ->
-            IO.warn(
-              "invalid association `#{assoc.field}` in schema #{inspect(module)}: #{message}",
-              Macro.Env.stacktrace(env)
-            )
-        end
+    for name <- module.__schema__(:associations) do
+      assoc = module.__schema__(:association, name)
+
+      case assoc.__struct__.after_verify_validation(assoc) do
+        :ok ->
+          :ok
+
+        {:error, message} ->
+          IO.warn(
+            "invalid association `#{assoc.field}` in schema #{inspect(module)}: #{message}",
+            module: module,
+            file: to_string(module.__info__(:compile)[:source] || "nofile")
+          )
       end
     end
 
@@ -2344,7 +2222,92 @@ defmodule Ecto.Schema do
   end
 
   @doc false
-  def __schema__(fields, field_sources, assocs, embeds, virtual_fields) do
+  def __schema__(module, line, source, meta?, type) do
+    if previous_line = Module.get_attribute(module, :ecto_schema_defined) do
+      raise "schema already defined for #{inspect(module)} on line #{previous_line}"
+    end
+
+    Module.put_attribute(module, :ecto_schema_defined, line)
+
+    if Code.can_await_module_compilation?() do
+      Module.put_attribute(module, :after_verify, Ecto.Schema)
+    end
+
+    Module.register_attribute(module, :ecto_changeset_fields, accumulate: true)
+    Module.register_attribute(module, :ecto_struct_fields, accumulate: true)
+
+    # Those module attributes are accessed only dynamically
+    # so we explicitly reference them here to avoid warnings.
+    Module.get_attribute(module, :foreign_key_type)
+    Module.get_attribute(module, :timestamps_opts)
+
+    prefix = Module.get_attribute(module, :schema_prefix)
+    context = Module.get_attribute(module, :schema_context)
+
+    if meta? do
+      unless is_binary(source) do
+        raise ArgumentError, "schema source must be a string, got: #{inspect(source)}"
+      end
+
+      meta = %Metadata{
+        state: :built,
+        source: source,
+        prefix: prefix,
+        context: context,
+        schema: module
+      }
+
+      Module.put_attribute(module, :ecto_struct_fields, {:__meta__, meta})
+    end
+
+    if Module.get_attribute(module, :primary_key) == nil do
+      Module.put_attribute(module, :primary_key, {:id, type, autogenerate: true})
+    end
+
+    case Module.get_attribute(module, :primary_key) do
+      false ->
+        []
+
+      {name, type, opts} ->
+        Ecto.Schema.__field__(module, name, type, [primary_key: true] ++ opts)
+        [name]
+
+      _other ->
+        raise ArgumentError, "@primary_key must be false or {name, type, opts}"
+    end
+
+    prefix
+  end
+
+  @doc false
+  def __schema__(module) do
+    fields = Module.get_attribute(module, :ecto_fields) |> Enum.reverse()
+    field_sources = Module.get_attribute(module, :ecto_field_sources) |> Enum.reverse()
+    assocs = Module.get_attribute(module, :ecto_assocs) |> Enum.reverse()
+    embeds = Module.get_attribute(module, :ecto_embeds) |> Enum.reverse()
+    virtual_fields = Module.get_attribute(module, :ecto_virtual_fields) |> Enum.reverse()
+    redacted_fields = Module.get_attribute(module, :ecto_redact_fields)
+    primary_key_fields = Module.get_attribute(module, :ecto_primary_keys) |> Enum.reverse()
+    query_fields = Module.get_attribute(module, :ecto_query_fields) |> Enum.reverse()
+    autogenerate = Module.get_attribute(module, :ecto_autogenerate) |> Enum.reverse()
+    autoupdate = Module.get_attribute(module, :ecto_autoupdate) |> Enum.reverse()
+    read_after_writes = Module.get_attribute(module, :ecto_raw) |> Enum.reverse()
+    autogenerate_id = Module.get_attribute(module, :ecto_autogenerate_id)
+
+    struct_fields = Module.get_attribute(module, :ecto_struct_fields) |> Enum.reverse()
+    derive = Module.get_attribute(module, :derive)
+
+    if redacted_fields != [] and not List.keymember?(derive, Inspect, 0) and
+         derive_inspect?(module) do
+      Module.put_attribute(module, :derive, {Inspect, except: redacted_fields})
+    end
+
+    loaded =
+      case Map.new([{:__struct__, module} | struct_fields]) do
+        %{__meta__: meta} = struct -> %{struct | __meta__: Map.put(meta, :state, :loaded)}
+        struct -> struct
+      end
+
     load =
       for {name, {type, _writable}} <- fields do
         if alias = field_sources[name] do
@@ -2388,11 +2351,43 @@ defmodule Ecto.Schema do
 
     embed_names = Enum.map(embeds, &elem(&1, 0))
 
+    updatable =
+      for {name, {_, writable}} <- fields, reduce: {[], []} do
+        {keep, drop} ->
+          case writable do
+            :always -> {[name | keep], drop}
+            _ -> {keep, [name | drop]}
+          end
+      end
+
+    insertable =
+      for {name, {_, writable}} <- fields, reduce: {[], []} do
+        {keep, drop} ->
+          case writable do
+            :never -> {keep, [name | drop]}
+            _ -> {[name | keep], drop}
+          end
+      end
+
     single_arg = [
       {[:dump], dump |> Map.new() |> Macro.escape()},
       {[:load], load |> Macro.escape()},
       {[:associations], assoc_names},
-      {[:embeds], embed_names}
+      {[:embeds], embed_names},
+      {[:updatable_fields], updatable},
+      {[:insertable_fields], insertable},
+      {[:redact_fields], redacted_fields},
+      {[:autogenerate_fields], Enum.flat_map(autogenerate, &elem(&1, 0))},
+      {[:virtual_fields], Enum.map(virtual_fields, &elem(&1, 0))},
+      {[:fields], Enum.map(fields, &elem(&1, 0))},
+      {[:query_fields], Enum.map(query_fields, &elem(&1, 0))},
+      {[:primary_key], primary_key_fields},
+      {[:hash], :erlang.phash2({primary_key_fields, query_fields})},
+      {[:read_after_writes], read_after_writes},
+      {[:autogenerate_id], Macro.escape(autogenerate_id)},
+      {[:autogenerate], Macro.escape(autogenerate)},
+      {[:autoupdate], Macro.escape(autoupdate)},
+      {[:loaded], Macro.escape(loaded)}
     ]
 
     catch_all = [
@@ -2403,15 +2398,32 @@ defmodule Ecto.Schema do
       {[:embed, quote(do: _)], nil}
     ]
 
-    [
-      single_arg,
-      field_sources_quoted,
-      types_quoted,
-      virtual_types_quoted,
-      assoc_quoted,
-      embed_quoted,
-      catch_all
-    ]
+    bags_of_clauses =
+      [
+        single_arg,
+        field_sources_quoted,
+        types_quoted,
+        virtual_types_quoted,
+        assoc_quoted,
+        embed_quoted,
+        catch_all
+      ]
+
+    {struct_fields, bags_of_clauses}
+  end
+
+  defp derive_inspect?(module) do
+    case Module.get_attribute(module, :ecto_derive_inspect_for_redacted_fields) do
+      false ->
+        IO.warn(
+          "@ecto_derive_inspect_for_redacted_fields is deprecated, set @derive_inspect_for_redacted_fields instead"
+        )
+
+        false
+
+      _ ->
+        Module.get_attribute(module, :derive_inspect_for_redacted_fields, true)
+    end
   end
 
   ## Private


Test patch:
diff --git a/test/ecto/schema_test.exs b/test/ecto/schema_test.exs
index ea6c1cd0..b0b91583 100644
--- a/test/ecto/schema_test.exs
+++ b/test/ecto/schema_test.exs
@@ -27,18 +27,53 @@ defmodule Ecto.SchemaTest do
     assert Schema.__schema__(:prefix) == nil
 
     assert Schema.__schema__(:fields) ==
-             [:id, :name, :email, :password, :count, :array, :uuid, :no_query_load, :unwritable, :non_updatable, :comment_id]
+             [
+               :id,
+               :name,
+               :email,
+               :password,
+               :count,
+               :array,
+               :uuid,
+               :no_query_load,
+               :unwritable,
+               :non_updatable,
+               :comment_id
+             ]
 
     assert Schema.__schema__(:insertable_fields) ==
-             {[:comment_id, :non_updatable, :no_query_load, :uuid, :array, :count, :password, :email, :name, :id], [:unwritable]}
+             {[
+                :comment_id,
+                :non_updatable,
+                :no_query_load,
+                :uuid,
+                :array,
+                :count,
+                :password,
+                :email,
+                :name,
+                :id
+              ], [:unwritable]}
 
     assert Schema.__schema__(:updatable_fields) ==
-             {[:comment_id, :no_query_load, :uuid, :array, :count, :password, :email, :name, :id], [:non_updatable, :unwritable]}
+             {[:comment_id, :no_query_load, :uuid, :array, :count, :password, :email, :name, :id],
+              [:non_updatable, :unwritable]}
 
     assert Schema.__schema__(:virtual_fields) == [:temp]
 
     assert Schema.__schema__(:query_fields) ==
-             [:id, :name, :email, :password, :count, :array, :uuid, :unwritable, :non_updatable, :comment_id]
+             [
+               :id,
+               :name,
+               :email,
+               :password,
+               :count,
+               :array,
+               :uuid,
+               :unwritable,
+               :non_updatable,
+               :comment_id
+             ]
 
     assert Schema.__schema__(:read_after_writes) == [:email, :count]
     assert Schema.__schema__(:primary_key) == [:id]
@@ -160,7 +195,7 @@ defmodule Ecto.SchemaTest do
   defmodule SchemaWithoutDeriveInspect do
     use Ecto.Schema
 
-    @ecto_derive_inspect_for_redacted_fields false
+    @derive_inspect_for_redacted_fields false
 
     schema "my_schema" do
       field :password, :string, redact: true
@@ -639,7 +674,7 @@ defmodule Ecto.SchemaTest do
                    end
                  end
 
-     assert_raise ArgumentError,
+    assert_raise ArgumentError,
                  "autogenerated fields must always be writable",
                  fn ->
                    defmodule AutogenerateFail do