Repository: elixir-ecto/ecto

Issue #2615: Primary Key ignores custom Ecto.Type#load after Repo.insert, Repo.all since 2.2.0
Created: 2018-07-11T12:07:22Z
When updating from 2.1.6 to 2.2.0 (or higher) we are running into troubles with "deserializing" custom Ecto.Types for primary key. It works fine with normal columns. All our primary keys are UUIDs inside the Postgresql DB .
Our Ecto.Type Custom.PrefixedId transparently adds/removes a prefix to the ids. So clients can work with friendly uuids that indicate their type: e.g. blog_12323-..-23121, post_12323-..-123123
defmodule Blog.Post do
  # adds a "virtual" prefix to a uuid column.  
  alias Custom.PrefixedId

  @primary_key {:id, PrefixedId, autogenerate: true}
  @derive {Phoenix.Param, key: :id}
  @foreign_key_type Ecto.UUID

  schema "posts" do  
    field(:other_id, PrefixedId) # database type is :uuid
  end
end
BEFORE. Correct behaviour with ecto 2.1.6
b = Repo.insert!(%Blog.Post{other_id: UUID.uuid4})
b.id       # => lorem_c0690cf7-3e57-4338-bb0c-9e2279d1d242
b.other_id # => lorem_86399b0f-f9b7-4fd5-93ea-c4665f991cc1

b = Repo.get(Blog.Post, "lorem_c0690cf7-3e57-4338-bb0c-9e2279d1d242")
b.id       # => "lorem_c0690cf7-3e57-4338-bb0c-9e2279d1d242"
b.other_id # => "lorem_86399b0f-f9b7-4fd5-93ea-c4665f991cc1"

[b] = Repo.all(Blog.Post)
b.id       # => "lorem_c0690cf7-3e57-4338-bb0c-9e2279d1d242"
b.other_id # => "lorem_86399b0f-f9b7-4fd5-93ea-c4665f991cc1"
AFTER. with ecto ~> 2.2.0 (all versions)
Blog.Post#id is no longer consistently "deserialized", but other columns with the same Ecto.Type are.
b = Repo.insert!(%Blog.Post{other_id: UUID.uuid4})
b.id       # => c0690cf7-3e57-4338-bb0c-9e2279d1d242
b.other_id # => lorem_86399b0f-f9b7-4fd5-93ea-c4665f991cc1

# But with get it seems to work just fine
b = Repo.get(Blog.Post, "lorem_c0690cf7-3e57-4338-bb0c-9e2279d1d242")
b.id       # => "lorem_c0690cf7-3e57-4338-bb0c-9e2279d1d242" 
b.other_id # => "lorem_86399b0f-f9b7-4fd5-93ea-c4665f991cc1"

# Updating that records other_id keeps the id correct, but does not prefix other_id
b = Repo.get(Blog.Post, "lorem_c0690cf7-3e57-4338-bb0c-9e2279d1d242")
u = Ecto.Changeset.change(b, other_id: UUID.uuid4) |> Repo.update!
u.id       # => "lorem_c0690cf7-3e57-4338-bb0c-9e2279d1d242" 
u.other_id # => "000e235d-e77b-4ef5-91a1-e7bfc3db130e"

[b] = Repo.all(Blog.Post)
b.id       # => "lorem_c0690cf7-3e57-4338-bb0c-9e2279d1d242"
b.other_id # => "lorem_86399b0f-f9b7-4fd5-93ea-c4665f991cc1"
Most probably somehow the Ecto.Type given in @primary_key is not properly registered. Or not consistently applied with primary key columns.
Environment:
PostgreSQL 9.6
Erlang/OTP 20 [erts-9.2.1] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]
Elixir 1.6.4 (compiled with OTP 19)

Updates

I made a mistake, Repo.all is actually working fine in the new version. Updated description.
Updated with a description when updating

Duplicate issue #3704: Primary key ignores custom type autogenerate at Repo.insert
Created: 2021-08-06T11:06:16Z
Environment

Elixir version (elixir -v):

Erlang/OTP 24 [erts-12.0] [source] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:1] [jit]

Elixir 1.12.2 (compiled with Erlang/OTP 24)


Database and version: PostgreSQL 13.3
Ecto version (mix deps):

* ecto 3.6.1 (Hex package) (mix)
  locked at 3.6.1 (ecto) cbb3294a
  ok
* ecto_sql 3.6.1 (Hex package) (mix)
  locked at 3.6.1 (ecto_sql) 66f35c3f
  ok


Database adapter and version (mix deps):

* postgrex 0.15.9 (Hex package) (mix)
  locked at 0.15.9 (postgrex) 61071910
  ok


Operating system: Arch Linux x86_64

Current behaviour
I have a schema set up with the following primary_key compile time variable (Note: SnowflakeType is an alias for Snowchat.Postgres.Types.Snowflake)
@primary_key {:id, SnowflakeType, autogenerate: true}
Custom type code:
defmodule Snowchat.Postgres.Types.Snowflake do
  use Ecto.Type

  @spec autogenerate :: {:error, :backwards_clock} | {:ok, integer}
  def autogenerate() do
    IO.puts("!!!! Autogenerate")
    Snowflake.next_id()
  end

  def type, do: :integer

  # external to internal
  def cast(id) when is_integer(id) do
    {:ok, id}
  end
  def cast(_), do: :error

  # database to internal
  def load(id) when is_integer(id) do
    {:ok, id}
  end
  def load(_), do: :error

  # internal to database
  def dump(id) when is_integer(id) do
    {:ok, id}
  end
  def dump(_), do: :error
end
Changeset:
  import Ecto.Changeset

  @required ~w(username password display_name status is_developer)a
  @optional ~w(avatar_url banner_url reason_for_ban last_online last_ip custom_status bio)a

  @spec changeset(
          {map, map}
          | %{
              :__struct__ => atom | %{:__changeset__ => map, optional(any) => any},
              optional(atom) => any
            },
          :invalid | %{optional(:__struct__) => none, optional(atom | binary) => any}
        ) :: Ecto.Changeset.t()
  def changeset(user, attrs \\ %{}) do
    user
    |> cast(attrs, @optional ++ @required)
    |> validate_required(@required)
    |> update_change(:display_name, &String.trim/1)
    |> validate_format(:username, ~r/^([a-z0-9\-_,.]){4,15}$/)
    |> validate_length(:display_name, min: 3, max: 20)
    |> validate_length(:bio, min: 0, max: 160)
    |> unique_constraint(:username)
  end
Error:
13:06:07.493 [error] #PID<0.733.0> running Snowchat.HTTP.Router (connection #PID<0.732.0>, stream id 1) terminated
Server: localhost:80 (http)
Request: POST /api/auth/register
** (exit) an exception was raised:
    ** (Postgrex.Error) ERROR 23502 (not_null_violation) null value in column "id" of relation "users" violates not-null constraint

    table: users
    column: id

When running Repo.insert/2 with a changeset (without the id key-value pair) and the returning option set to true it logs a not-null error of the id column.
Expected behavior
I expect to not get the error and have the id value get automatically generated.

Closing comment (2021-08-06T11:07:09Z): Possibly related to #2615