Repository: elixir-ecto/ecto

Issue #2534: Ecto preload function fails for many_to_many and has_many through
Created: 2018-05-04T12:39:26Z
Ecto 2.2.10
This is for the preload function in the form like:
comment_fn = fn post_ids -> load_comments(post_ids) end
from p in Post, preload: [comment: ^comment_fn]

From a discussion at: https://elixirforum.com/t/ecto-preload-function-discarding-data/14031/5
As far as I can tell:
has_many through passes a list of child ids to the preload function and will exclude anything returned that doesn’t have an id that matches one of those ids. (Only allows a use case where you want the exact same items that would be returned from the main query). This is unexpected because it’s a different set of ids compared to the other cases. I would expect that instead it would be passing the parent ids and letting the query figure out how to load the data.
many_to_many passes the parent ids to the preload function and will exclude anything returned that doesn’t have an id that matches those ids. (Only would work accidentally if the parent and child had the same ids). I would expect the matching of children should be the referenced parent_id in the child and not the child's own id.
has_many passes the parent ids to the preload function and will work as expected matching the parent id to the “parent_id” property on the child items. (Works as expected)
I implemented a simple test app: https://github.com/geofflane/ecto_preload_test
It's uses Post, Comment, and Tag to show has_many, many_to_many, and has_many through examples. It's a bit contrived, but should show the issue.

Duplicate issue #2664: Preload not loading all data?
Created: 2018-08-21T12:13:49Z
I'm facing Ecto behaviour I'm having hard time explaining to myself. Consider this example, which is also published on github as a standalone mix app – ecto_redundancy_issue.

Schemas (hidden for brevity)
defmodule Author do
  use Ecto.Schema

  schema "authors" do
    field :full_name, :string
    field :email, :string
  end
end

defmodule Comment do
  use Ecto.Schema

  schema "comments" do
    field :subject, :string
    field :content, :string
    field :post_id, :integer

    belongs_to :author, Author
  end
end

defmodule Post do
  use Ecto.Schema

  schema "posts" do
    field :title, :string
    field :body, :string

    has_many :comments, Comment
    has_many :commenters, through: [:comments, :author]
  end
end

Data:
RedundancyRepo.insert_all("posts", [
  %{id: 1, title: "My title", body: "My body"}
])

RedundancyRepo.insert_all("authors", [
  %{id: 1, full_name: "Full name 1", email: "client1@server.com"},
  %{id: 2, full_name: "Full name 2", email: "client2@server.com"},
  %{id: 3, full_name: "Full name 3", email: "client3@server.com"}
])

RedundancyRepo.insert_all("comments", [
  %{id: 1, post_id: 1, author_id: 1, subject: "Subject 1", content: "Big content 1"},
  %{id: 2, post_id: 1, author_id: 2, subject: "Subject 2", content: "Big content 2"},
  %{id: 3, post_id: 1, author_id: 3, subject: "Subject 3", content: "Big content 3"},
  %{id: 4, post_id: 1, author_id: 1, subject: "Subject 4", content: "Big content 4"},
  %{id: 5, post_id: 1, author_id: 2, subject: "Subject 5", content: "Big content 5"},
  %{id: 6, post_id: 1, author_id: 3, subject: "Subject 6", content: "Big content 6"}
])
Test code:
commenter_emails_query = from a in Author, select: struct(a, [:email])
posts_query = from p in Post, preload: [commenters: ^commenter_emails_query]

[post] = RedundancyRepo.all(posts_query)
post.commenters == Enum.map(post.comments, & &1.author) |> Enum.uniq |> Enum.sort # => false
I expect the final statement to be true. What am I missing?
Environment

Elixir version: Elixir 1.7.2 (compiled with Erlang/OTP 21)
Database and version: PostgreSQL 10.5
Ecto version (mix deps): master branch, specifically 63d075d
Database adapter and version (mix deps): 0.13.5
Operating system: macOS

Current behavior
Preload is not loading all authors.
Expected behavior
Preload loads all authors.

Closing comment (2018-08-21T12:37:08Z): Oh, thanks! I believe this is a duplicate of #2534. You can confirm it by looking at the SQL queries in logs.