Issue 1:
I have a monorepo with the following structure:

AdmissionSuite
Admit
AdmitConfiguration
AdmissionTargets

AdmissionSuite has no "real" source code on its own (though to avoid Pkg errors I have to add a dummy src/AdmissionSuite.jl with an empty module), its real role is to contain Project.toml and Manifest.toml files with all 3 subdirectories devved. This is an attempt to distribute a coherent collection of unregistered mutually-dependent packages (xref https://discourse.julialang.org/t/having-local-packages-in-a-monorepo-depend-on-one-another/50454/4?u=tim.holy).

AdmitConfiguration uses Preferences.jl and there is a AdmissionSuite/AdmitConfiguration/LocalPreferences.toml file. For debugging purposes, to base/loading.jl I added the following:

diff --git a/base/loading.jl b/base/loading.jl
index 0a2d1b2ad9..68e41b6ce0 100644
--- a/base/loading.jl
+++ b/base/loading.jl
@@ -1741,6 +1741,7 @@ function get_uuid_name(project_toml::String, uuid::UUID)
end

function collect_preferences(project_toml::String, uuid::UUID)
+ @show project_toml uuid
# We'll return a list of dicts to be merged
dicts = Dict{String, Any}[]

@@ -1770,6 +1771,7 @@ function collect_preferences(project_toml::String, uuid::UUID)
end
end

+ @show dicts
return dicts
end

and then I get the following:

julia> Revise.track(Base)

(AdmissionSuite) pkg> st
Status `~/.julia/dev/AdmissionSuite/Project.toml`
[9b29e739] AdmissionTargets v0.1.0 `AdmissionTargets`
[e9cde6de] Admit v0.1.0 `Admit`
[19a2e0a6] AdmitConfiguration v0.1.0 `AdmitConfiguration`

julia> using AdmitConfiguration
project_toml = "/home/tim/.julia/environments/v1.8/Project.toml"
uuid = UUID("19a2e0a6-566e-4c00-8888-a0edb19dbca0")
project_toml = "/home/tim/.julia/dev/AdmissionSuite/Project.toml"
uuid = UUID("19a2e0a6-566e-4c00-8888-a0edb19dbca0")
dicts = Dict{String, Any}[]
project_toml = "/home/tim/.julia/environments/v1.8/Project.toml"
uuid = UUID("ae029012-a4dd-5104-9daa-d747884805df")
dicts = Dict{String, Any}[]
project_toml = "/home/tim/.julia/dev/AdmissionSuite/Project.toml"
uuid = UUID("ae029012-a4dd-5104-9daa-d747884805df")
project_toml = "/home/tim/.julia/environments/v1.8/Project.toml"
uuid = UUID("9a962f9c-6df0-11e9-0e5d-c546b8b5ee8a")
project_toml = "/home/tim/.julia/dev/AdmissionSuite/Project.toml"
uuid = UUID("9a962f9c-6df0-11e9-0e5d-c546b8b5ee8a")
project_toml = "/home/tim/.julia/environments/v1.8/Project.toml"
uuid = UUID("21216c6a-2e73-6563-6e65-726566657250")
project_toml = "/home/tim/.julia/dev/AdmissionSuite/Project.toml"
uuid = UUID("21216c6a-2e73-6563-6e65-726566657250")
project_toml = "/home/tim/.julia/environments/v1.8/Project.toml"
uuid = UUID("e1d29d7a-bbdc-5cf2-9ac0-f12de2c33e28")
project_toml = "/home/tim/.julia/dev/AdmissionSuite/Project.toml"
uuid = UUID("e1d29d7a-bbdc-5cf2-9ac0-f12de2c33e28")
here
project_toml = "/home/tim/.julia/environments/v1.8/Project.toml"
uuid = UUID("19a2e0a6-566e-4c00-8888-a0edb19dbca0")
project_toml = "/home/tim/.julia/dev/AdmissionSuite/Project.toml"
uuid = UUID("19a2e0a6-566e-4c00-8888-a0edb19dbca0")
dicts = Dict{String, Any}[]
project_toml = "/home/tim/.julia/environments/v1.8/Project.toml"
uuid = UUID("19a2e0a6-566e-4c00-8888-a0edb19dbca0")
project_toml = "/home/tim/.julia/dev/AdmissionSuite/Project.toml"
uuid = UUID("19a2e0a6-566e-4c00-8888-a0edb19dbca0")
dicts = Dict{String, Any}[]
project_toml = "/home/tim/.julia/environments/v1.8/Project.toml"
uuid = UUID("19a2e0a6-566e-4c00-8888-a0edb19dbca0")
project_toml = "/home/tim/.julia/dev/AdmissionSuite/Project.toml"
uuid = UUID("19a2e0a6-566e-4c00-8888-a0edb19dbca0")
dicts = Dict{String, Any}[]
project_toml = "/home/tim/.julia/environments/v1.8/Project.toml"
uuid = UUID("19a2e0a6-566e-4c00-8888-a0edb19dbca0")
project_toml = "/home/tim/.julia/dev/AdmissionSuite/Project.toml"
uuid = UUID("19a2e0a6-566e-4c00-8888-a0edb19dbca0")
dicts = Dict{String, Any}[]

indicating that the preferences are never loaded (here is printed at the entry point of AdmitConfiguration's attempt to load its own preferences). This is because Base.collect_preferences never receives the project_toml of the subdirectory.

There appear to be two potential solutions:

fix the path problem for subdirectories

store the configuration preferences in AdmissionSuite rather than the subdir package

Since the configuration written by AdmitConfiguration is intended for consumption by all the subpackages, there is a fair amount of sense in the latter. To attempt this, I added a name and uuid to AdmissionSuite/Project.toml, and then to AdmitConfiguration I added using AdmissionSuite and used set_preferences!(uuid, ...) where uuid is the uuid of AdmissionSuite. This initially seemed like it might work, but LocalPreferences.jl file got stored in my v1.6/environments despite the fact that AdmissionSuite was not in the [deps] of my global environment, and the preferences never ended up getting loaded. So, instead I modified the function that created the preferences file to use

tomlfile = joinpath(pkgdir(AdmissionSuite), "LocalPreferences.toml")
set_preferences!(tomlfile, "AdmissionSuite", prefs...) # this method is undocumented

and the __init__ for AdmitConfiguration to use

var1 = load_preference(AdmissionSuite, "var1")
...

This worked (the preferences get loaded with using AdmitConfiguration), but it is a narrow path to success and I only got there by reading the source code.

Issue 2:
While attempting to move FFTW over to using Preferences instead of environment variables/mutating its own state to record which backend is currently in use, I have discovered that Pkg generates a new project when running tests, and that new project drops unknown metadata. I believe the culprit is this section:

Pkg.jl/src/Operations.jl

Lines 1516 to 1534
in
0fe89b1

# collect relevant info from source

source_env = EnvCache(projectfile_path(source_path))

# collect regular dependencies

test_project.deps = source_env.project.deps

# collect test dependencies

for name in get(source_env.project.targets, target, String[])

uuid = get(source_env.project.extras, name, nothing)

if uuid === nothing

pkgerror("`$name` declared as a `$target` dependency, but no such entry in `extras`")

end

test_project.deps[name] = uuid

end

# collect compat entries

for (name, uuid) in test_project.deps

compat = get_compat_str(source_env.project, name)

compat === nothing && continue

set_compat(test_project, name, compat)

end

return test_project

Since users (and other packages) are storing pieces of information inside of Project.toml files, I think we need to preserve as much information in the Project.toml as possible.

For preferences in particular, we also need to copy any LocalPreferences.toml files to the temporary directory where the project is stored as well. I am currently working around this by generating an explicit test/Project.toml, but this is obviously sub-optimal.