Repository: SciML/SciMLSensitivity.jl
Pull request #1168: Feat: Handle Adjoints through Initialization
Author: login: DhairyaLGandhi
Created: 2025-03-06T12:28:09Z | Merged: 2025-04-24T22:57:15Z

## Checklist

- [x] Appropriate tests were added
- [x] Any code changes were done in a way that does not break public API
- [ ] All documentation related to code changes were updated
- [x] The new code follows the
  [contributor guidelines](https://github.com/SciML/.github/blob/master/CONTRIBUTING.md), in particular the [SciML Style Guide](https://github.com/SciML/SciMLStyle) and
  [COLPRAC](https://github.com/SciML/COLPRAC).
- [ ] Any new documentation only uses public API
  
## Additional context

MTK and SciML construct an initialization problem before starting the time stepping to ensure the starting values of the unknowns and parameters adhere to any constraints needed for the system. This PR adds handling for adjoint sensitivities of the NonlinearProblem, NonlinearSquaresProblem, SCCNonlinearProblem etc.

I am opening this to get some feedback regarding how we can accumulate gradients correctly. I have also included a test case for a DAE which I will update to use the values out of SciMLSensitivity.

Add any other context about the problem here.

Currently the gradients get calculated but don't get accumulated, we need to be able to update the gradients for the parameters. Since this is a manual dispatch, the usual graph building in AD is bypassed, and we need to handle this manually. Ideally, we should make it so the cfg itself includes the initialization so we would not have gotten incorrect gradients in the first place 😅  We are also forced to use a LinearProblem instead of  `\` because it cannot handle singular jacobians. 

cc @ChrisRackauckas 



Diff:
diff --git a/.github/workflows/CI.yml b/.github/workflows/CI.yml
index 622cb9b4b..545a1f06b 100644
--- a/.github/workflows/CI.yml
+++ b/.github/workflows/CI.yml
@@ -24,6 +24,7 @@ jobs:
           - Core5
           - Core6
           - Core7
+          - Core8
           - QA
           - SDE1
           - SDE2
diff --git a/Project.toml b/Project.toml
index 6cffa8c20..e044d9ffb 100644
--- a/Project.toml
+++ b/Project.toml
@@ -24,6 +24,7 @@ GPUArraysCore = "46192b85-c4d5-4398-a991-12ede77f4527"
 LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
 LinearSolve = "7ed4a6bd-45f5-4d41-b270-4a48e9bafcae"
 Markdown = "d6f4376e-aef5-505a-96c1-9c027394607a"
+OrdinaryDiffEqCore = "bbf590c4-e513-4bbe-9b18-05decba2e5d8"
 PreallocationTools = "d236fae5-4411-538c-8e31-a6e3d9e00b46"
 QuadGK = "1fd47b50-473d-5c70-9696-f719f8f3bcdc"
 Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
@@ -76,13 +77,15 @@ LinearAlgebra = "1.10"
 LinearSolve = "2, 3"
 Lux = "1"
 Markdown = "1.10"
-ModelingToolkit = "9.42"
+ModelingToolkit = "9.74"
+ModelingToolkitStandardLibrary = "2"
 Mooncake = "0.4.52"
 NLsolve = "4.5.1"
 NonlinearSolve = "3.0.1, 4"
 Optimization = "4"
 OptimizationOptimisers = "0.3"
 OrdinaryDiffEq = "6.81.1"
+OrdinaryDiffEqCore = "1"
 Pkg = "1.10"
 PreallocationTools = "0.4.4"
 QuadGK = "2.9.1"
@@ -117,6 +120,7 @@ DelayDiffEq = "bcd4f6db-9728-5f36-b5f7-82caef46ccdb"
 Distributed = "8ba89e20-285c-5b6f-9357-94700520ee1b"
 Lux = "b2108857-7c20-44ae-9111-449ecde12c47"
 ModelingToolkit = "961ee093-0014-501f-94e3-6117800e7a78"
+ModelingToolkitStandardLibrary = "16a59e39-deab-5bd0-87e4-056b12336739"
 Mooncake = "da2b9cff-9c12-43a0-ae48-6db2b0edb7d6"
 NLsolve = "2774e3e8-f4cf-5e23-947b-6d7e65073b56"
 NonlinearSolve = "8913a72c-1f9b-4ce2-8d82-65094dcecaec"
@@ -131,4 +135,4 @@ StochasticDiffEq = "789caeaf-c7a9-5a7d-9973-96adeb23e2a0"
 Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
 
 [targets]
-test = ["AlgebraicMultigrid", "Aqua", "Calculus", "ComponentArrays", "DelayDiffEq", "Distributed", "Lux", "ModelingToolkit", "Mooncake", "NLsolve", "NonlinearSolve", "Optimization", "OptimizationOptimisers", "OrdinaryDiffEq", "Pkg", "SafeTestsets", "SparseArrays", "SteadyStateDiffEq", "StochasticDiffEq", "Test"]
+test = ["AlgebraicMultigrid", "Aqua", "Calculus", "ComponentArrays", "DelayDiffEq", "Distributed", "Lux", "ModelingToolkit", "ModelingToolkitStandardLibrary", "Mooncake", "NLsolve", "NonlinearSolve", "Optimization", "OptimizationOptimisers", "OrdinaryDiffEq", "Pkg", "SafeTestsets", "SparseArrays", "SteadyStateDiffEq", "StochasticDiffEq", "Test"]
\ No newline at end of file
diff --git a/src/SciMLSensitivity.jl b/src/SciMLSensitivity.jl
index 46dbc62d9..a3569f384 100644
--- a/src/SciMLSensitivity.jl
+++ b/src/SciMLSensitivity.jl
@@ -37,16 +37,19 @@ using SciMLBase: SciMLBase, AbstractOverloadingSensitivityAlgorithm,
                  RODEFunction, RODEProblem, ReturnCode, SDEFunction,
                  SDEProblem, VectorContinuousCallback, deleteat!,
                  get_tmp_cache, has_adjoint, isinplace, reinit!, remake,
-                 solve, u_modified!, LinearAliasSpecifier
+                 solve, u_modified!, LinearAliasSpecifier, OverrideInit, CheckInit
+
+using OrdinaryDiffEqCore: OrdinaryDiffEqCore, BrownFullBasicInit, DefaultInit, default_nlsolve, has_autodiff
 
 # AD Backends
-using ChainRulesCore: unthunk, @thunk, NoTangent, @not_implemented, Tangent, ZeroTangent
+using ChainRulesCore: unthunk, @thunk, NoTangent, @not_implemented, Tangent, ZeroTangent, AbstractThunk
 using Enzyme: Enzyme
 using FiniteDiff: FiniteDiff
 using ForwardDiff: ForwardDiff
 using Tracker: Tracker, TrackedArray
 using ReverseDiff: ReverseDiff
 using Zygote: Zygote
+using SciMLBase.ConstructionBase
 
 # Std Libs
 using LinearAlgebra: LinearAlgebra, Diagonal, I, UniformScaling, adjoint, axpy!,
@@ -56,6 +59,8 @@ using Markdown: Markdown, @doc_str
 using Random: Random, rand!
 using Statistics: Statistics, mean
 
+using LinearAlgebra: diag
+
 abstract type SensitivityFunction end
 abstract type TransformedFunction end
 
diff --git a/src/adjoint_common.jl b/src/adjoint_common.jl
index 8b708f00c..01838ae70 100644
--- a/src/adjoint_common.jl
+++ b/src/adjoint_common.jl
@@ -78,7 +78,7 @@ function adjointdiffcache(g::G, sensealg, discrete, sol, dgdu::DG1, dgdp::DG2, f
     unwrappedf = unwrapped_f(f)
 
     numparams = p === nothing || p === SciMLBase.NullParameters() ? 0 : length(tunables)
-    numindvar = length(u0)
+    numindvar = isnothing(u0) ? nothing : length(u0)
     isautojacvec = get_jacvec(sensealg)
 
     issemiexplicitdae = false
@@ -106,7 +106,7 @@ function adjointdiffcache(g::G, sensealg, discrete, sol, dgdu::DG1, dgdp::DG2, f
         isempty(algevar_idxs) || (issemiexplicitdae = true)
     end
     if !issemiexplicitdae
-        diffvar_idxs = eachindex(u0)
+        diffvar_idxs = isnothing(u0) ? nothing : eachindex(u0)
         algevar_idxs = 1:0
     end
 
@@ -114,10 +114,14 @@ function adjointdiffcache(g::G, sensealg, discrete, sol, dgdu::DG1, dgdp::DG2, f
         J = nothing
     else
         if alg === nothing || SciMLBase.forwarddiffs_model_time(alg)
-            # 1 chunk is fine because it's only t
-            _J = similar(u0, numindvar, numindvar)
-            _J .= 0
-            J = dualcache(_J, ForwardDiff.pickchunksize(length(u0)))
+            if !isnothing(u0)
+                # 1 chunk is fine because it's only t
+                _J = similar(u0, numindvar, numindvar)
+                _J .= 0
+                J = dualcache(_J, ForwardDiff.pickchunksize(length(u0)))
+            else
+                J = nothing
+            end
         else
             J = similar(u0, numindvar, numindvar)
             J .= 0
@@ -133,8 +137,12 @@ function adjointdiffcache(g::G, sensealg, discrete, sol, dgdu::DG1, dgdp::DG2, f
                 dg_val[1] .= false
                 dg_val[2] .= false
             else
-                dg_val = similar(u0, numindvar) # number of funcs size
-                dg_val .= false
+                if !isnothing(u0)
+                    dg_val = similar(u0, numindvar) # number of funcs size
+                    dg_val .= false
+                else
+                    dg_val = nothing
+                end
             end
         else
             pgpu = UGradientWrapper(g, _t, p)
@@ -241,8 +249,12 @@ function adjointdiffcache(g::G, sensealg, discrete, sol, dgdu::DG1, dgdp::DG2, f
     pJ = if (quad || !(autojacvec isa Bool))
         nothing
     else
-        _pJ = similar(u0, numindvar, numparams)
-        _pJ .= false
+        if !isnothing(u0)
+            _pJ = similar(u0, numindvar, numparams)
+            _pJ .= false
+        else
+            _pJ = nothing
+        end
     end
 
     f_cache = isinplace ? deepcopy(u0) : nothing
@@ -379,11 +391,11 @@ function get_paramjac_config(autojacvec::ReverseDiffVJP, p, f, y, _p, _t;
         if !isRODE
             __p = p isa SciMLBase.NullParameters ? _p :
                   SciMLStructures.replace(Tunable(), p, _p)
-            tape = ReverseDiff.GradientTape((y, __p, [_t])) do u, p, t
+            tape = ReverseDiff.GradientTape((y, _p, [_t])) do u, p, t
                 du1 = (p !== nothing && p !== SciMLBase.NullParameters()) ?
                       similar(p, size(u)) : similar(u)
                 du1 .= false
-                f(du1, u, p, first(t))
+                f(du1, u, repack(p), first(t))
                 return vec(du1)
             end
         else
@@ -402,8 +414,8 @@ function get_paramjac_config(autojacvec::ReverseDiffVJP, p, f, y, _p, _t;
             # because hasportion(Tunable(), NullParameters) == false
             __p = p isa SciMLBase.NullParameters ? _p :
                   SciMLStructures.replace(Tunable(), p, _p)
-            tape = ReverseDiff.GradientTape((y, __p, [_t])) do u, p, t
-                vec(f(u, p, first(t)))
+            tape = ReverseDiff.GradientTape((y, _p, [_t])) do u, p, t
+                vec(f(u, repack(p), first(t)))
             end
         else
             tape = ReverseDiff.GradientTape((y, _p, [_t], _W)) do u, p, t, W
diff --git a/src/concrete_solve.jl b/src/concrete_solve.jl
index a0bb665e5..ba6e0a1bc 100644
--- a/src/concrete_solve.jl
+++ b/src/concrete_solve.jl
@@ -46,15 +46,16 @@ function inplace_vjp(prob, u0, p, verbose, repack)
 
     vjp = try
         f = unwrapped_f(prob.f)
+        tspan_ = prob isa AbstractNonlinearProblem ? nothing : [prob.tspan[1]]
         if p === nothing || p isa SciMLBase.NullParameters
-            ReverseDiff.GradientTape((copy(u0), [prob.tspan[1]])) do u, t
+            ReverseDiff.GradientTape((copy(u0), tspan_)) do u, t
                 du1 = similar(u, size(u))
                 du1 .= 0
                 f(du1, u, p, first(t))
                 return vec(du1)
             end
         else
-            ReverseDiff.GradientTape((copy(u0), p, [prob.tspan[1]])) do u, p, t
+            ReverseDiff.GradientTape((copy(u0), p, tspan_)) do u, p, t
                 du1 = similar(u, size(u))
                 du1 .= 0
                 f(du1, u, repack(p), first(t))
@@ -299,6 +300,7 @@ function DiffEqBase._concrete_solve_adjoint(
         tunables, repack = Functors.functor(p)
     end
 
+    u0 = state_values(prob) === nothing ? Float64[] : u0
     default_sensealg = automatic_sensealg_choice(prob, u0, tunables, verbose, repack)
     DiffEqBase._concrete_solve_adjoint(prob, alg, default_sensealg, u0, p,
         originator::SciMLBase.ADOriginator, args...; verbose,
@@ -371,6 +373,7 @@ function DiffEqBase._concrete_solve_adjoint(
         args...; save_start = true, save_end = true,
         saveat = eltype(prob.tspan)[],
         save_idxs = nothing,
+        initializealg_default = SciMLBase.OverrideInit(; abstol = 1e-6, reltol = 1e-3),
         kwargs...)
     if !(sensealg isa GaussAdjoint) &&
        !(p isa Union{Nothing, SciMLBase.NullParameters, AbstractArray}) ||
@@ -412,16 +415,61 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+    kwargs_init = kwargs_adj[Base.diff_names(Base._nt_names(kwargs_adj), (:initializealg,))]
+
+    if haskey(kwargs, :initializealg) || haskey(prob.kwargs, :initializealg)
+        initializealg = haskey(kwargs, :initializealg) ? kwargs[:initializealg] : prob.kwargs[:initializealg]
+    else
+        initializealg = DefaultInit()
+    end
+
+    default_inits = Union{OverrideInit, Nothing, DefaultInit}
+    igs, new_u0, new_p, new_initializealg = if (SciMLBase.has_initialization_data(_prob.f) && initializealg isa default_inits)
+        local new_u0
+        local new_p
+        initializeprob = prob.f.initialization_data.initializeprob
+        iu0 = state_values(initializeprob)
+        isAD = if iu0 === nothing
+            AutoForwardDiff
+        elseif has_autodiff(alg)
+            OrdinaryDiffEqCore.alg_autodiff(alg) isa AutoForwardDiff
+        else
+            true 
+        end
+        nlsolve_alg = default_nlsolve(nothing, Val(isinplace(_prob)), iu0, initializeprob, isAD)
+        initializealg = initializealg isa Union{Nothing, DefaultInit} ? initializealg_default : initializealg
+
+        iy, back = Zygote.pullback(tunables) do tunables
+            new_prob = remake(_prob, p = repack(tunables))
+            new_u0, new_p, _ = SciMLBase.get_initial_values(new_prob, new_prob, new_prob.f, initializealg, Val(isinplace(new_prob));
+                                                            sensealg = SteadyStateAdjoint(autojacvec = sensealg.autojacvec),
+                                                            nlsolve_alg,
+                                                            kwargs_init...)
+            new_tunables, _, _ = SciMLStructures.canonicalize(SciMLStructures.Tunable(), new_p)
+            if SciMLBase.initialization_status(_prob) == SciMLBase.OVERDETERMINED
+                sum(new_tunables)
+            else
+                sum(new_u0) + sum(new_tunables)
+            end
+        end
+        igs = back(one(iy))[1] .- one(eltype(tunables))
+
+        igs, new_u0, new_p, SciMLBase.NoInit()
+    else
+        nothing, u0, p, initializealg
+    end
+    _prob = remake(_prob, u0 = new_u0, p = new_p)
+
     if sensealg isa BacksolveAdjoint
-        sol = solve(_prob, alg, args...; save_noise = true,
+        sol = solve(_prob, alg, args...; initializealg = new_initializealg, save_noise = true,
             save_start = save_start, save_end = save_end,
             saveat = saveat, kwargs_fwd...)
     elseif ischeckpointing(sensealg)
-        sol = solve(_prob, alg, args...; save_noise = true,
+        sol = solve(_prob, alg, args...; initializealg = new_initializealg, save_noise = true,
             save_start = true, save_end = true,
             saveat = saveat, kwargs_fwd...)
     else
-        sol = solve(_prob, alg, args...; save_noise = true, save_start = true,
+        sol = solve(_prob, alg, args...; initializealg = new_initializealg, save_noise = true, save_start = true,
             save_end = true, kwargs_fwd...)
     end
 
@@ -491,6 +539,7 @@ function DiffEqBase._concrete_solve_adjoint(
     _save_idxs = save_idxs === nothing ? Colon() : save_idxs
 
     function adjoint_sensitivity_backpass(Δ)
+        Δ = Δ isa AbstractThunk ? unthunk(Δ) : Δ
         function df_iip(_out, u, p, t, i)
             outtype = _out isa SubArray ?
                       ArrayInterface.parameterless_type(_out.parent) :
@@ -628,13 +677,13 @@ function DiffEqBase._concrete_solve_adjoint(
                 dgdu_discrete = df_iip,
                 sensealg = sensealg,
                 callback = cb2,
-                kwargs_adj...)
+                kwargs_init...)
         else
             du0, dp = adjoint_sensitivities(sol, alg, args...; t = ts,
                 dgdu_discrete = df_oop,
                 sensealg = sensealg,
                 callback = cb2,
-                kwargs_adj...)
+                kwargs_init...)
         end
 
         du0 = reshape(du0, size(u0))
@@ -642,6 +691,8 @@ function DiffEqBase._concrete_solve_adjoint(
         dp = p === nothing || p === DiffEqBase.NullParameters() ? nothing :
              dp isa AbstractArray ? reshape(dp', size(tunables)) : dp
 
+        dp = Zygote.accum(dp, igs)
+
         _, repack_adjoint = if p === nothing || p === DiffEqBase.NullParameters() ||
                                !isscimlstructure(p)
             nothing, x -> (x,)
@@ -1679,6 +1730,7 @@ function DiffEqBase._concrete_solve_adjoint(
         u0, p, originator::SciMLBase.ADOriginator,
         args...; save_idxs = nothing, kwargs...)
     _prob = remake(prob, u0 = u0, p = p)
+
     sol = solve(_prob, alg, args...; kwargs...)
     _save_idxs = save_idxs === nothing ? Colon() : save_idxs
 
@@ -1688,7 +1740,20 @@ function DiffEqBase._concrete_solve_adjoint(
         out = SciMLBase.sensitivity_solution(sol, sol[_save_idxs])
     end
 
+    _, repack_adjoint = if isscimlstructure(p)
+        Zygote.pullback(p) do p
+            t, _, _ = canonicalize(Tunable(), p)
+            t
+        end
+    elseif isfunctor(p)
+        ps, re = Functors.functor(p)
+        ps, x -> (re(x),)
+    else
+        nothing, x -> (x,)
+    end
+
     function steadystatebackpass(Δ)
+        Δ = Δ isa AbstractThunk ? unthunk(Δ) : Δ
         # Δ = dg/dx or diffcache.dg_val
         # del g/del p = 0
         function df(_out, u, p, t, i)
@@ -1696,18 +1761,53 @@ function DiffEqBase._concrete_solve_adjoint(
                 _out[_save_idxs] = Δ[_save_idxs]
             elseif Δ isa Number
                 @. _out[_save_idxs] = Δ
-            else
+            elseif Δ isa AbstractArray{<:AbstractArray} || Δ isa AbstractVectorOfArray || Δ isa AbstractArray
                 @. _out[_save_idxs] = Δ[_save_idxs]
+            elseif isnothing(_out)
+                _out
+            else
+                @. _out[_save_idxs] = Δ.u[_save_idxs]
+            end
+        end
+        dp = adjoint_sensitivities(sol, alg; sensealg = sensealg, dgdu = df, initializealg = BrownFullBasicInit())
+
+        dp, Δtunables = if Δ isa AbstractArray || Δ isa Number
+            # if Δ isa AbstractArray, the gradients correspond to `u`
+            # this is something that needs changing in the future, but
+            # this is the applicable till the movement to structuaral
+            # tangents is completed
+            dp, Δtunables = if isscimlstructure(dp)
+                dp, _, _ = canonicalize(Tunable(), dp)
+                dp, nothing
+            elseif isfunctor(dp)
+                dp, _ = Functors.functor(dp)
+                dp, nothing
+            else
+                dp, nothing
+            end
+        else
+            dp, Δtunables = if isscimlstructure(p)
+                Δp = setproperties(dp, to_nt(Δ.prob.p))
+                Δtunables, _, _ = canonicalize(Tunable(), Δp)
+                dp, _, _ = canonicalize(Tunable(), dp)
+                dp, Δtunables
+            elseif isfunctor(p)
+                dp, _ = Functors.functor(dp)
+                Δtunables, _ = Functors.functor(Δ.prob.p)
+                dp, Δtunables
+            else
+                dp, Δ.prob.p
             end
         end
-        dp = adjoint_sensitivities(sol, alg; sensealg = sensealg, dgdu = df)
+
+        dp = Zygote.accum(dp, (isnothing(Δtunables) || isempty(Δtunables)) ? nothing : Δtunables)
 
         if originator isa SciMLBase.TrackerOriginator ||
            originator isa SciMLBase.ReverseDiffOriginator
-            (NoTangent(), NoTangent(), NoTangent(), dp, NoTangent(),
+            (NoTangent(), NoTangent(), NoTangent(), repack_adjoint(dp)[1], NoTangent(),
                 ntuple(_ -> NoTangent(), length(args))...)
         else
-            (NoTangent(), NoTangent(), NoTangent(), NoTangent(), dp, NoTangent(),
+            (NoTangent(), NoTangent(), NoTangent(), NoTangent(), repack_adjoint(dp)[1], NoTangent(),
                 ntuple(_ -> NoTangent(), length(args))...)
         end
     end
diff --git a/src/derivative_wrappers.jl b/src/derivative_wrappers.jl
index 9c3de2218..6c0d5a1cc 100644
--- a/src/derivative_wrappers.jl
+++ b/src/derivative_wrappers.jl
@@ -144,6 +144,13 @@ function jacobian(f, x::AbstractArray{<:Number},
     return J
 end
 
+function jacobian!(J::Nothing, f, x::AbstractArray{<:Number},
+    fx::Union{Nothing, AbstractArray{<:Number}},
+    alg::AbstractOverloadingSensitivityAlgorithm, jac_config::Nothing)
+    @assert isempty(x)
+    J
+end
+jacobian!(J::PreallocationTools.DiffCache, x::SciMLBase.UJacobianWrapper, args...) = jacobian!(J.du, x, args...)
 function jacobian!(J::AbstractMatrix{<:Number}, f, x::AbstractArray{<:Number},
         fx::Union{Nothing, AbstractArray{<:Number}},
         alg::AbstractOverloadingSensitivityAlgorithm, jac_config)
@@ -456,9 +463,10 @@ function _vecjacobian!(dλ, y, λ, p, t, S::TS, isautojacvec::ReverseDiffVJP, dg
     elseif inplace_sensitivity(S)
         _y = eltype(y) === eltype(λ) ? y : convert.(promote_type(eltype(y), eltype(λ)), y)
         if W === nothing
-            tape = ReverseDiff.GradientTape((_y, _p, [t])) do u, p, t
+            _tunables, _repack, _ = canonicalize(Tunable(), _p)
+            tape = ReverseDiff.GradientTape((_y, _tunables, [t])) do u, p, t
                 du1 = similar(u, size(u))
-                f(du1, u, p, first(t))
+                f(du1, u, _repack(p), first(t))
                 return vec(du1)
             end
         else
@@ -474,8 +482,9 @@ function _vecjacobian!(dλ, y, λ, p, t, S::TS, isautojacvec::ReverseDiffVJP, dg
     else
         _y = eltype(y) === eltype(λ) ? y : convert.(promote_type(eltype(y), eltype(λ)), y)
         if W === nothing
-            tape = ReverseDiff.GradientTape((_y, _p, [t])) do u, p, t
-                vec(f(u, p, first(t)))
+            _tunables, _repack, _ = canonicalize(Tunable(), _p)
+            tape = ReverseDiff.GradientTape((_y, _tunables, [t])) do u, p, t
+                vec(f(u, _repack(p), first(t)))
             end
         else
             _W = eltype(W) === eltype(λ) ? W :
@@ -1047,6 +1056,7 @@ function accumulate_cost(dλ, y, p, t, S::TS,
     return dλ, dgrad
 end
 
+build_jac_config(alg, uf, u::Nothing) = nothing
 function build_jac_config(alg, uf, u)
     if alg_autodiff(alg)
         jac_config = ForwardDiff.JacobianConfig(uf, u, u,
@@ -1068,9 +1078,10 @@ end
 
 function build_param_jac_config(alg, pf, u, p)
     if alg_autodiff(alg)
-        jac_config = ForwardDiff.JacobianConfig(pf, u, p,
+        tunables, repack, aliases = canonicalize(Tunable(), p)
+        jac_config = ForwardDiff.JacobianConfig(pf, u, tunables,
             ForwardDiff.Chunk{
-                determine_chunksize(p,
+                determine_chunksize(tunables,
                 alg)}())
     else
         if diff_type(alg) != Val{:complex}
diff --git a/src/parameters_handling.jl b/src/parameters_handling.jl
index e30ddb232..ed3ae3903 100644
--- a/src/parameters_handling.jl
+++ b/src/parameters_handling.jl
@@ -15,6 +15,9 @@ end
 recursive_copyto!(y::T, x::T) where {T} = fmap(recursive_copyto!, y, x)
 recursive_copyto!(y, ::Nothing) = y
 recursive_copyto!(::Nothing, ::Nothing) = nothing
+function recursive_copyto!(y::T, x::NamedTuple) where T
+    fmap(recursive_copyto!, y, x)
+end
 
 """
     neg!(x)
@@ -61,14 +64,14 @@ recursive_add!(::Nothing, ::Nothing) = nothing
 
 `similar(λ, size(x))` for generic `x`. This is used to handle non-array parameters!
 """
-allocate_vjp(λ::AbstractArray, x::AbstractArray) = similar(λ, size(x))
+allocate_vjp(λ::AbstractArray{T}, x::AbstractArray) where T = fill!(similar(λ, size(x)), zero(T))
 allocate_vjp(λ::AbstractArray, x::Tuple) = allocate_vjp.((λ,), x)
 function allocate_vjp(λ::AbstractArray, x::NamedTuple{F}) where {F}
     NamedTuple{F}(allocate_vjp.((λ,), values(x)))
 end
 allocate_vjp(λ::AbstractArray, x) = fmap(Base.Fix1(allocate_vjp, λ), x)
 
-allocate_vjp(x::AbstractArray) = similar(x)
+allocate_vjp(x::AbstractArray) = zero(x) # similar(x)
 allocate_vjp(x::Tuple) = allocate_vjp.(x)
 allocate_vjp(x::NamedTuple{F}) where {F} = NamedTuple{F}(allocate_vjp.(values(x)))
 allocate_vjp(x) = fmap(allocate_vjp, x)
diff --git a/src/quadrature_adjoint.jl b/src/quadrature_adjoint.jl
index 91a45670e..692bd3cb8 100644
--- a/src/quadrature_adjoint.jl
+++ b/src/quadrature_adjoint.jl
@@ -521,4 +521,4 @@ function _update_integrand_and_dgrad(res, sensealg::QuadratureAdjoint, cb, integ
     vecjacobian!(dλ, integrand.y, dλ, integrand.p, t, fakeS; dgrad = dgrad)
     res .-= dgrad
     return integrand
-end
+end
\ No newline at end of file
diff --git a/src/steadystate_adjoint.jl b/src/steadystate_adjoint.jl
index 45d690fed..6f8aa6d7e 100644
--- a/src/steadystate_adjoint.jl
+++ b/src/steadystate_adjoint.jl
@@ -41,7 +41,7 @@ end
         error("Either `dgdu`, `dgdp`, or `g` must be specified.")
 
     needs_jac = ifelse(has_adjoint(f), false,
-        ifelse(sensealg.linsolve === nothing, length(u0) ≤ 50,
+        ifelse(sensealg.linsolve === nothing, isnothing(u0) || length(u0) ≤ 50,
             __needs_concrete_A(sensealg.linsolve)))
 
     p === SciMLBase.NullParameters() &&
@@ -50,7 +50,6 @@ end
     sense = SteadyStateAdjointSensitivityFunction(g, sensealg, alg, sol, dgdu, dgdp,
         f, f.colorvec, needs_jac)
     (; diffcache, y, sol, λ, vjp, linsolve) = sense
-
     if needs_jac
         if SciMLBase.has_jac(f)
             f.jac(diffcache.J, y, p, nothing)
@@ -84,7 +83,7 @@ end
         end
     end
 
-    if !needs_jac
+    if !needs_jac && !isempty(y)
         # Current SciMLJacobianOperators requires specifying the problem as a NonlinearProblem
         usize = size(y)
         if SciMLBase.isinplace(f)
@@ -102,18 +101,17 @@ end
         solve(linear_problem, linsolve; alias = LinearAliasSpecifier(alias_A = true),
             sensealg.linsolve_kwargs...)
     else
-        if linsolve === nothing && isempty(sensealg.linsolve_kwargs)
-            # For the default case use `\` to avoid any form of unnecessary cache allocation
-            vec(λ) .= diffcache.J' \ vec(dgdu_val)
-        else
-            linear_problem = LinearProblem(diffcache.J', vec(dgdu_val'); u0 = vec(λ))
-            solve(linear_problem, linsolve; alias = LinearAliasSpecifier(alias_A = true),
-                sensealg.linsolve_kwargs...) # u is vec(λ)
+        if !isempty(y)
+            J_ = diffcache.J isa PreallocationTools.DiffCache ? diffcache.J.du' : diffcache.J'
+            linear_problem = LinearProblem(J_, vec(dgdu_val'); u0 = vec(λ))
+            solve(linear_problem, linsolve; alias = LinearAliasSpecifier(alias_A = true), sensealg.linsolve_kwargs...) # u is vec(λ)
         end
     end
 
     try
-        vecjacobian!(vec(dgdu_val), y, λ, p, nothing, sense; dgrad = vjp, dy = nothing)
+        if !isempty(y)
+            vecjacobian!(vec(dgdu_val), y, λ, p, nothing, sense; dgrad = vjp, dy = nothing)
+        end
     catch e
         if sense.sensealg.autojacvec === nothing
             @warn "Automatic AD choice of autojacvec failed in nonlinear solve adjoint, failing back to ODE adjoint + numerical vjp"
diff --git a/src/utils.jl b/src/utils.jl
index 37b4e7217..71e278683 100644
--- a/src/utils.jl
+++ b/src/utils.jl
@@ -1,3 +1,7 @@
 function isfunctor(x)
     !isempty(Functors.children(x))
 end
+
+function to_nt(s::T) where T
+    NamedTuple{propertynames(s)}(map(x -> getproperty(s, x), propertynames(s)))
+end
diff --git a/test/adjoint.jl b/test/adjoint.jl
index 4df7dad9c..abb5cf7a5 100644
--- a/test/adjoint.jl
+++ b/test/adjoint.jl
@@ -972,9 +972,9 @@ for iip in [true, false]
     f = ODEFunction{iip}(rober, mass_matrix = M)
     p = [0.04, 3e7, 1e4]
 
-    prob_singular_mm = ODEProblem(f, [1.0, 0.0, 0.0], (0.0, 100), p)
-    sol_singular_mm = solve(prob_singular_mm, Rodas4(autodiff = false),
-        reltol = 1e-12, abstol = 1e-12)
+    prob_singular_mm = ODEProblem(f, [1.0, 0.0, 1.0], (0.0, 100), p)
+    sol_singular_mm = solve(prob_singular_mm, FBDF(autodiff = false),
+        reltol = 1e-12, abstol = 1e-12, initializealg = BrownFullBasicInit())
     ts = [50, sol_singular_mm.t[end]]
     dg_singular(out, u, p, t, i) = (fill!(out, 0); out[end] = 1)
     _, res = adjoint_sensitivities(sol_singular_mm, alg, t = ts,
diff --git a/test/desauty_dae_mwe.jl b/test/desauty_dae_mwe.jl
new file mode 100644
index 000000000..68baa0277
--- /dev/null
+++ b/test/desauty_dae_mwe.jl
@@ -0,0 +1,56 @@
+using ModelingToolkit, OrdinaryDiffEq
+using ModelingToolkitStandardLibrary.Electrical
+using ModelingToolkitStandardLibrary.Blocks: Sine
+using NonlinearSolve
+import SciMLStructures as SS
+import SciMLSensitivity
+using SymbolicIndexingInterface
+import ModelingToolkit as MTK
+using Zygote
+
+function create_model(; C₁ = 3e-5, C₂ = 1e-6)
+    @variables t
+    @named resistor1 = Resistor(R = 5.0)
+    @named resistor2 = Resistor(R = 2.0)
+    @named capacitor1 = Capacitor(C = C₁)
+    @named capacitor2 = Capacitor(C = C₂)
+    @named source = Voltage()
+    @named input_signal = Sine(frequency = 100.0)
+    @named ground = Ground()
+    @named ampermeter = CurrentSensor()
+
+    eqs = [connect(input_signal.output, source.V)
+        connect(source.p, capacitor1.n, capacitor2.n)
+        connect(source.n, resistor1.p, resistor2.p, ground.g)
+        connect(resistor1.n, capacitor1.p, ampermeter.n)
+        connect(resistor2.n, capacitor2.p, ampermeter.p)]
+
+    @named circuit_model = ODESystem(eqs, t,
+        systems = [
+            resistor1, resistor2, capacitor1, capacitor2,
+            source, input_signal, ground, ampermeter,
+        ])
+end
+
+desauty_model = create_model()
+sys = structural_simplify(desauty_model)
+
+
+prob = ODEProblem(sys, [], (0.0, 0.1), guesses = [sys.resistor1.v => 1.])
+iprob = prob.f.initialization_data.initializeprob
+isys = iprob.f.sys
+
+tunables, repack, aliases = SS.canonicalize(SS.Tunable(), parameter_values(iprob))
+
+linsolve = LinearSolve.DefaultLinearSolver(LinearSolve.DefaultAlgorithmChoice.QRFactorization)
+sensealg = SciMLSensitivity.SteadyStateAdjoint(autojacvec = SciMLSensitivity.ZygoteVJP())
+# sensealg = SciMLSensitivity.SteadyStateAdjoint(autojacvec = SciMLSensitivity.ZygoteVJP(), linsolve = linsolve)
+igs, = Zygote.gradient(tunables) do p
+    iprob2 = remake(iprob, p = repack(p))
+    sol = solve(iprob2,
+                sensealg = sensealg
+    )
+    sum(Array(sol))
+end
+
+@test !iszero(sum(igs))
diff --git a/test/mtk.jl b/test/mtk.jl
index b42c14883..1f2e3d237 100644
--- a/test/mtk.jl
+++ b/test/mtk.jl
@@ -1,10 +1,12 @@
 using ModelingToolkit, OrdinaryDiffEq
 using ModelingToolkit: t_nounits as t, D_nounits as D
 using OrdinaryDiffEq
+using OrdinaryDiffEqCore
 using SciMLSensitivity
 using ForwardDiff
 using Zygote
 using Statistics
+using Test
 
 @parameters σ ρ β A[1:3]
 @variables x(t) y(t) z(t) w(t) w2(t)
@@ -32,8 +34,119 @@ prob = ODEProblem(sys, u0, tspan, p, jac = true)
 sol = solve(prob, Tsit5())
 mtkparams = SciMLSensitivity.parameter_values(sol)
 
-gt = rand(5501)
+gt = rand(length(sol.u))
 dmtk, = Zygote.gradient(mtkparams) do p
-    new_sol = solve(prob, Rosenbrock23(), p = p)
+    new_sol = solve(prob, Tsit5(), p = p)
+    Zygote.ChainRules.ChainRulesCore.ignore_derivatives() do
+        @test all(isapprox.(new_sol[x + y + z + 2 * β - w], 0, atol = 1e-12))
+    end
     mean(abs.(new_sol[sys.x] .- gt))
 end
+
+# Force DAE handling for Initialization
+
+eqs = [D(D(x)) ~ σ * (y - x),
+    D(y) ~ x * (ρ - z) - y,
+    D(z) ~ x * y - β * z,
+    w ~ x + y + z + 2 * β,
+    0 ~ x^2 + y^2 - w2^2
+]
+
+@mtkbuild sys = ODESystem(eqs, t)
+
+u0_incorrect = [D(x) => 2.0,
+    x => 1.0,
+    y => 0.0,
+    z => 0.0]
+
+p = [σ => 28.0,
+    ρ => 10.0,
+    β => 8 / 3]
+
+tspan = (0.0, 100.0)
+
+# Check that the gradients for the `solve` are the same both for an initialization
+# (for the algebraic variables) initialized poorly (therefore needs correction with BrownBasicInit)
+# and with the initialization corrected to satisfy the algebraic equation
+prob_incorrectu0 = ODEProblem(sys, u0_incorrect, tspan, p, jac = true, guesses = [w2 => 0.0])
+mtkparams_incorrectu0 = SciMLSensitivity.parameter_values(prob_incorrectu0)
+
+u0_timedep = [D(x) => 2.0,
+    x => 1.0,
+    y => t,
+    z => 0.0,
+    w2 => 0.0,]
+# this ensures that `y => t` is not applied in the adjoint equation
+# If the MTK init is called for the reverse, then `y0` in the backwards
+# pass will be extremely far off and cause an incorrect gradient
+prob_timedepu0 = ODEProblem(sys, u0_timedep, tspan, p, jac = true, guesses = [w2 => 0.0])
+mtkparams_timedepu0 = SciMLSensitivity.parameter_values(prob_incorrectu0)
+
+u0_correct = [D(x) => 2.0,
+    x => 1.0,
+    y => 0.0,
+    z => 0.0,
+    w2 => -1.0,]
+prob_correctu0 = ODEProblem(sys, u0_correct, tspan, p, jac = true, guesses = [w2 => -1.0])
+mtkparams_correctu0 = SciMLSensitivity.parameter_values(prob_correctu0)
+prob_correctu0.u0[5] = -1.0
+
+u0_overdetermined = [D(x) => 2.0,
+    x => 1.0,
+    y => 0.0,
+    z => 0.0,
+    w2 => -1.0,]
+prob_overdetermined = ODEProblem(sys, u0_overdetermined, tspan, p, jac = true)
+mtkparams_overdetermined = SciMLSensitivity.parameter_values(prob_overdetermined)
+
+sensealg = GaussAdjoint(; autojacvec = SciMLSensitivity.ZygoteVJP())
+
+setups = [
+          (prob_correctu0, mtkparams_correctu0, CheckInit()), # Source of truth first
+    
+          (prob_incorrectu0, mtkparams_incorrectu0, BrownFullBasicInit()),
+          (prob_incorrectu0, mtkparams_incorrectu0, OrdinaryDiffEqCore.DefaultInit()),
+          (prob_incorrectu0, mtkparams_incorrectu0, nothing), 
+
+          (prob_timedepu0, mtkparams_timedepu0, BrownFullBasicInit()),
+          (prob_timedepu0, mtkparams_timedepu0, OrdinaryDiffEqCore.DefaultInit()),
+          (prob_timedepu0, mtkparams_timedepu0, nothing), 
+
+          (prob_correctu0, mtkparams_correctu0, BrownFullBasicInit()),
+          (prob_correctu0, mtkparams_correctu0, OrdinaryDiffEqCore.DefaultInit()),
+          
+          (prob_correctu0, mtkparams_correctu0, NoInit()), 
+          (prob_correctu0, mtkparams_correctu0, nothing),
+
+          (prob_overdetermined, mtkparams_overdetermined, BrownFullBasicInit()),
+          (prob_overdetermined, mtkparams_overdetermined, OrdinaryDiffEq.OrdinaryDiffEqCore.DefaultInit()),
+
+          (prob_overdetermined, mtkparams_overdetermined, NoInit()),
+          (prob_overdetermined, mtkparams_overdetermined, nothing),
+]
+
+grads = map(setups) do setup
+    prob, ps, init = setup
+    @show init
+    u0 = prob.u0
+    Zygote.gradient(u0, ps) do u0,p
+        if init === nothing
+            new_sol = solve(prob, Rodas5P(); u0 = u0, p = ps, sensealg, abstol = 1e-6, reltol = 1e-3)
+        else
+            new_sol = solve(prob, Rodas5P(); u0 = u0, p = ps, initializealg = init, sensealg, abstol = 1e-6, reltol = 1e-3)
+        end
+        gt = Zygote.ChainRules.ChainRulesCore.ignore_derivatives() do
+            @test new_sol.retcode == SciMLBase.ReturnCode.Success
+            # Test that beginning of forward pass init'd correctly
+            @test all(isapprox.(new_sol[x + y + z + 2 * β - w], 0, atol = 1e-12))
+            @test all(isapprox.(new_sol[x^2 + y^2 - w2^2], 0, atol = 1e-5, rtol = 1e0))
+            zeros(size(new_sol, 2))
+        end
+        mean(abs.(new_sol[sys.x] .- gt))
+    end
+end
+
+u0grads = getindex.(grads,1)
+pgrads = getproperty.(getindex.(grads, 2), (:tunable,))
+@test all(x ≈ u0grads[1] for x in grads)
+@test all(x ≈ pgrads[1] for x in grads)
diff --git a/test/parameter_handling.jl b/test/parameter_handling.jl
index acd7cd86b..3e2712df8 100644
--- a/test/parameter_handling.jl
+++ b/test/parameter_handling.jl
@@ -1,21 +1,23 @@
 using SciMLSensitivity, Lux, Random, Zygote, NonlinearSolve, OrdinaryDiffEq, Test
+using ComponentArrays
 
 @info "Testing Nonlinear Solve Adjoint with Nested Parameters"
 
 const model_nls = Chain(Dense(2 => 2, tanh), Dense(2 => 2))
 ps, st = Lux.setup(Random.default_rng(), model_nls)
 const st_nls = st
+ca = ComponentArray(ps)
 
 x = ones(Float32, 2, 3)
 
 nlprob(u, p) = first(model_nls(u, p, st_nls)) .- u
 
-prob = NonlinearProblem(nlprob, zeros(2, 3), ps)
+prob = NonlinearProblem(nlprob, zeros(2, 3), ca)
 
 @test_nowarn solve(prob, NewtonRaphson())
 
-gs = only(Zygote.gradient(ps) do ps
-    prob = NonlinearProblem(nlprob, zero.(x), ps)
+gs = only(Zygote.gradient(ca) do ca
+    prob = NonlinearProblem(nlprob, zero.(x), ca)
     sol = solve(prob, NewtonRaphson())
     return sum(sol.u)
 end)
@@ -30,17 +32,18 @@ end)
 const model_ga = Chain(Dense(2 => 2, tanh), Dense(2 => 2))
 ps, st = Lux.setup(Random.default_rng(), model_ga)
 const st_ga = st
+ca = ComponentArray(ps)
 
 x = ones(Float32, 2, 3)
 
 odeprob(u, p, t) = first(model_ga(u, p, st_ga))
 
-prob = ODEProblem(odeprob, ones(2, 3), (0.0f0, 1.0f0), ps)
+prob = ODEProblem(odeprob, ones(2, 3), (0.0f0, 1.0f0), ca)
 
 @test_nowarn solve(prob, Tsit5())
 
-gs = only(Zygote.gradient(ps) do ps
-    prob = ODEProblem(odeprob, ones(2, 3), (0.0f0, 1.0f0), ps)
+gs = only(Zygote.gradient(ca) do ca
+    prob = ODEProblem(odeprob, ones(2, 3), (0.0f0, 1.0f0), ca)
     sol = solve(prob, Tsit5(); sensealg = GaussAdjoint(; autojacvec = ZygoteVJP()))
     return sum(last(sol.u))
 end)
diff --git a/test/parameter_initialization.jl b/test/parameter_initialization.jl
new file mode 100644
index 000000000..4280abcbd
--- /dev/null
+++ b/test/parameter_initialization.jl
@@ -0,0 +1,99 @@
+using ModelingToolkit
+using ModelingToolkit: t_nounits as t, D_nounits as D
+using SymbolicIndexingInterface
+using SciMLStructures
+import SymbolicIndexingInterface as SII
+import SciMLStructures as SS
+import ModelingToolkit as MTK
+using SciMLSensitivity
+using OrdinaryDiffEq
+using Zygote
+using Test
+
+@parameters σ ρ β
+@variables x(t) y(t) z(t) w(t)
+
+eqs = [D(D(x)) ~ σ * (y - x), 
+    D(y) ~ x * (ρ - z) - y,
+    D(z) ~ x * y - β * z,
+    w ~ x + y + z + 2 * β]
+
+@mtkbuild sys = ODESystem(eqs, t, defaults = [ρ => missing,], guesses = [ρ => 10.0])
+
+u0 = [D(x) => 2.0,
+    x => 1.0,
+    y => 0.0,
+    z => 0.0]
+
+p = [σ => 28.0,
+    ρ => 10.0,
+    β => 8 / 3]
+
+tspan = (0.0, 100.0)
+prob = ODEProblem(sys, u0, tspan, p, jac = true)
+sol = solve(prob, Tsit5())
+
+tunables, repack, _ = SS.canonicalize(SS.Tunable(), parameter_values(prob))
+
+@testset "Adjoint through Parameter Initialization" begin
+    @testset "Forward Mode" begin
+        gs_fwd, = Zygote.gradient(tunables) do tunables
+            new_prob = remake(prob; p = repack(tunables))
+            iprob = new_prob.f.initialization_data.initializeprob
+            isol = solve(iprob)
+            isol[w]
+        end
+        @test any(!iszero, gs_fwd)
+    end
+
+    @testset "Reverse Mode" begin
+        sensealg = SciMLSensitivity.SteadyStateAdjoint(autojacvec = SciMLSensitivity.ReverseDiffVJP())
+        gs_reverse, = Zygote.gradient(tunables) do tunables
+            new_prob = remake(prob; p = repack(tunables))
+            iprob = new_prob.f.initialization_data.initializeprob
+            isol = solve(iprob; sensealg)
+            isol[w]
+        end
+        @test any(!iszero, gs_reverse)
+
+        sensealg = SciMLSensitivity.SteadyStateAdjoint(autojacvec = SciMLSensitivity.ZygoteVJP())
+        gs_zyg, = Zygote.gradient(tunables) do tunables
+            new_prob = remake(prob; p = repack(tunables))
+            iprob = new_prob.f.initialization_data.initializeprob
+            isol = solve(iprob; sensealg)
+            isol[w]
+        end
+        @test any(!iszero, gs_zyg)
+
+        sensealg = SciMLSensitivity.SteadyStateAdjoint(autojacvec = SciMLSensitivity.ZygoteVJP())
+        gs_obs, = Zygote.gradient(tunables) do tunables
+            new_prob = remake(prob; p = repack(tunables))
+            iprob = new_prob.f.initialization_data.initializeprob
+            isol = solve(iprob; sensealg)
+            obsfn = Zygote.ignore() do
+                SII.observed(isol.prob.f.sys, w).f_oop
+            end
+            obsfn(iprob.u0, iprob.p)
+        end
+        @test gs_zyg ≈ gs_obs
+    end
+
+    @testset "Adjoint through Prob" begin
+        sensealg = SciMLSensitivity.GaussAdjoint(autojacvec = SciMLSensitivity.ZygoteVJP())
+        gs_prob, = Zygote.gradient(tunables) do tunables
+            new_prob = remake(prob; p = repack(tunables))
+            sol = solve(new_prob; sensealg)
+            sum(sol)
+        end
+        @test any(!iszero, gs_prob)
+
+        gs_prob, = Zygote.gradient(tunables) do tunables
+            new_prob = remake(prob; p = repack(tunables))
+            sol = solve(new_prob; sensealg, initializealg = SciMLBase.OverrideInit(), abstol = 1e-6)
+            o = sol[w]
+            o[2]
+        end
+        @test any(!iszero, gs_prob)
+    end
+
+end
diff --git a/test/runtests.jl b/test/runtests.jl
index 3e9af1197..98d80277b 100644
--- a/test/runtests.jl
+++ b/test/runtests.jl
@@ -102,6 +102,13 @@ end
         end
     end
 
+    if GROUP == "All" || GROUP == "Core8"
+        @testset "Core 8" begin
+            @time @safetestset "Adjoints through NonlinearProblem" include("parameter_initialization.jl")
+            @time @safetestset "Initialization with MTK" include("desauty_dae_mwe.jl")
+        end
+    end
+
     if GROUP == "All" || GROUP == "QA"
         @time @safetestset "Quality Assurance" include("aqua.jl")
     end


Reviews:
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-08T12:48:56Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-08T12:54:38Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-08T12:56:27Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-08T13:05:06Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-08T13:07:30Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-08T13:08:10Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-08T13:08:54Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-08T13:11:26Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-08T13:12:10Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-12T16:21:36Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-13T10:52:44Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-13T10:53:59Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-14T13:26:46Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-14T13:27:01Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-14T13:29:24Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-14T13:31:24Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-14T13:33:47Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-15T09:34:59Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-15T10:40:28Z
user: login: AayushSabharwal
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-15T13:13:59Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-15T20:43:36Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-15T21:03:49Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-16T06:26:20Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-16T06:30:47Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-16T07:17:51Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-16T09:52:37Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-16T10:22:01Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-16T10:35:46Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-16T10:38:34Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-16T10:44:38Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-16T11:00:54Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-16T11:03:04Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-16T11:19:00Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-16T11:36:03Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-16T12:16:40Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-16T12:20:12Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-16T12:44:40Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-16T12:47:12Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-16T12:49:28Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-16T12:52:26Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-17T09:31:50Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-17T10:47:14Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-17T11:10:12Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-17T13:53:15Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-17T14:00:19Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-17T14:00:36Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-17T14:01:04Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-17T14:11:09Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-17T14:22:05Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-21T17:21:14Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-21T17:24:11Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-21T17:24:58Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-21T17:27:09Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-21T17:27:27Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-21T17:30:49Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-21T17:31:13Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-21T17:32:14Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-21T17:36:43Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-21T17:37:01Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-21T18:13:45Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-21T19:53:09Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-22T03:35:03Z
user: login: ChrisRackauckas
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-22T11:37:43Z
user: login: ChrisRackauckas
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-22T11:38:57Z
user: login: AayushSabharwal
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-22T11:49:35Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-22T12:04:48Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-22T14:33:15Z
user: login: ChrisRackauckas
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-23T10:56:33Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-23T18:05:49Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-23T21:02:04Z
user: login: AayushSabharwal
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-24T04:49:08Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-24T09:47:54Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-24T12:22:33Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-24T12:40:11Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-24T12:40:23Z
user: login: ChrisRackauckas
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-24T13:39:10Z
user: login: DhairyaLGandhi
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-24T13:50:12Z
user: login: ChrisRackauckas
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-24T22:04:45Z

Review comments:
diff_hunk: @@ -425,6 +425,21 @@ function DiffEqBase._concrete_solve_adjoint(
             save_end = true, kwargs_fwd...)
     end
 
+    # Get gradients for the initialization problem if it exists
+    igs = if _prob.f.initialization_data.initializeprob != nothing
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: this should be before the solve, since you can use the initialization solution from here in the `remake`s of 397-405 in order to set new `u0` and `p` and thus skip running the initialization a second time.
created_at: 2025-03-08T12:48:56Z
author_association: MEMBER
diff_hunk: @@ -103,15 +102,18 @@ end
     else
         if linsolve === nothing && isempty(sensealg.linsolve_kwargs)
             # For the default case use `\` to avoid any form of unnecessary cache allocation
path: src/steadystate_adjoint.jl
user: login: ChrisRackauckas
body: Yeah I don't know about that comment. I think it's just old. (a) `\` always allocates because it uses `lu` instead of `lu!`, so it's re-allocating the while matrix which is larger than any LinearSolve allocation, and (b) we have since 2023 setup tests on StaticArrays, so the immutable path is non-allocating. I don't think (b) was true when this was written.
created_at: 2025-03-08T12:54:38Z
author_association: MEMBER
diff_hunk: @@ -425,6 +425,21 @@ function DiffEqBase._concrete_solve_adjoint(
             save_end = true, kwargs_fwd...)
     end
 
+    # Get gradients for the initialization problem if it exists
+    igs = if _prob.f.initialization_data.initializeprob != nothing
+        iprob = _prob.f.initialization_data.initializeprob
+        ip = parameter_values(iprob)
+        itunables, irepack, ialiases = canonicalize(Tunable(), ip)
+        igs, = Zygote.gradient(ip) do ip
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: This gradient isn't used? I think this would go into the backpass and if I'm thinking clearly, the resulting return is `dp .* igs`?
created_at: 2025-03-08T12:56:27Z
author_association: MEMBER
diff_hunk: @@ -425,6 +425,21 @@ function DiffEqBase._concrete_solve_adjoint(
             save_end = true, kwargs_fwd...)
     end
 
+    # Get gradients for the initialization problem if it exists
+    igs = if _prob.f.initialization_data.initializeprob != nothing
+        iprob = _prob.f.initialization_data.initializeprob
+        ip = parameter_values(iprob)
+        itunables, irepack, ialiases = canonicalize(Tunable(), ip)
+        igs, = Zygote.gradient(ip) do ip
path: src/concrete_solve.jl
user: login: DhairyaLGandhi
body: Not yet. These gradients are currently against the parameters of the initialization problem, not the system exactly. And the mapping between the two is ill defined, so we cannot simply `accum`

I spoke with @AayushSabharwal about a way to map, it seems `initialization_data.intializeprobmap` might have some support to return the correctly shaped vector,  but there are cases where we cannot know the ordering of `dp` either.
created_at: 2025-03-08T13:05:06Z
author_association: MEMBER
diff_hunk: @@ -425,6 +425,21 @@ function DiffEqBase._concrete_solve_adjoint(
             save_end = true, kwargs_fwd...)
     end
 
+    # Get gradients for the initialization problem if it exists
+    igs = if _prob.f.initialization_data.initializeprob != nothing
+        iprob = _prob.f.initialization_data.initializeprob
+        ip = parameter_values(iprob)
+        itunables, irepack, ialiases = canonicalize(Tunable(), ip)
+        igs, = Zygote.gradient(ip) do ip
path: src/concrete_solve.jl
user: login: DhairyaLGandhi
body: There's another subtlety. I am not sure we haven't missed some part of the cfg by manually handling accumulation of gradients. Or any transforms we might need to calculate gradients for. The regular AD graph building typically took care of these details for us, but in this case we would need to worry about incorrect gradients manually
created_at: 2025-03-08T13:07:30Z
author_association: MEMBER
diff_hunk: @@ -103,15 +102,18 @@ end
     else
         if linsolve === nothing && isempty(sensealg.linsolve_kwargs)
             # For the default case use `\` to avoid any form of unnecessary cache allocation
path: src/steadystate_adjoint.jl
user: login: DhairyaLGandhi
body: So glad we can remove this branch altogether. 
created_at: 2025-03-08T13:08:09Z
author_association: MEMBER
diff_hunk: @@ -425,6 +425,21 @@ function DiffEqBase._concrete_solve_adjoint(
             save_end = true, kwargs_fwd...)
     end
 
+    # Get gradients for the initialization problem if it exists
+    igs = if _prob.f.initialization_data.initializeprob != nothing
path: src/concrete_solve.jl
user: login: DhairyaLGandhi
body: How can I indicate to `solve` to avoid running initialization?
created_at: 2025-03-08T13:08:54Z
author_association: MEMBER
diff_hunk: @@ -425,6 +425,21 @@ function DiffEqBase._concrete_solve_adjoint(
             save_end = true, kwargs_fwd...)
     end
 
+    # Get gradients for the initialization problem if it exists
+    igs = if _prob.f.initialization_data.initializeprob != nothing
+        iprob = _prob.f.initialization_data.initializeprob
+        ip = parameter_values(iprob)
+        itunables, irepack, ialiases = canonicalize(Tunable(), ip)
+        igs, = Zygote.gradient(ip) do ip
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: Oh yes, you need to use the initializeprobmap https://github.com/SciML/SciMLBase.jl/blob/master/src/initialization.jl#L268 to map it back to the shape of the initial parameters. 

> but there are cases where we cannot know the ordering of dp either.

p and dp just need the same ordering, so initializeprobmap should do the trick.

> There's another subtlety. I am not sure we haven't missed some part of the cfg by manually handling accumulation of gradients. Or any transforms we might need to calculate gradients for. The regular AD graph building typically took care of these details for us, but in this case we would need to worry about incorrect gradients manually

This is the only change to `(u0,p)` before solving, so this would account for it, given `initializeprobmap` is just an index map so an identity function. 
created_at: 2025-03-08T13:11:26Z
author_association: MEMBER
diff_hunk: @@ -425,6 +425,21 @@ function DiffEqBase._concrete_solve_adjoint(
             save_end = true, kwargs_fwd...)
     end
 
+    # Get gradients for the initialization problem if it exists
+    igs = if _prob.f.initialization_data.initializeprob != nothing
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: `initializealg = NoInit()`. Should probably just do `CheckInit()` for safety but either is fine.
created_at: 2025-03-08T13:12:10Z
author_association: MEMBER
diff_hunk: @@ -425,6 +426,25 @@ function DiffEqBase._concrete_solve_adjoint(
             save_end = true, kwargs_fwd...)
     end
 
+    # Get gradients for the initialization problem if it exists
+    igs = if _prob.f.initialization_data != nothing
+        Zygote.gradient(tunables) do tunables
+            new_prob = remake(_prob, p = repack(tunables))
+            new_u0, new_p, _ = SciMLBase.get_initial_values(new_prob, new_prob, new_prob.f, SciMLBase.OverrideInit(), Val(true);
+                                                            abstol = 1e-6,
+                                                            reltol = 1e-6,
+                                                            sensealg = SteadyStateAdjoint(autojacvec = ZygoteVJP()))
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: shouldn't default to ZygoteVJP. Should use the autojacvec of the ODE
created_at: 2025-03-12T16:21:36Z
author_association: MEMBER
diff_hunk: @@ -425,6 +426,25 @@ function DiffEqBase._concrete_solve_adjoint(
             save_end = true, kwargs_fwd...)
     end
 
+    # Get gradients for the initialization problem if it exists
+    igs = if _prob.f.initialization_data != nothing
+        Zygote.gradient(tunables) do tunables
+            new_prob = remake(_prob, p = repack(tunables))
+            new_u0, new_p, _ = SciMLBase.get_initial_values(new_prob, new_prob, new_prob.f, SciMLBase.OverrideInit(), Val(true);
+                                                            abstol = 1e-6,
+                                                            reltol = 1e-6,
+                                                            sensealg = SteadyStateAdjoint(autojacvec = ZygoteVJP()))
path: src/concrete_solve.jl
user: login: DhairyaLGandhi
body: Addressed in [9a8a845](https://github.com/SciML/SciMLSensitivity.jl/pull/1168/commits/9a8a845803a7739ae6bbaa61f1dcb8e95215bec0)

created_at: 2025-03-13T10:52:44Z
author_association: MEMBER
diff_hunk: @@ -425,6 +425,21 @@ function DiffEqBase._concrete_solve_adjoint(
             save_end = true, kwargs_fwd...)
     end
 
+    # Get gradients for the initialization problem if it exists
+    igs = if _prob.f.initialization_data.initializeprob != nothing
+        iprob = _prob.f.initialization_data.initializeprob
+        ip = parameter_values(iprob)
+        itunables, irepack, ialiases = canonicalize(Tunable(), ip)
+        igs, = Zygote.gradient(ip) do ip
path: src/concrete_solve.jl
user: login: DhairyaLGandhi
body: Addressed this occurance in [95ebbf3](https://github.com/SciML/SciMLSensitivity.jl/pull/1168/commits/95ebbf31b76ac80eeeda45bc24e30185924890d3) to check if this is correct. Will need to work around the `global` call
created_at: 2025-03-13T10:53:59Z
author_association: MEMBER
diff_hunk: @@ -379,11 +391,11 @@ function get_paramjac_config(autojacvec::ReverseDiffVJP, p, f, y, _p, _t;
         if !isRODE
             __p = p isa SciMLBase.NullParameters ? _p :
                   SciMLStructures.replace(Tunable(), p, _p)
-            tape = ReverseDiff.GradientTape((y, __p, [_t])) do u, p, t
+            tape = ReverseDiff.GradientTape((y, _p, [_t])) do u, p, t
path: src/adjoint_common.jl
user: login: ChrisRackauckas
body: are you sure this is the right one?
created_at: 2025-04-14T13:26:46Z
author_association: MEMBER
diff_hunk: @@ -402,8 +414,8 @@ function get_paramjac_config(autojacvec::ReverseDiffVJP, p, f, y, _p, _t;
             # because hasportion(Tunable(), NullParameters) == false
             __p = p isa SciMLBase.NullParameters ? _p :
                   SciMLStructures.replace(Tunable(), p, _p)
-            tape = ReverseDiff.GradientTape((y, __p, [_t])) do u, p, t
-                vec(f(u, p, first(t)))
path: src/adjoint_common.jl
user: login: ChrisRackauckas
body: same down here?
created_at: 2025-04-14T13:27:00Z
author_association: MEMBER
diff_hunk: @@ -412,16 +414,42 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
+        local new_u0
+        local new_p
+        iy, back = Zygote.pullback(tunables) do tunables
+            new_prob = remake(_prob, p = repack(tunables))
+            new_u0, new_p, _ = SciMLBase.get_initial_values(new_prob, new_prob, new_prob.f, SciMLBase.OverrideInit(), Val(true);
+                                                            abstol = 1e-6,
+                                                            reltol = 1e-6,
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: These don't make sense.
created_at: 2025-04-14T13:29:24Z
author_association: MEMBER
diff_hunk: @@ -412,16 +414,42 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
+        local new_u0
+        local new_p
+        iy, back = Zygote.pullback(tunables) do tunables
+            new_prob = remake(_prob, p = repack(tunables))
+            new_u0, new_p, _ = SciMLBase.get_initial_values(new_prob, new_prob, new_prob.f, SciMLBase.OverrideInit(), Val(true);
+                                                            abstol = 1e-6,
+                                                            reltol = 1e-6,
path: src/concrete_solve.jl
user: login: DhairyaLGandhi
body: Yes, these should probably inherit from `kwargs` or be set up to some default. Note that we must specify a tol for this dispatch. 
created_at: 2025-04-14T13:31:24Z
author_association: MEMBER
diff_hunk: @@ -379,11 +391,11 @@ function get_paramjac_config(autojacvec::ReverseDiffVJP, p, f, y, _p, _t;
         if !isRODE
             __p = p isa SciMLBase.NullParameters ? _p :
                   SciMLStructures.replace(Tunable(), p, _p)
-            tape = ReverseDiff.GradientTape((y, __p, [_t])) do u, p, t
+            tape = ReverseDiff.GradientTape((y, _p, [_t])) do u, p, t
path: src/adjoint_common.jl
user: login: DhairyaLGandhi
body: The tunables are replaced into the original structure (`p`) from `_p` and we repack the `_p` tunables again into `p` again when passing into the `f`
created_at: 2025-04-14T13:33:47Z
author_association: MEMBER
diff_hunk: @@ -412,16 +414,42 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
+        local new_u0
+        local new_p
+        iy, back = Zygote.pullback(tunables) do tunables
+            new_prob = remake(_prob, p = repack(tunables))
+            new_u0, new_p, _ = SciMLBase.get_initial_values(new_prob, new_prob, new_prob.f, SciMLBase.OverrideInit(), Val(true);
+                                                            abstol = 1e-6,
+                                                            reltol = 1e-6,
path: src/concrete_solve.jl
user: login: DhairyaLGandhi
body: Addressed in https://github.com/SciML/SciMLSensitivity.jl/pull/1168/commits/984c2ce1ff36a29a75c4bae759eba3e1bc6e2a49
created_at: 2025-04-15T09:34:59Z
author_association: MEMBER
diff_hunk: @@ -379,11 +391,11 @@ function get_paramjac_config(autojacvec::ReverseDiffVJP, p, f, y, _p, _t;
         if !isRODE
             __p = p isa SciMLBase.NullParameters ? _p :
                   SciMLStructures.replace(Tunable(), p, _p)
-            tape = ReverseDiff.GradientTape((y, __p, [_t])) do u, p, t
+            tape = ReverseDiff.GradientTape((y, _p, [_t])) do u, p, t
path: src/adjoint_common.jl
user: login: DhairyaLGandhi
body: The difference is that we were passing just `tunables` into `f`, now we are passing the SciMLStructure, which is what the `f` expects. cc @AayushSabharwal for whether this makes sense
created_at: 2025-04-15T10:40:28Z
author_association: MEMBER
diff_hunk: @@ -379,11 +391,11 @@ function get_paramjac_config(autojacvec::ReverseDiffVJP, p, f, y, _p, _t;
         if !isRODE
             __p = p isa SciMLBase.NullParameters ? _p :
                   SciMLStructures.replace(Tunable(), p, _p)
-            tape = ReverseDiff.GradientTape((y, __p, [_t])) do u, p, t
+            tape = ReverseDiff.GradientTape((y, _p, [_t])) do u, p, t
path: src/adjoint_common.jl
user: login: AayushSabharwal
body: Yeah it looks okay
created_at: 2025-04-15T13:13:59Z
author_association: MEMBER
diff_hunk: @@ -379,11 +391,11 @@ function get_paramjac_config(autojacvec::ReverseDiffVJP, p, f, y, _p, _t;
         if !isRODE
             __p = p isa SciMLBase.NullParameters ? _p :
                   SciMLStructures.replace(Tunable(), p, _p)
-            tape = ReverseDiff.GradientTape((y, __p, [_t])) do u, p, t
+            tape = ReverseDiff.GradientTape((y, _p, [_t])) do u, p, t
path: src/adjoint_common.jl
user: login: ChrisRackauckas
body: >The difference is that we were passing just tunables into f

Makes sense.
created_at: 2025-04-15T20:43:36Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,40 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: Also needs to check that initializealg is not set, is the default, or is using OverrideInit. Should test this is not triggered with say manual BrownBasicInit
created_at: 2025-04-15T20:58:01Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,40 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
+        local new_u0
+        local new_p
+        iy, back = Zygote.pullback(tunables) do tunables
+            new_prob = remake(_prob, p = repack(tunables))
+            new_u0, new_p, _ = SciMLBase.get_initial_values(new_prob, new_prob, new_prob.f, initializealg, Val(isinplace(new_prob));
+                                                            sensealg = SteadyStateAdjoint(autojacvec = sensealg.autojacvec),
+                                                            kwargs_fwd...)
+            new_tunables, _, _ = SciMLStructures.canonicalize(SciMLStructures.Tunable(), new_p)
+            if SciMLBase.initialization_status(_prob) == SciMLBase.OVERDETERMINED
+                sum(new_tunables)
+            else
+                sum(new_u0) + sum(new_tunables)
+            end
+        end
+        igs = back(one(iy))[1] .- one(eltype(tunables))
+
+        igs, new_u0, new_p
+    else
+        nothing, u0, p
+    end
+    _prob = remake(_prob, u0 = new_u0, p = new_p)
+
     if sensealg isa BacksolveAdjoint
-        sol = solve(_prob, alg, args...; save_noise = true,
+        sol = solve(_prob, alg, args...; initializealg = SciMLBase.NoInit(), save_noise = true,
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: It should only noinit if the previous case was ran. Won't this right now break the brownbasic tests?
created_at: 2025-04-15T21:00:45Z
author_association: MEMBER
diff_hunk: @@ -1688,26 +1719,56 @@ function DiffEqBase._concrete_solve_adjoint(
         out = SciMLBase.sensitivity_solution(sol, sol[_save_idxs])
     end
 
+    _, repack_adjoint = if isscimlstructure(p)
+        Zygote.pullback(p) do p
+            t, _, _ = canonicalize(Tunable(), p)
+            t
+        end
+    else
+        nothing, x -> (x,)
+    end
+
     function steadystatebackpass(Δ)
+        Δ = Δ isa AbstractThunk ? unthunk(Δ) : Δ
         # Δ = dg/dx or diffcache.dg_val
         # del g/del p = 0
         function df(_out, u, p, t, i)
             if _save_idxs isa Number
                 _out[_save_idxs] = Δ[_save_idxs]
             elseif Δ isa Number
                 @. _out[_save_idxs] = Δ
-            else
+            elseif Δ isa AbstractArray{<:AbstractArray} || Δ isa AbstractVectorOfArray || Δ isa AbstractArray
                 @. _out[_save_idxs] = Δ[_save_idxs]
+            elseif isnothing(_out)
+                _out
+            else
+                @. _out[_save_idxs] = Δ.u[_save_idxs]
             end
         end
         dp = adjoint_sensitivities(sol, alg; sensealg = sensealg, dgdu = df)
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: When the new reverse ode is built it needs to drop the initial eqs but still keep the dae constraints. It can brownbasic?
created_at: 2025-04-15T21:02:16Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,40 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
path: src/concrete_solve.jl
user: login: DhairyaLGandhi
body: Got it. My understanding was that OverrideInit was what we strictly needed. We can check for BrownBasicInit/ defaults here.
created_at: 2025-04-16T06:26:20Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,40 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
+        local new_u0
+        local new_p
+        iy, back = Zygote.pullback(tunables) do tunables
+            new_prob = remake(_prob, p = repack(tunables))
+            new_u0, new_p, _ = SciMLBase.get_initial_values(new_prob, new_prob, new_prob.f, initializealg, Val(isinplace(new_prob));
+                                                            sensealg = SteadyStateAdjoint(autojacvec = sensealg.autojacvec),
+                                                            kwargs_fwd...)
+            new_tunables, _, _ = SciMLStructures.canonicalize(SciMLStructures.Tunable(), new_p)
+            if SciMLBase.initialization_status(_prob) == SciMLBase.OVERDETERMINED
+                sum(new_tunables)
+            else
+                sum(new_u0) + sum(new_tunables)
+            end
+        end
+        igs = back(one(iy))[1] .- one(eltype(tunables))
+
+        igs, new_u0, new_p
+    else
+        nothing, u0, p
+    end
+    _prob = remake(_prob, u0 = new_u0, p = new_p)
+
     if sensealg isa BacksolveAdjoint
-        sol = solve(_prob, alg, args...; save_noise = true,
+        sol = solve(_prob, alg, args...; initializealg = SciMLBase.NoInit(), save_noise = true,
path: src/concrete_solve.jl
user: login: DhairyaLGandhi
body: If there was no intialization data, it won't have ran the initialization problem at all.

If I can genetically ignore handling `initializealg` and pass it directly to `get_initial_values`, that would be good. Then I can also pass `NoInit` here genetically.
created_at: 2025-04-16T06:30:46Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,40 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
path: src/concrete_solve.jl
user: login: DhairyaLGandhi
body: There doesn't seem to be a method which can take a `BrownFullBasicInit()`. I get a `MethodError`:

```julia
ERROR: MethodError: no method matching get_initial_values(::ODEProblem{…}, ::ODEProblem{…}, ::ODEFunction{…}, ::BrownFullBasicInit{…}, ::Val{…}; sensealg::SteadyStateAdjoint{…}, nlsolve_alg::Nothing)

Closest candidates are:
  get_initial_values(::Any, ::Any, ::Any, ::NoInit, ::Any; kwargs...)
   @ SciMLBase ~/Downloads/arpa/jsmo/t2/SciMLBase.jl/src/initialization.jl:282
  get_initial_values(::Any, ::Any, ::Any, ::SciMLBase.OverrideInit, ::Union{Val{true}, Val{false}}; nlsolve_alg, abstol, reltol, kwargs...)
   @ SciMLBase ~/Downloads/arpa/jsmo/t2/SciMLBase.jl/src/initialization.jl:224
  get_initial_values(::SciMLBase.AbstractDEProblem, ::SciMLBase.DEIntegrator, ::Any, ::CheckInit, ::Union{Val{true}, Val{false}}; abstol, kwargs...)
   @ SciMLBase ~/Downloads/arpa/jsmo/t2/SciMLBase.jl/src/initialization.jl:161
  ...

```

Only `CheckInit`, `NoInit`, and `OverrideInit` have dispatches.
created_at: 2025-04-16T07:17:51Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,40 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
path: src/concrete_solve.jl
user: login: DhairyaLGandhi
body: Having chatted with @AayushSabharwal on this, it seems like BrownBasic and ShampineCollocation do not yet have a path through `get_initial_values` and that would need to be fixed in OrdianryDiffEq. Further, as SciMLSensitivity does not depend on OrdinaryDiffEq, it cannot check for whether there is a default initialisation.since those are defined there. Depending on it also seems like a pretty big hammer for a dep. 
created_at: 2025-04-16T09:52:37Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,40 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
path: src/concrete_solve.jl
user: login: DhairyaLGandhi
body: What would be the best course of action here? Seems like supporting BrownBasicInit is a dispatch that will automatically be utilised when it is moved into SciMLBase. 
created_at: 2025-04-16T10:22:01Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,40 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: Yes I mean BrownBasicInit should not be taking this path. But that's a problem because then they will be disabled in the next stage below, and that needs to be accounted for. This dispatch is already built and setup for BrownBasicInit and there are tests on that.
created_at: 2025-04-16T10:35:46Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,40 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
+        local new_u0
+        local new_p
+        iy, back = Zygote.pullback(tunables) do tunables
+            new_prob = remake(_prob, p = repack(tunables))
+            new_u0, new_p, _ = SciMLBase.get_initial_values(new_prob, new_prob, new_prob.f, initializealg, Val(isinplace(new_prob));
+                                                            sensealg = SteadyStateAdjoint(autojacvec = sensealg.autojacvec),
+                                                            kwargs_fwd...)
+            new_tunables, _, _ = SciMLStructures.canonicalize(SciMLStructures.Tunable(), new_p)
+            if SciMLBase.initialization_status(_prob) == SciMLBase.OVERDETERMINED
+                sum(new_tunables)
+            else
+                sum(new_u0) + sum(new_tunables)
+            end
+        end
+        igs = back(one(iy))[1] .- one(eltype(tunables))
+
+        igs, new_u0, new_p
+    else
+        nothing, u0, p
+    end
+    _prob = remake(_prob, u0 = new_u0, p = new_p)
+
     if sensealg isa BacksolveAdjoint
-        sol = solve(_prob, alg, args...; save_noise = true,
+        sol = solve(_prob, alg, args...; initializealg = SciMLBase.NoInit(), save_noise = true,
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: No that is not correct. If there was no initialization data then it will use the built in initialization, defaulting to BrownBasicInit. It's impossible for a DAE solver to generally work without running initialization of some form, the MTK one is just a new specialized one but there has always been a numerical one in the solver. And if it hits that case, this code will now disable that. 

https://github.com/SciML/SciMLSensitivity.jl/blob/master/test/adjoint.jl#L952-L978 this code will hit that. I think it's not failing because it's not so pronounced here. You might want to change that test to https://github.com/SciML/SciMLSensitivity.jl/blob/master/test/adjoint.jl#L975C5-L975C69 `prob_singular_mm = ODEProblem(f, [1.0, 0.0, 1.0], (0.0, 100), p)` and it would pass before and fail now.
created_at: 2025-04-16T10:38:34Z
author_association: MEMBER
diff_hunk: @@ -1688,26 +1719,56 @@ function DiffEqBase._concrete_solve_adjoint(
         out = SciMLBase.sensitivity_solution(sol, sol[_save_idxs])
     end
 
+    _, repack_adjoint = if isscimlstructure(p)
+        Zygote.pullback(p) do p
+            t, _, _ = canonicalize(Tunable(), p)
+            t
+        end
+    else
+        nothing, x -> (x,)
+    end
+
     function steadystatebackpass(Δ)
+        Δ = Δ isa AbstractThunk ? unthunk(Δ) : Δ
         # Δ = dg/dx or diffcache.dg_val
         # del g/del p = 0
         function df(_out, u, p, t, i)
             if _save_idxs isa Number
                 _out[_save_idxs] = Δ[_save_idxs]
             elseif Δ isa Number
                 @. _out[_save_idxs] = Δ
-            else
+            elseif Δ isa AbstractArray{<:AbstractArray} || Δ isa AbstractVectorOfArray || Δ isa AbstractArray
                 @. _out[_save_idxs] = Δ[_save_idxs]
+            elseif isnothing(_out)
+                _out
+            else
+                @. _out[_save_idxs] = Δ.u[_save_idxs]
             end
         end
         dp = adjoint_sensitivities(sol, alg; sensealg = sensealg, dgdu = df)
path: src/concrete_solve.jl
user: login: DhairyaLGandhi
body: Is there a way to drop the initial eqs after its solved? The assumption was since we run with `NoInit`, no initialization is run post the first call to `get_initial_values` and we accumulate those gradients independently of the adaptive solve. 
created_at: 2025-04-16T10:44:38Z
author_association: MEMBER
diff_hunk: @@ -1688,26 +1719,56 @@ function DiffEqBase._concrete_solve_adjoint(
         out = SciMLBase.sensitivity_solution(sol, sol[_save_idxs])
     end
 
+    _, repack_adjoint = if isscimlstructure(p)
+        Zygote.pullback(p) do p
+            t, _, _ = canonicalize(Tunable(), p)
+            t
+        end
+    else
+        nothing, x -> (x,)
+    end
+
     function steadystatebackpass(Δ)
+        Δ = Δ isa AbstractThunk ? unthunk(Δ) : Δ
         # Δ = dg/dx or diffcache.dg_val
         # del g/del p = 0
         function df(_out, u, p, t, i)
             if _save_idxs isa Number
                 _out[_save_idxs] = Δ[_save_idxs]
             elseif Δ isa Number
                 @. _out[_save_idxs] = Δ
-            else
+            elseif Δ isa AbstractArray{<:AbstractArray} || Δ isa AbstractVectorOfArray || Δ isa AbstractArray
                 @. _out[_save_idxs] = Δ[_save_idxs]
+            elseif isnothing(_out)
+                _out
+            else
+                @. _out[_save_idxs] = Δ.u[_save_idxs]
             end
         end
         dp = adjoint_sensitivities(sol, alg; sensealg = sensealg, dgdu = df)
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: But the reverse pass needs to run with some form of initialization or the starting algebraic conditions may not be satisfied. Don't run this one with NoInit(), that would be prone to hiding issue. For this one, at most CheckInit(), but I'm saying that BrownBasicInit() is likely the one justified here since the 0 initial condition is only true on the differential variables, while the algebraic variable initial conditions will be unknown, but the Newton solve will have zero derivative because all of the inputs are just Newton guesses, so BrownBasic will work out for the reverse. We should probably hardcode that since it's always the solution there. 
created_at: 2025-04-16T11:00:54Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,40 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
+        local new_u0
+        local new_p
+        iy, back = Zygote.pullback(tunables) do tunables
+            new_prob = remake(_prob, p = repack(tunables))
+            new_u0, new_p, _ = SciMLBase.get_initial_values(new_prob, new_prob, new_prob.f, initializealg, Val(isinplace(new_prob));
+                                                            sensealg = SteadyStateAdjoint(autojacvec = sensealg.autojacvec),
+                                                            kwargs_fwd...)
+            new_tunables, _, _ = SciMLStructures.canonicalize(SciMLStructures.Tunable(), new_p)
+            if SciMLBase.initialization_status(_prob) == SciMLBase.OVERDETERMINED
+                sum(new_tunables)
+            else
+                sum(new_u0) + sum(new_tunables)
+            end
+        end
+        igs = back(one(iy))[1] .- one(eltype(tunables))
+
+        igs, new_u0, new_p
+    else
+        nothing, u0, p
+    end
+    _prob = remake(_prob, u0 = new_u0, p = new_p)
+
     if sensealg isa BacksolveAdjoint
-        sol = solve(_prob, alg, args...; save_noise = true,
+        sol = solve(_prob, alg, args...; initializealg = SciMLBase.NoInit(), save_noise = true,
path: src/concrete_solve.jl
user: login: DhairyaLGandhi
body: You're right of course for the DAEs, but since `BrownBasicInit` is defined in OrdinaryDiffEq, and this package does not depend on it, I need a way for us to be able to dispatch to it. So if I understand the comment from earlier, we need a check for the default initialization, and add a branch that solves for that `prob`, and collect all the outputs.
created_at: 2025-04-16T11:03:04Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,40 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
+        local new_u0
+        local new_p
+        iy, back = Zygote.pullback(tunables) do tunables
+            new_prob = remake(_prob, p = repack(tunables))
+            new_u0, new_p, _ = SciMLBase.get_initial_values(new_prob, new_prob, new_prob.f, initializealg, Val(isinplace(new_prob));
+                                                            sensealg = SteadyStateAdjoint(autojacvec = sensealg.autojacvec),
+                                                            kwargs_fwd...)
+            new_tunables, _, _ = SciMLStructures.canonicalize(SciMLStructures.Tunable(), new_p)
+            if SciMLBase.initialization_status(_prob) == SciMLBase.OVERDETERMINED
+                sum(new_tunables)
+            else
+                sum(new_u0) + sum(new_tunables)
+            end
+        end
+        igs = back(one(iy))[1] .- one(eltype(tunables))
+
+        igs, new_u0, new_p
+    else
+        nothing, u0, p
+    end
+    _prob = remake(_prob, u0 = new_u0, p = new_p)
+
     if sensealg isa BacksolveAdjoint
-        sol = solve(_prob, alg, args...; save_noise = true,
+        sol = solve(_prob, alg, args...; initializealg = SciMLBase.NoInit(), save_noise = true,
path: src/concrete_solve.jl
user: login: DhairyaLGandhi
body: Both `BrownBasicInit` and `OrdinaryDiffEqCore.DefaultInit` require us to depend on a whole package for the default dispatch. Can it be exposed as a dispatch of `get_initial_values` instead?
created_at: 2025-04-16T11:18:59Z
author_association: MEMBER
diff_hunk: @@ -1688,26 +1719,56 @@ function DiffEqBase._concrete_solve_adjoint(
         out = SciMLBase.sensitivity_solution(sol, sol[_save_idxs])
     end
 
+    _, repack_adjoint = if isscimlstructure(p)
+        Zygote.pullback(p) do p
+            t, _, _ = canonicalize(Tunable(), p)
+            t
+        end
+    else
+        nothing, x -> (x,)
+    end
+
     function steadystatebackpass(Δ)
+        Δ = Δ isa AbstractThunk ? unthunk(Δ) : Δ
         # Δ = dg/dx or diffcache.dg_val
         # del g/del p = 0
         function df(_out, u, p, t, i)
             if _save_idxs isa Number
                 _out[_save_idxs] = Δ[_save_idxs]
             elseif Δ isa Number
                 @. _out[_save_idxs] = Δ
-            else
+            elseif Δ isa AbstractArray{<:AbstractArray} || Δ isa AbstractVectorOfArray || Δ isa AbstractArray
                 @. _out[_save_idxs] = Δ[_save_idxs]
+            elseif isnothing(_out)
+                _out
+            else
+                @. _out[_save_idxs] = Δ.u[_save_idxs]
             end
         end
         dp = adjoint_sensitivities(sol, alg; sensealg = sensealg, dgdu = df)
path: src/concrete_solve.jl
user: login: DhairyaLGandhi
body: Ok, that will require us to add an OrdinaryDiffEqCore dep in this package. I will add that. 
created_at: 2025-04-16T11:36:02Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,40 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
+        local new_u0
+        local new_p
+        iy, back = Zygote.pullback(tunables) do tunables
+            new_prob = remake(_prob, p = repack(tunables))
+            new_u0, new_p, _ = SciMLBase.get_initial_values(new_prob, new_prob, new_prob.f, initializealg, Val(isinplace(new_prob));
+                                                            sensealg = SteadyStateAdjoint(autojacvec = sensealg.autojacvec),
+                                                            kwargs_fwd...)
+            new_tunables, _, _ = SciMLStructures.canonicalize(SciMLStructures.Tunable(), new_p)
+            if SciMLBase.initialization_status(_prob) == SciMLBase.OVERDETERMINED
+                sum(new_tunables)
+            else
+                sum(new_u0) + sum(new_tunables)
+            end
+        end
+        igs = back(one(iy))[1] .- one(eltype(tunables))
+
+        igs, new_u0, new_p
+    else
+        nothing, u0, p
+    end
+    _prob = remake(_prob, u0 = new_u0, p = new_p)
+
     if sensealg isa BacksolveAdjoint
-        sol = solve(_prob, alg, args...; save_noise = true,
+        sol = solve(_prob, alg, args...; initializealg = SciMLBase.NoInit(), save_noise = true,
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: We can just take the dep. It's pretty much required anyways.
created_at: 2025-04-16T12:16:40Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,40 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
+        local new_u0
+        local new_p
+        iy, back = Zygote.pullback(tunables) do tunables
+            new_prob = remake(_prob, p = repack(tunables))
+            new_u0, new_p, _ = SciMLBase.get_initial_values(new_prob, new_prob, new_prob.f, initializealg, Val(isinplace(new_prob));
+                                                            sensealg = SteadyStateAdjoint(autojacvec = sensealg.autojacvec),
+                                                            kwargs_fwd...)
+            new_tunables, _, _ = SciMLStructures.canonicalize(SciMLStructures.Tunable(), new_p)
+            if SciMLBase.initialization_status(_prob) == SciMLBase.OVERDETERMINED
+                sum(new_tunables)
+            else
+                sum(new_u0) + sum(new_tunables)
+            end
+        end
+        igs = back(one(iy))[1] .- one(eltype(tunables))
+
+        igs, new_u0, new_p
+    else
+        nothing, u0, p
+    end
+    _prob = remake(_prob, u0 = new_u0, p = new_p)
+
     if sensealg isa BacksolveAdjoint
-        sol = solve(_prob, alg, args...; save_noise = true,
+        sol = solve(_prob, alg, args...; initializealg = SciMLBase.NoInit(), save_noise = true,
path: src/concrete_solve.jl
user: login: DhairyaLGandhi
body: https://github.com/SciML/SciMLSensitivity.jl/pull/1168/commits/82cd5fe2959be3411fffb5ea979b9fcefef075cb made the changes to the test, however they managed to pass. that is with a tolerance of 1e-5. I am sure it can be made to fail with tighter tolerances.
created_at: 2025-04-16T12:20:11Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,40 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
+        local new_u0
+        local new_p
+        iy, back = Zygote.pullback(tunables) do tunables
+            new_prob = remake(_prob, p = repack(tunables))
+            new_u0, new_p, _ = SciMLBase.get_initial_values(new_prob, new_prob, new_prob.f, initializealg, Val(isinplace(new_prob));
+                                                            sensealg = SteadyStateAdjoint(autojacvec = sensealg.autojacvec),
+                                                            kwargs_fwd...)
+            new_tunables, _, _ = SciMLStructures.canonicalize(SciMLStructures.Tunable(), new_p)
+            if SciMLBase.initialization_status(_prob) == SciMLBase.OVERDETERMINED
+                sum(new_tunables)
+            else
+                sum(new_u0) + sum(new_tunables)
+            end
+        end
+        igs = back(one(iy))[1] .- one(eltype(tunables))
+
+        igs, new_u0, new_p
+    else
+        nothing, u0, p
+    end
+    _prob = remake(_prob, u0 = new_u0, p = new_p)
+
     if sensealg isa BacksolveAdjoint
-        sol = solve(_prob, alg, args...; save_noise = true,
+        sol = solve(_prob, alg, args...; initializealg = SciMLBase.NoInit(), save_noise = true,
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: ```julia
using OrdinaryDiffEq

function rober(du, u, p, t)
    y₁, y₂, y₃ = u
    k₁, k₂, k₃ = p
    du[1] = -k₁ * y₁ + k₃ * y₂ * y₃
    du[2] = k₁ * y₁ - k₂ * y₂^2 - k₃ * y₂ * y₃
    du[3] = y₁ + y₂ + y₃ - 1
    nothing
end
function rober(u, p, t)
    y₁, y₂, y₃ = u
    k₁, k₂, k₃ = p
    return [-k₁ * y₁ + k₃ * y₂ * y₃,
        k₁ * y₁ - k₂ * y₂^2 - k₃ * y₂ * y₃,
        y₁ + y₂ + y₃ - 1]
end
M = [1.0 0 0
     0 1.0 0
     0 0 0]

f = ODEFunction{true}(rober, mass_matrix = M)
p = [0.04, 3e7, 1e4]

prob_singular_mm = ODEProblem(f, [1.0, 0.0, 1.0], (0.0, 100), p)
sol_singular_mm = solve(prob_singular_mm, FBDF(autodiff = false),
    reltol = 1e-12, abstol = 1e-12, initializealg = NoInit())
```

The solvers have gotten good enough that they end up fixing this in the first step 😅 but the initial condition is wrong. So a test to add here is:

```julia
sol_singular_mm = solve(prob_singular_mm, FBDF(autodiff = false),
    reltol = 1e-12, abstol = 1e-12, initializealg = BrownFullBasicInit())
@test all(u[1] + u[2] + u[3] ≈ 1 for u in sol_singular_mm.u)
```

This test `all(u[1] + u[2] + u[3] ≈ 1 for u in sol_singular_mm.u)` will fail without the fix when done on the solution that is generated in the adjoint call, and harder DAEs will just fail to start solving.
created_at: 2025-04-16T12:44:40Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,40 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
+        local new_u0
+        local new_p
+        iy, back = Zygote.pullback(tunables) do tunables
+            new_prob = remake(_prob, p = repack(tunables))
+            new_u0, new_p, _ = SciMLBase.get_initial_values(new_prob, new_prob, new_prob.f, initializealg, Val(isinplace(new_prob));
+                                                            sensealg = SteadyStateAdjoint(autojacvec = sensealg.autojacvec),
+                                                            kwargs_fwd...)
+            new_tunables, _, _ = SciMLStructures.canonicalize(SciMLStructures.Tunable(), new_p)
+            if SciMLBase.initialization_status(_prob) == SciMLBase.OVERDETERMINED
+                sum(new_tunables)
+            else
+                sum(new_u0) + sum(new_tunables)
+            end
+        end
+        igs = back(one(iy))[1] .- one(eltype(tunables))
+
+        igs, new_u0, new_p
+    else
+        nothing, u0, p
+    end
+    _prob = remake(_prob, u0 = new_u0, p = new_p)
+
     if sensealg isa BacksolveAdjoint
-        sol = solve(_prob, alg, args...; save_noise = true,
+        sol = solve(_prob, alg, args...; initializealg = SciMLBase.NoInit(), save_noise = true,
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: Note that this is in the concrete solve part, so it would need to be tested by checking if the forward solution used in a loss function satisfies that property. The manual solve of that test should be okay since it won't go down this code path at all. So we might need to make another mass matrix DAE test with concrete solve if we don't have one.
created_at: 2025-04-16T12:47:12Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,40 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
+        local new_u0
+        local new_p
+        iy, back = Zygote.pullback(tunables) do tunables
+            new_prob = remake(_prob, p = repack(tunables))
+            new_u0, new_p, _ = SciMLBase.get_initial_values(new_prob, new_prob, new_prob.f, initializealg, Val(isinplace(new_prob));
+                                                            sensealg = SteadyStateAdjoint(autojacvec = sensealg.autojacvec),
+                                                            kwargs_fwd...)
+            new_tunables, _, _ = SciMLStructures.canonicalize(SciMLStructures.Tunable(), new_p)
+            if SciMLBase.initialization_status(_prob) == SciMLBase.OVERDETERMINED
+                sum(new_tunables)
+            else
+                sum(new_u0) + sum(new_tunables)
+            end
+        end
+        igs = back(one(iy))[1] .- one(eltype(tunables))
+
+        igs, new_u0, new_p
+    else
+        nothing, u0, p
+    end
+    _prob = remake(_prob, u0 = new_u0, p = new_p)
+
     if sensealg isa BacksolveAdjoint
-        sol = solve(_prob, alg, args...; save_noise = true,
+        sol = solve(_prob, alg, args...; initializealg = SciMLBase.NoInit(), save_noise = true,
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: So a way to do this same test is then in here https://github.com/SciML/SciMLSensitivity.jl/blob/master/test/mtk.jl#L37 have one where you set it to BrownFullBasicInit check whether `w ~ x + y + z + 2 * β` is satisfied at every step (with the issue potentially being at step 1)
created_at: 2025-04-16T12:49:28Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,40 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
+        local new_u0
+        local new_p
+        iy, back = Zygote.pullback(tunables) do tunables
+            new_prob = remake(_prob, p = repack(tunables))
+            new_u0, new_p, _ = SciMLBase.get_initial_values(new_prob, new_prob, new_prob.f, initializealg, Val(isinplace(new_prob));
+                                                            sensealg = SteadyStateAdjoint(autojacvec = sensealg.autojacvec),
+                                                            kwargs_fwd...)
+            new_tunables, _, _ = SciMLStructures.canonicalize(SciMLStructures.Tunable(), new_p)
+            if SciMLBase.initialization_status(_prob) == SciMLBase.OVERDETERMINED
+                sum(new_tunables)
+            else
+                sum(new_u0) + sum(new_tunables)
+            end
+        end
+        igs = back(one(iy))[1] .- one(eltype(tunables))
+
+        igs, new_u0, new_p
+    else
+        nothing, u0, p
+    end
+    _prob = remake(_prob, u0 = new_u0, p = new_p)
+
     if sensealg isa BacksolveAdjoint
-        sol = solve(_prob, alg, args...; save_noise = true,
+        sol = solve(_prob, alg, args...; initializealg = SciMLBase.NoInit(), save_noise = true,
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: I forget how to opt easily out of AD, but something like adding to this file:

```julia
dmtk, = Zygote.gradient(mtkparams) do p
    new_sol = solve(prob, Rosenbrock23(), p = p, initializealg = BrownFullBasicInit())
    @nodifferentiate @test all(new_sol[x + y + z + 2 * β - w] .≈ 0)
    mean(abs.(new_sol[sys.x] .- gt))
end
```
created_at: 2025-04-16T12:52:26Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,40 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
+        local new_u0
+        local new_p
+        iy, back = Zygote.pullback(tunables) do tunables
+            new_prob = remake(_prob, p = repack(tunables))
+            new_u0, new_p, _ = SciMLBase.get_initial_values(new_prob, new_prob, new_prob.f, initializealg, Val(isinplace(new_prob));
+                                                            sensealg = SteadyStateAdjoint(autojacvec = sensealg.autojacvec),
+                                                            kwargs_fwd...)
+            new_tunables, _, _ = SciMLStructures.canonicalize(SciMLStructures.Tunable(), new_p)
+            if SciMLBase.initialization_status(_prob) == SciMLBase.OVERDETERMINED
+                sum(new_tunables)
+            else
+                sum(new_u0) + sum(new_tunables)
+            end
+        end
+        igs = back(one(iy))[1] .- one(eltype(tunables))
+
+        igs, new_u0, new_p
+    else
+        nothing, u0, p
+    end
+    _prob = remake(_prob, u0 = new_u0, p = new_p)
+
     if sensealg isa BacksolveAdjoint
-        sol = solve(_prob, alg, args...; save_noise = true,
+        sol = solve(_prob, alg, args...; initializealg = SciMLBase.NoInit(), save_noise = true,
path: src/concrete_solve.jl
user: login: DhairyaLGandhi
body: I printed the results with `BrownFullBasicInit`. We get  

```julia
dmtk, = Zygote.gradient(mtkparams) do p
    new_sol = solve(prob, Rosenbrock23(), p = p, initializealg = BrownFullBasicInit())
    Zygote.ChainRules.ChainRulesCore.ignore_derivatives() do
        @show extrema(new_sol[x + y + z + 2 * β - w])
        @test all(new_sol[x + y + z + 2 * β - w]  .≈ 0)
    end
    mean(abs.(new_sol[sys.x] .- gt))
end
```

```julia
extrema(new_sol[(x + y + z + 2β) - w]) = (-7.882583474838611e-15, 7.105427357601002e-15)
```

There are some errors with a `MethodError` for `initialize_dae!(::SDEIntegrator, ...)`
created_at: 2025-04-17T09:31:50Z
author_association: MEMBER
diff_hunk: @@ -1688,26 +1719,56 @@ function DiffEqBase._concrete_solve_adjoint(
         out = SciMLBase.sensitivity_solution(sol, sol[_save_idxs])
     end
 
+    _, repack_adjoint = if isscimlstructure(p)
+        Zygote.pullback(p) do p
+            t, _, _ = canonicalize(Tunable(), p)
+            t
+        end
+    else
+        nothing, x -> (x,)
+    end
+
     function steadystatebackpass(Δ)
+        Δ = Δ isa AbstractThunk ? unthunk(Δ) : Δ
         # Δ = dg/dx or diffcache.dg_val
         # del g/del p = 0
         function df(_out, u, p, t, i)
             if _save_idxs isa Number
                 _out[_save_idxs] = Δ[_save_idxs]
             elseif Δ isa Number
                 @. _out[_save_idxs] = Δ
-            else
+            elseif Δ isa AbstractArray{<:AbstractArray} || Δ isa AbstractVectorOfArray || Δ isa AbstractArray
                 @. _out[_save_idxs] = Δ[_save_idxs]
+            elseif isnothing(_out)
+                _out
+            else
+                @. _out[_save_idxs] = Δ.u[_save_idxs]
             end
         end
         dp = adjoint_sensitivities(sol, alg; sensealg = sensealg, dgdu = df)
path: src/concrete_solve.jl
user: login: DhairyaLGandhi
body: Is the 0 derivative also applicable to parameters? Or only the unknowns?

created_at: 2025-04-17T10:47:14Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,40 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
+        local new_u0
+        local new_p
+        iy, back = Zygote.pullback(tunables) do tunables
+            new_prob = remake(_prob, p = repack(tunables))
+            new_u0, new_p, _ = SciMLBase.get_initial_values(new_prob, new_prob, new_prob.f, initializealg, Val(isinplace(new_prob));
+                                                            sensealg = SteadyStateAdjoint(autojacvec = sensealg.autojacvec),
+                                                            kwargs_fwd...)
+            new_tunables, _, _ = SciMLStructures.canonicalize(SciMLStructures.Tunable(), new_p)
+            if SciMLBase.initialization_status(_prob) == SciMLBase.OVERDETERMINED
+                sum(new_tunables)
+            else
+                sum(new_u0) + sum(new_tunables)
+            end
+        end
+        igs = back(one(iy))[1] .- one(eltype(tunables))
+
+        igs, new_u0, new_p
+    else
+        nothing, u0, p
+    end
+    _prob = remake(_prob, u0 = new_u0, p = new_p)
+
     if sensealg isa BacksolveAdjoint
-        sol = solve(_prob, alg, args...; save_noise = true,
+        sol = solve(_prob, alg, args...; initializealg = SciMLBase.NoInit(), save_noise = true,
path: src/concrete_solve.jl
user: login: DhairyaLGandhi
body: Seems like SDEs don't work with `BrownBasicInit`. I tried to get the "default `initializealg` for `solve`" from the `kwargs` and `prob.kwargs`, but those are both empty. And `prob.f.initialization_data` is still `nothing`. 
created_at: 2025-04-17T11:10:12Z
author_association: MEMBER
diff_hunk: @@ -1688,26 +1719,56 @@ function DiffEqBase._concrete_solve_adjoint(
         out = SciMLBase.sensitivity_solution(sol, sol[_save_idxs])
     end
 
+    _, repack_adjoint = if isscimlstructure(p)
+        Zygote.pullback(p) do p
+            t, _, _ = canonicalize(Tunable(), p)
+            t
+        end
+    else
+        nothing, x -> (x,)
+    end
+
     function steadystatebackpass(Δ)
+        Δ = Δ isa AbstractThunk ? unthunk(Δ) : Δ
         # Δ = dg/dx or diffcache.dg_val
         # del g/del p = 0
         function df(_out, u, p, t, i)
             if _save_idxs isa Number
                 _out[_save_idxs] = Δ[_save_idxs]
             elseif Δ isa Number
                 @. _out[_save_idxs] = Δ
-            else
+            elseif Δ isa AbstractArray{<:AbstractArray} || Δ isa AbstractVectorOfArray || Δ isa AbstractArray
                 @. _out[_save_idxs] = Δ[_save_idxs]
+            elseif isnothing(_out)
+                _out
+            else
+                @. _out[_save_idxs] = Δ.u[_save_idxs]
             end
         end
         dp = adjoint_sensitivities(sol, alg; sensealg = sensealg, dgdu = df)
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: Its applicable to all Newton guess values. There is no parameter init going on to reverse so it's only for algebraic conditions so it's only Newton guesses.
created_at: 2025-04-17T13:53:15Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,40 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
+        local new_u0
+        local new_p
+        iy, back = Zygote.pullback(tunables) do tunables
+            new_prob = remake(_prob, p = repack(tunables))
+            new_u0, new_p, _ = SciMLBase.get_initial_values(new_prob, new_prob, new_prob.f, initializealg, Val(isinplace(new_prob));
+                                                            sensealg = SteadyStateAdjoint(autojacvec = sensealg.autojacvec),
+                                                            kwargs_fwd...)
+            new_tunables, _, _ = SciMLStructures.canonicalize(SciMLStructures.Tunable(), new_p)
+            if SciMLBase.initialization_status(_prob) == SciMLBase.OVERDETERMINED
+                sum(new_tunables)
+            else
+                sum(new_u0) + sum(new_tunables)
+            end
+        end
+        igs = back(one(iy))[1] .- one(eltype(tunables))
+
+        igs, new_u0, new_p
+    else
+        nothing, u0, p
+    end
+    _prob = remake(_prob, u0 = new_u0, p = new_p)
+
     if sensealg isa BacksolveAdjoint
-        sol = solve(_prob, alg, args...; save_noise = true,
+        sol = solve(_prob, alg, args...; initializealg = SciMLBase.NoInit(), save_noise = true,
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: Oh 😅 . That case was too simple, MTK turns it into an ODE. Let's make it a DAE.

```julia
@parameters σ ρ β A[1:3]
@variables x(t) y(t) z(t) w(t) w2(t)
eqs = [D(D(x)) ~ σ * (y - x),
    D(y) ~ x * (ρ - z) - y,
    D(z) ~ x * y - β * z,
    w ~ x + y + z + 2 * β
    0 ~ x^2 + y^2 - w2^2
]
@mtkbuild sys = ODESystem(eqs, t)
```

That should make it so that it eliminates the `w` term, but doesn't eliminate the `w2` term. The DAE check is on the w2 term, the observed handling check is on the w term.
created_at: 2025-04-17T14:00:19Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,40 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
+        local new_u0
+        local new_p
+        iy, back = Zygote.pullback(tunables) do tunables
+            new_prob = remake(_prob, p = repack(tunables))
+            new_u0, new_p, _ = SciMLBase.get_initial_values(new_prob, new_prob, new_prob.f, initializealg, Val(isinplace(new_prob));
+                                                            sensealg = SteadyStateAdjoint(autojacvec = sensealg.autojacvec),
+                                                            kwargs_fwd...)
+            new_tunables, _, _ = SciMLStructures.canonicalize(SciMLStructures.Tunable(), new_p)
+            if SciMLBase.initialization_status(_prob) == SciMLBase.OVERDETERMINED
+                sum(new_tunables)
+            else
+                sum(new_u0) + sum(new_tunables)
+            end
+        end
+        igs = back(one(iy))[1] .- one(eltype(tunables))
+
+        igs, new_u0, new_p
+    else
+        nothing, u0, p
+    end
+    _prob = remake(_prob, u0 = new_u0, p = new_p)
+
     if sensealg isa BacksolveAdjoint
-        sol = solve(_prob, alg, args...; save_noise = true,
+        sol = solve(_prob, alg, args...; initializealg = SciMLBase.NoInit(), save_noise = true,
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: That will need to change the integrator to `Rodas5P`, Tsit5 will not be compatible with this form.
created_at: 2025-04-17T14:00:36Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,40 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
+        local new_u0
+        local new_p
+        iy, back = Zygote.pullback(tunables) do tunables
+            new_prob = remake(_prob, p = repack(tunables))
+            new_u0, new_p, _ = SciMLBase.get_initial_values(new_prob, new_prob, new_prob.f, initializealg, Val(isinplace(new_prob));
+                                                            sensealg = SteadyStateAdjoint(autojacvec = sensealg.autojacvec),
+                                                            kwargs_fwd...)
+            new_tunables, _, _ = SciMLStructures.canonicalize(SciMLStructures.Tunable(), new_p)
+            if SciMLBase.initialization_status(_prob) == SciMLBase.OVERDETERMINED
+                sum(new_tunables)
+            else
+                sum(new_u0) + sum(new_tunables)
+            end
+        end
+        igs = back(one(iy))[1] .- one(eltype(tunables))
+
+        igs, new_u0, new_p
+    else
+        nothing, u0, p
+    end
+    _prob = remake(_prob, u0 = new_u0, p = new_p)
+
     if sensealg isa BacksolveAdjoint
-        sol = solve(_prob, alg, args...; save_noise = true,
+        sol = solve(_prob, alg, args...; initializealg = SciMLBase.NoInit(), save_noise = true,
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: For SDEs, we will just need to make it compatible with BrownBasicInit.
created_at: 2025-04-17T14:01:04Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,40 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
+        local new_u0
+        local new_p
+        iy, back = Zygote.pullback(tunables) do tunables
+            new_prob = remake(_prob, p = repack(tunables))
+            new_u0, new_p, _ = SciMLBase.get_initial_values(new_prob, new_prob, new_prob.f, initializealg, Val(isinplace(new_prob));
+                                                            sensealg = SteadyStateAdjoint(autojacvec = sensealg.autojacvec),
+                                                            kwargs_fwd...)
+            new_tunables, _, _ = SciMLStructures.canonicalize(SciMLStructures.Tunable(), new_p)
+            if SciMLBase.initialization_status(_prob) == SciMLBase.OVERDETERMINED
+                sum(new_tunables)
+            else
+                sum(new_u0) + sum(new_tunables)
+            end
+        end
+        igs = back(one(iy))[1] .- one(eltype(tunables))
+
+        igs, new_u0, new_p
+    else
+        nothing, u0, p
+    end
+    _prob = remake(_prob, u0 = new_u0, p = new_p)
+
     if sensealg isa BacksolveAdjoint
-        sol = solve(_prob, alg, args...; save_noise = true,
+        sol = solve(_prob, alg, args...; initializealg = SciMLBase.NoInit(), save_noise = true,
path: src/concrete_solve.jl
user: login: DhairyaLGandhi
body: Ah, I was so confused why it worked out, I see the `InitialFailure` now.
created_at: 2025-04-17T14:11:09Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,40 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+
+    igs, new_u0, new_p = if _prob.f.initialization_data !== nothing
+        local new_u0
+        local new_p
+        iy, back = Zygote.pullback(tunables) do tunables
+            new_prob = remake(_prob, p = repack(tunables))
+            new_u0, new_p, _ = SciMLBase.get_initial_values(new_prob, new_prob, new_prob.f, initializealg, Val(isinplace(new_prob));
+                                                            sensealg = SteadyStateAdjoint(autojacvec = sensealg.autojacvec),
+                                                            kwargs_fwd...)
+            new_tunables, _, _ = SciMLStructures.canonicalize(SciMLStructures.Tunable(), new_p)
+            if SciMLBase.initialization_status(_prob) == SciMLBase.OVERDETERMINED
+                sum(new_tunables)
+            else
+                sum(new_u0) + sum(new_tunables)
+            end
+        end
+        igs = back(one(iy))[1] .- one(eltype(tunables))
+
+        igs, new_u0, new_p
+    else
+        nothing, u0, p
+    end
+    _prob = remake(_prob, u0 = new_u0, p = new_p)
+
     if sensealg isa BacksolveAdjoint
-        sol = solve(_prob, alg, args...; save_noise = true,
+        sol = solve(_prob, alg, args...; initializealg = SciMLBase.NoInit(), save_noise = true,
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: Okay good. Yeah because MTK is too smart and makes lots of simple examples not DAEs 😅. But now you got the DAE, and if not running the built in init then you get the error I was expecting. The fix is that it needs to run brownbasic before solving for the same reason reverse needs to. Good we worked out a test for this
created_at: 2025-04-17T14:22:05Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,61 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+    kwargs_init = kwargs_adj[Base.diff_names(Base._nt_names(kwargs_adj), (:initializealg,))]
+
+    if haskey(kwargs, :initializealg) || haskey(prob.kwargs, :initializealg)
+        initializealg = haskey(kwargs, :initializealg) ? kwargs[:initializealg] : prob.kwargs[:initializealg]
+    else
+        initializealg = DefaultInit()
+    end
+
+    default_inits = Union{OverrideInit, Nothing, DefaultInit}
+    igs, new_u0, new_p, new_initializealg = if (SciMLBase.has_initialization_data(_prob.f) && initializealg isa default_inits) || initializealg isa CheckInit
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: ```suggestion
    igs, new_u0, new_p, new_initializealg = if (SciMLBase.has_initialization_data(_prob.f) && initializealg isa default_inits)
```

I don't see why `CheckInit` would take this branch: if CheckInit then you effectively "do nothing" (after validating it's correct to do nothing)
created_at: 2025-04-21T17:21:13Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,61 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+    kwargs_init = kwargs_adj[Base.diff_names(Base._nt_names(kwargs_adj), (:initializealg,))]
+
+    if haskey(kwargs, :initializealg) || haskey(prob.kwargs, :initializealg)
+        initializealg = haskey(kwargs, :initializealg) ? kwargs[:initializealg] : prob.kwargs[:initializealg]
+    else
+        initializealg = DefaultInit()
+    end
+
+    default_inits = Union{OverrideInit, Nothing, DefaultInit}
+    igs, new_u0, new_p, new_initializealg = if (SciMLBase.has_initialization_data(_prob.f) && initializealg isa default_inits) || initializealg isa CheckInit
path: src/concrete_solve.jl
user: login: DhairyaLGandhi
body: But if we must do something, then, does it run the equivalent of OverrideInit?
created_at: 2025-04-21T17:24:11Z
author_association: MEMBER
diff_hunk: @@ -1688,26 +1740,56 @@ function DiffEqBase._concrete_solve_adjoint(
         out = SciMLBase.sensitivity_solution(sol, sol[_save_idxs])
     end
 
+    _, repack_adjoint = if isscimlstructure(p)
+        Zygote.pullback(p) do p
+            t, _, _ = canonicalize(Tunable(), p)
+            t
+        end
+    else
+        nothing, x -> (x,)
+    end
+
     function steadystatebackpass(Δ)
+        Δ = Δ isa AbstractThunk ? unthunk(Δ) : Δ
         # Δ = dg/dx or diffcache.dg_val
         # del g/del p = 0
         function df(_out, u, p, t, i)
             if _save_idxs isa Number
                 _out[_save_idxs] = Δ[_save_idxs]
             elseif Δ isa Number
                 @. _out[_save_idxs] = Δ
-            else
+            elseif Δ isa AbstractArray{<:AbstractArray} || Δ isa AbstractVectorOfArray || Δ isa AbstractArray
                 @. _out[_save_idxs] = Δ[_save_idxs]
+            elseif isnothing(_out)
+                _out
+            else
+                @. _out[_save_idxs] = Δ.u[_save_idxs]
             end
         end
         dp = adjoint_sensitivities(sol, alg; sensealg = sensealg, dgdu = df)
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: ```suggestion
        dp = adjoint_sensitivities(sol, alg; sensealg = sensealg, dgdu = df, initializealg = BrownBasicInit())
```
created_at: 2025-04-21T17:24:58Z
author_association: MEMBER
diff_hunk: @@ -32,8 +33,73 @@ prob = ODEProblem(sys, u0, tspan, p, jac = true)
 sol = solve(prob, Tsit5())
 mtkparams = SciMLSensitivity.parameter_values(sol)
 
-gt = rand(5501)
+gt = rand(1495)
 dmtk, = Zygote.gradient(mtkparams) do p
-    new_sol = solve(prob, Rosenbrock23(), p = p)
+    new_sol = solve(prob, Tsit5(), p = p)
+    Zygote.ChainRules.ChainRulesCore.ignore_derivatives() do
+        @test all(isapprox.(new_sol[x + y + z + 2 * β - w], 0, atol = 1e-12))
+    end
     mean(abs.(new_sol[sys.x] .- gt))
 end
+
+# Force DAE handling for Initialization
+
+eqs = [D(D(x)) ~ σ * (y - x),
+    D(y) ~ x * (ρ - z) - y,
+    D(z) ~ x * y - β * z,
+    w ~ x + y + z + 2 * β,
+    0 ~ x^2 + y^2 - w2^2
+]
+
+@mtkbuild sys = ODESystem(eqs, t)
+
+u0_incorrect = [D(x) => 2.0,
+    x => 1.0,
+    y => 0.0,
+    z => 0.0,
+    w2 => 0.0,]
+
+p = [σ => 28.0,
+    ρ => 10.0,
+    β => 8 / 3]
+
+tspan = (0.0, 100.0)
+
+# Check that the gradients for the `solve` are the same both for an initialization
+# (for the algebraic variables) initialized poorly (therefore needs correction with BrownBasicInit)
+# and with the initialization corrected to satisfy the algebraic equation
+prob_incorrectu0 = ODEProblem(sys, u0_incorrect, tspan, p, jac = true)
+mtkparams_incorrectu0 = SciMLSensitivity.parameter_values(prob_incorrectu0)
+
+gt = rand(3609)
+
+sensealg = GaussAdjoint(; autojacvec = SciMLSensitivity.ZygoteVJP())
+dmtk_incorrectu0, = Zygote.gradient(mtkparams_incorrectu0) do p
+    new_sol = solve(prob_incorrectu0, Rodas5P(); p = p, initializealg = BrownFullBasicInit(), sensealg, abstol = 1e-6, reltol = 1e-3)
+    Zygote.ChainRules.ChainRulesCore.ignore_derivatives() do
+        @test new_sol.retcode == SciMLBase.ReturnCode.Success
+        @test all(isapprox.(new_sol[x + y + z + 2 * β - w], 0, atol = 1e-12))
+        @test all(isapprox.(new_sol[x^2 + y^2 - w2^2], 0, atol = 1e-5, rtol = 1e0))
+    end
+    mean(abs.(new_sol[sys.x] .- gt))
+end
path: test/mtk.jl
user: login: ChrisRackauckas
body: ```suggestion
dmtk_overrideinit_incorrectu0, = Zygote.gradient(mtkparams_incorrectu0) do p
    new_sol = solve(prob_incorrectu0, Rodas5P(); p = p, initializealg = OverrideInit(), sensealg, abstol = 1e-6, reltol = 1e-3)
    Zygote.ChainRules.ChainRulesCore.ignore_derivatives() do
        @test new_sol.retcode == SciMLBase.ReturnCode.Success
        @test all(isapprox.(new_sol[x + y + z + 2 * β - w], 0, atol = 1e-12))
        @test all(isapprox.(new_sol[x^2 + y^2 - w2^2], 0, atol = 1e-5, rtol = 1e0))
    end
    mean(abs.(new_sol[sys.x] .- gt))
end

dmtk_incorrectu0, = Zygote.gradient(mtkparams_incorrectu0) do p
    new_sol = solve(prob_incorrectu0, Rodas5P(); p = p, initializealg = BrownFullBasicInit(), sensealg, abstol = 1e-6, reltol = 1e-3)
    Zygote.ChainRules.ChainRulesCore.ignore_derivatives() do
        @test new_sol.retcode == SciMLBase.ReturnCode.Success
        @test all(isapprox.(new_sol[x + y + z + 2 * β - w], 0, atol = 1e-12))
        @test all(isapprox.(new_sol[x^2 + y^2 - w2^2], 0, atol = 1e-5, rtol = 1e0))
    end
    mean(abs.(new_sol[sys.x] .- gt))
end
```
created_at: 2025-04-21T17:27:08Z
author_association: MEMBER
diff_hunk: @@ -32,8 +33,73 @@ prob = ODEProblem(sys, u0, tspan, p, jac = true)
 sol = solve(prob, Tsit5())
 mtkparams = SciMLSensitivity.parameter_values(sol)
 
-gt = rand(5501)
+gt = rand(1495)
 dmtk, = Zygote.gradient(mtkparams) do p
-    new_sol = solve(prob, Rosenbrock23(), p = p)
+    new_sol = solve(prob, Tsit5(), p = p)
+    Zygote.ChainRules.ChainRulesCore.ignore_derivatives() do
+        @test all(isapprox.(new_sol[x + y + z + 2 * β - w], 0, atol = 1e-12))
+    end
     mean(abs.(new_sol[sys.x] .- gt))
 end
+
+# Force DAE handling for Initialization
+
+eqs = [D(D(x)) ~ σ * (y - x),
+    D(y) ~ x * (ρ - z) - y,
+    D(z) ~ x * y - β * z,
+    w ~ x + y + z + 2 * β,
+    0 ~ x^2 + y^2 - w2^2
+]
+
+@mtkbuild sys = ODESystem(eqs, t)
+
+u0_incorrect = [D(x) => 2.0,
+    x => 1.0,
+    y => 0.0,
+    z => 0.0,
+    w2 => 0.0,]
+
+p = [σ => 28.0,
+    ρ => 10.0,
+    β => 8 / 3]
+
+tspan = (0.0, 100.0)
+
+# Check that the gradients for the `solve` are the same both for an initialization
+# (for the algebraic variables) initialized poorly (therefore needs correction with BrownBasicInit)
+# and with the initialization corrected to satisfy the algebraic equation
+prob_incorrectu0 = ODEProblem(sys, u0_incorrect, tspan, p, jac = true)
+mtkparams_incorrectu0 = SciMLSensitivity.parameter_values(prob_incorrectu0)
+
+gt = rand(3609)
+
+sensealg = GaussAdjoint(; autojacvec = SciMLSensitivity.ZygoteVJP())
+dmtk_incorrectu0, = Zygote.gradient(mtkparams_incorrectu0) do p
+    new_sol = solve(prob_incorrectu0, Rodas5P(); p = p, initializealg = BrownFullBasicInit(), sensealg, abstol = 1e-6, reltol = 1e-3)
+    Zygote.ChainRules.ChainRulesCore.ignore_derivatives() do
+        @test new_sol.retcode == SciMLBase.ReturnCode.Success
+        @test all(isapprox.(new_sol[x + y + z + 2 * β - w], 0, atol = 1e-12))
+        @test all(isapprox.(new_sol[x^2 + y^2 - w2^2], 0, atol = 1e-5, rtol = 1e0))
+    end
+    mean(abs.(new_sol[sys.x] .- gt))
+end
path: test/mtk.jl
user: login: ChrisRackauckas
body: We should test OverrideInit and DefaultInit
created_at: 2025-04-21T17:27:26Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,61 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+    kwargs_init = kwargs_adj[Base.diff_names(Base._nt_names(kwargs_adj), (:initializealg,))]
+
+    if haskey(kwargs, :initializealg) || haskey(prob.kwargs, :initializealg)
+        initializealg = haskey(kwargs, :initializealg) ? kwargs[:initializealg] : prob.kwargs[:initializealg]
+    else
+        initializealg = DefaultInit()
+    end
+
+    default_inits = Union{OverrideInit, Nothing, DefaultInit}
+    igs, new_u0, new_p, new_initializealg = if (SciMLBase.has_initialization_data(_prob.f) && initializealg isa default_inits) || initializealg isa CheckInit
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: If the init is not correct out of the gate, it errors. So CheckInit is just, check if we need to initialize and if we do, error.
created_at: 2025-04-21T17:30:49Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,61 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+    kwargs_init = kwargs_adj[Base.diff_names(Base._nt_names(kwargs_adj), (:initializealg,))]
+
+    if haskey(kwargs, :initializealg) || haskey(prob.kwargs, :initializealg)
+        initializealg = haskey(kwargs, :initializealg) ? kwargs[:initializealg] : prob.kwargs[:initializealg]
+    else
+        initializealg = DefaultInit()
+    end
+
+    default_inits = Union{OverrideInit, Nothing, DefaultInit}
+    igs, new_u0, new_p, new_initializealg = if (SciMLBase.has_initialization_data(_prob.f) && initializealg isa default_inits) || initializealg isa CheckInit
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: @AayushSabharwal there isn't a custom CheckInit path right?
created_at: 2025-04-21T17:31:13Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,61 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+    kwargs_init = kwargs_adj[Base.diff_names(Base._nt_names(kwargs_adj), (:initializealg,))]
+
+    if haskey(kwargs, :initializealg) || haskey(prob.kwargs, :initializealg)
+        initializealg = haskey(kwargs, :initializealg) ? kwargs[:initializealg] : prob.kwargs[:initializealg]
+    else
+        initializealg = DefaultInit()
+    end
+
+    default_inits = Union{OverrideInit, Nothing, DefaultInit}
+    igs, new_u0, new_p, new_initializealg = if (SciMLBase.has_initialization_data(_prob.f) && initializealg isa default_inits) || initializealg isa CheckInit
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: https://github.com/SciML/SciMLBase.jl/blob/master/src/initialization.jl#L192 there is, but the derivative is still guaranteed to be zero.
created_at: 2025-04-21T17:32:14Z
author_association: MEMBER
diff_hunk: @@ -32,8 +33,73 @@ prob = ODEProblem(sys, u0, tspan, p, jac = true)
 sol = solve(prob, Tsit5())
 mtkparams = SciMLSensitivity.parameter_values(sol)
 
-gt = rand(5501)
+gt = rand(1495)
 dmtk, = Zygote.gradient(mtkparams) do p
-    new_sol = solve(prob, Rosenbrock23(), p = p)
+    new_sol = solve(prob, Tsit5(), p = p)
+    Zygote.ChainRules.ChainRulesCore.ignore_derivatives() do
+        @test all(isapprox.(new_sol[x + y + z + 2 * β - w], 0, atol = 1e-12))
+    end
     mean(abs.(new_sol[sys.x] .- gt))
 end
+
+# Force DAE handling for Initialization
+
+eqs = [D(D(x)) ~ σ * (y - x),
+    D(y) ~ x * (ρ - z) - y,
+    D(z) ~ x * y - β * z,
+    w ~ x + y + z + 2 * β,
+    0 ~ x^2 + y^2 - w2^2
+]
+
+@mtkbuild sys = ODESystem(eqs, t)
+
+u0_incorrect = [D(x) => 2.0,
+    x => 1.0,
+    y => 0.0,
+    z => 0.0,
+    w2 => 0.0,]
+
+p = [σ => 28.0,
+    ρ => 10.0,
+    β => 8 / 3]
+
+tspan = (0.0, 100.0)
+
+# Check that the gradients for the `solve` are the same both for an initialization
+# (for the algebraic variables) initialized poorly (therefore needs correction with BrownBasicInit)
+# and with the initialization corrected to satisfy the algebraic equation
+prob_incorrectu0 = ODEProblem(sys, u0_incorrect, tspan, p, jac = true)
+mtkparams_incorrectu0 = SciMLSensitivity.parameter_values(prob_incorrectu0)
+
+gt = rand(3609)
+
+sensealg = GaussAdjoint(; autojacvec = SciMLSensitivity.ZygoteVJP())
+dmtk_incorrectu0, = Zygote.gradient(mtkparams_incorrectu0) do p
+    new_sol = solve(prob_incorrectu0, Rodas5P(); p = p, initializealg = BrownFullBasicInit(), sensealg, abstol = 1e-6, reltol = 1e-3)
+    Zygote.ChainRules.ChainRulesCore.ignore_derivatives() do
+        @test new_sol.retcode == SciMLBase.ReturnCode.Success
+        @test all(isapprox.(new_sol[x + y + z + 2 * β - w], 0, atol = 1e-12))
+        @test all(isapprox.(new_sol[x^2 + y^2 - w2^2], 0, atol = 1e-5, rtol = 1e0))
+    end
+    mean(abs.(new_sol[sys.x] .- gt))
+end
+
+u0_correct = [D(x) => 2.0,
+    x => 1.0,
+    y => 0.0,
path: test/mtk.jl
user: login: ChrisRackauckas
body: ```suggestion
    y => t,
```

This should stress the BrownBasicInit part that I suggested above, as using the MTK init would then start with a vastly wrong value of `y`
created_at: 2025-04-21T17:36:43Z
author_association: MEMBER
diff_hunk: @@ -32,8 +33,73 @@ prob = ODEProblem(sys, u0, tspan, p, jac = true)
 sol = solve(prob, Tsit5())
 mtkparams = SciMLSensitivity.parameter_values(sol)
 
-gt = rand(5501)
+gt = rand(1495)
 dmtk, = Zygote.gradient(mtkparams) do p
-    new_sol = solve(prob, Rosenbrock23(), p = p)
+    new_sol = solve(prob, Tsit5(), p = p)
+    Zygote.ChainRules.ChainRulesCore.ignore_derivatives() do
+        @test all(isapprox.(new_sol[x + y + z + 2 * β - w], 0, atol = 1e-12))
+    end
     mean(abs.(new_sol[sys.x] .- gt))
 end
+
+# Force DAE handling for Initialization
+
+eqs = [D(D(x)) ~ σ * (y - x),
+    D(y) ~ x * (ρ - z) - y,
+    D(z) ~ x * y - β * z,
+    w ~ x + y + z + 2 * β,
+    0 ~ x^2 + y^2 - w2^2
+]
+
+@mtkbuild sys = ODESystem(eqs, t)
+
+u0_incorrect = [D(x) => 2.0,
+    x => 1.0,
+    y => 0.0,
path: test/mtk.jl
user: login: ChrisRackauckas
body: ```suggestion
    y => t,
```
created_at: 2025-04-21T17:37:00Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,56 @@
+using ModelingToolkit, OrdinaryDiffEq
+using ModelingToolkitStandardLibrary.Electrical
+using ModelingToolkitStandardLibrary.Blocks: Sine
+using NonlinearSolve
+import SciMLStructures as SS
+import SciMLSensitivity
+using SymbolicIndexingInterface
+import ModelingToolkit as MTK
+using Zygote
+
+function create_model(; C₁ = 3e-5, C₂ = 1e-6)
+    @variables t
+    @named resistor1 = Resistor(R = 5.0)
+    @named resistor2 = Resistor(R = 2.0)
+    @named capacitor1 = Capacitor(C = C₁)
+    @named capacitor2 = Capacitor(C = C₂)
+    @named source = Voltage()
+    @named input_signal = Sine(frequency = 100.0)
+    @named ground = Ground()
+    @named ampermeter = CurrentSensor()
+
+    eqs = [connect(input_signal.output, source.V)
+        connect(source.p, capacitor1.n, capacitor2.n)
+        connect(source.n, resistor1.p, resistor2.p, ground.g)
+        connect(resistor1.n, capacitor1.p, ampermeter.n)
+        connect(resistor2.n, capacitor2.p, ampermeter.p)]
+
+    @named circuit_model = ODESystem(eqs, t,
+        systems = [
+            resistor1, resistor2, capacitor1, capacitor2,
+            source, input_signal, ground, ampermeter,
+        ])
+end
+
+desauty_model = create_model()
+sys = structural_simplify(desauty_model)
+
+
+prob = ODEProblem(sys, [], (0.0, 0.1), guesses = [sys.resistor1.v => 1.])
+iprob = prob.f.initialization_data.initializeprob
+isys = iprob.f.sys
+
+tunables, repack, aliases = SS.canonicalize(SS.Tunable(), parameter_values(iprob))
+
+linsolve = LinearSolve.DefaultLinearSolver(LinearSolve.DefaultAlgorithmChoice.QRFactorization)
+sensealg = SciMLSensitivity.SteadyStateAdjoint(autojacvec = SciMLSensitivity.ZygoteVJP())
+# sensealg = SciMLSensitivity.SteadyStateAdjoint(autojacvec = SciMLSensitivity.ZygoteVJP(), linsolve = linsolve)
+igs, = Zygote.gradient(tunables) do p
+    iprob2 = remake(iprob, p = repack(p))
+    sol = solve(iprob2,
+                sensealg = sensealg
path: test/desauty_dae_mwe.jl
user: login: ChrisRackauckas
body: This is an ODE though, why is SteadyStateAdjoint used here if it's not a steady state ODE?
created_at: 2025-04-21T18:13:45Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,56 @@
+using ModelingToolkit, OrdinaryDiffEq
+using ModelingToolkitStandardLibrary.Electrical
+using ModelingToolkitStandardLibrary.Blocks: Sine
+using NonlinearSolve
+import SciMLStructures as SS
+import SciMLSensitivity
+using SymbolicIndexingInterface
+import ModelingToolkit as MTK
+using Zygote
+
+function create_model(; C₁ = 3e-5, C₂ = 1e-6)
+    @variables t
+    @named resistor1 = Resistor(R = 5.0)
+    @named resistor2 = Resistor(R = 2.0)
+    @named capacitor1 = Capacitor(C = C₁)
+    @named capacitor2 = Capacitor(C = C₂)
+    @named source = Voltage()
+    @named input_signal = Sine(frequency = 100.0)
+    @named ground = Ground()
+    @named ampermeter = CurrentSensor()
+
+    eqs = [connect(input_signal.output, source.V)
+        connect(source.p, capacitor1.n, capacitor2.n)
+        connect(source.n, resistor1.p, resistor2.p, ground.g)
+        connect(resistor1.n, capacitor1.p, ampermeter.n)
+        connect(resistor2.n, capacitor2.p, ampermeter.p)]
+
+    @named circuit_model = ODESystem(eqs, t,
+        systems = [
+            resistor1, resistor2, capacitor1, capacitor2,
+            source, input_signal, ground, ampermeter,
+        ])
+end
+
+desauty_model = create_model()
+sys = structural_simplify(desauty_model)
+
+
+prob = ODEProblem(sys, [], (0.0, 0.1), guesses = [sys.resistor1.v => 1.])
+iprob = prob.f.initialization_data.initializeprob
+isys = iprob.f.sys
+
+tunables, repack, aliases = SS.canonicalize(SS.Tunable(), parameter_values(iprob))
+
+linsolve = LinearSolve.DefaultLinearSolver(LinearSolve.DefaultAlgorithmChoice.QRFactorization)
+sensealg = SciMLSensitivity.SteadyStateAdjoint(autojacvec = SciMLSensitivity.ZygoteVJP())
+# sensealg = SciMLSensitivity.SteadyStateAdjoint(autojacvec = SciMLSensitivity.ZygoteVJP(), linsolve = linsolve)
+igs, = Zygote.gradient(tunables) do p
+    iprob2 = remake(iprob, p = repack(p))
+    sol = solve(iprob2,
+                sensealg = sensealg
path: test/desauty_dae_mwe.jl
user: login: DhairyaLGandhi
body: `iprob` is the initialization problem
created_at: 2025-04-21T19:53:09Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,56 @@
+using ModelingToolkit, OrdinaryDiffEq
+using ModelingToolkitStandardLibrary.Electrical
+using ModelingToolkitStandardLibrary.Blocks: Sine
+using NonlinearSolve
+import SciMLStructures as SS
+import SciMLSensitivity
+using SymbolicIndexingInterface
+import ModelingToolkit as MTK
+using Zygote
+
+function create_model(; C₁ = 3e-5, C₂ = 1e-6)
+    @variables t
+    @named resistor1 = Resistor(R = 5.0)
+    @named resistor2 = Resistor(R = 2.0)
+    @named capacitor1 = Capacitor(C = C₁)
+    @named capacitor2 = Capacitor(C = C₂)
+    @named source = Voltage()
+    @named input_signal = Sine(frequency = 100.0)
+    @named ground = Ground()
+    @named ampermeter = CurrentSensor()
+
+    eqs = [connect(input_signal.output, source.V)
+        connect(source.p, capacitor1.n, capacitor2.n)
+        connect(source.n, resistor1.p, resistor2.p, ground.g)
+        connect(resistor1.n, capacitor1.p, ampermeter.n)
+        connect(resistor2.n, capacitor2.p, ampermeter.p)]
+
+    @named circuit_model = ODESystem(eqs, t,
+        systems = [
+            resistor1, resistor2, capacitor1, capacitor2,
+            source, input_signal, ground, ampermeter,
+        ])
+end
+
+desauty_model = create_model()
+sys = structural_simplify(desauty_model)
+
+
+prob = ODEProblem(sys, [], (0.0, 0.1), guesses = [sys.resistor1.v => 1.])
+iprob = prob.f.initialization_data.initializeprob
+isys = iprob.f.sys
+
+tunables, repack, aliases = SS.canonicalize(SS.Tunable(), parameter_values(iprob))
+
+linsolve = LinearSolve.DefaultLinearSolver(LinearSolve.DefaultAlgorithmChoice.QRFactorization)
+sensealg = SciMLSensitivity.SteadyStateAdjoint(autojacvec = SciMLSensitivity.ZygoteVJP())
+# sensealg = SciMLSensitivity.SteadyStateAdjoint(autojacvec = SciMLSensitivity.ZygoteVJP(), linsolve = linsolve)
+igs, = Zygote.gradient(tunables) do p
+    iprob2 = remake(iprob, p = repack(p))
+    sol = solve(iprob2,
+                sensealg = sensealg
path: test/desauty_dae_mwe.jl
user: login: ChrisRackauckas
body: oh I see.
created_at: 2025-04-22T03:35:03Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,61 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+    kwargs_init = kwargs_adj[Base.diff_names(Base._nt_names(kwargs_adj), (:initializealg,))]
+
+    if haskey(kwargs, :initializealg) || haskey(prob.kwargs, :initializealg)
+        initializealg = haskey(kwargs, :initializealg) ? kwargs[:initializealg] : prob.kwargs[:initializealg]
+    else
+        initializealg = DefaultInit()
+    end
+
+    default_inits = Union{OverrideInit, Nothing, DefaultInit}
+    igs, new_u0, new_p, new_initializealg = if (SciMLBase.has_initialization_data(_prob.f) && initializealg isa default_inits)
+        local new_u0
+        local new_p
+        initializeprob = prob.f.initialization_data.initializeprob
+        iu0 = state_values(initializeprob)
+        isAD = if iu0 === nothing
+            AutoForwardDiff
+        elseif has_autodiff(alg)
+            OrdinaryDiffEqCore.alg_autodiff(alg) isa AutoForwardDiff
+        else
+            true 
+        end
+        nlsolve_alg = default_nlsolve(nothing, Val(isinplace(_prob)), iu0, initializeprob, isAD)
path: src/concrete_solve.jl
user: login: AayushSabharwal
body: @ChrisRackauckas are `alg_autodiff`, `has_autodiff` and `default_nlsolve` public API of ODECore? If yes, we should PR to ODECore to (at the very least) add comments saying so. Or get around to adding `SciMLPublic.jl` with an `@public` macro.
created_at: 2025-04-22T11:45:14Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,99 @@
+using ModelingToolkit
+using ModelingToolkit: t_nounits as t, D_nounits as D
+using SymbolicIndexingInterface
+using SciMLStructures
+import SymbolicIndexingInterface as SII
+import SciMLStructures as SS
+import ModelingToolkit as MTK
+using SciMLSensitivity
+using OrdinaryDiffEq
+using Zygote
+using Test
+
+@parameters σ ρ β
+@variables x(t) y(t) z(t) w(t)
+
+eqs = [D(D(x)) ~ σ * (y - x), 
+    D(y) ~ x * (ρ - z) - y,
+    D(z) ~ x * y - β * z,
+    w ~ x + y + z + 2 * β]
+
+@mtkbuild sys = ODESystem(eqs, t)
+
+u0 = [D(x) => 2.0,
+    x => 1.0,
+    y => 0.0,
+    z => 0.0]
+
+p = [σ => 28.0,
+    ρ => 10.0,
+    β => 8 / 3]
+
+tspan = (0.0, 100.0)
+prob = ODEProblem(sys, u0, tspan, p, jac = true)
path: test/parameter_initialization.jl
user: login: AayushSabharwal
body: There isn't actually any parameter initialization being run here?
created_at: 2025-04-22T11:49:11Z
author_association: MEMBER
diff_hunk: @@ -412,16 +415,61 @@ function DiffEqBase._concrete_solve_adjoint(
         Base.diff_names(Base._nt_names(values(kwargs)),
         (:callback_adj, :callback))}(values(kwargs))
     isq = sensealg isa QuadratureAdjoint
+    kwargs_init = kwargs_adj[Base.diff_names(Base._nt_names(kwargs_adj), (:initializealg,))]
+
+    if haskey(kwargs, :initializealg) || haskey(prob.kwargs, :initializealg)
+        initializealg = haskey(kwargs, :initializealg) ? kwargs[:initializealg] : prob.kwargs[:initializealg]
+    else
+        initializealg = DefaultInit()
+    end
+
+    default_inits = Union{OverrideInit, Nothing, DefaultInit}
+    igs, new_u0, new_p, new_initializealg = if (SciMLBase.has_initialization_data(_prob.f) && initializealg isa default_inits)
+        local new_u0
+        local new_p
+        initializeprob = prob.f.initialization_data.initializeprob
+        iu0 = state_values(initializeprob)
+        isAD = if iu0 === nothing
+            AutoForwardDiff
+        elseif has_autodiff(alg)
+            OrdinaryDiffEqCore.alg_autodiff(alg) isa AutoForwardDiff
+        else
+            true 
+        end
+        nlsolve_alg = default_nlsolve(nothing, Val(isinplace(_prob)), iu0, initializeprob, isAD)
path: src/concrete_solve.jl
user: login: ChrisRackauckas
body: They have at least been used downstream here for a long while. We should comment that for sure.

created_at: 2025-04-22T12:04:48Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,99 @@
+using ModelingToolkit
+using ModelingToolkit: t_nounits as t, D_nounits as D
+using SymbolicIndexingInterface
+using SciMLStructures
+import SymbolicIndexingInterface as SII
+import SciMLStructures as SS
+import ModelingToolkit as MTK
+using SciMLSensitivity
+using OrdinaryDiffEq
+using Zygote
+using Test
+
+@parameters σ ρ β
+@variables x(t) y(t) z(t) w(t)
+
+eqs = [D(D(x)) ~ σ * (y - x), 
+    D(y) ~ x * (ρ - z) - y,
+    D(z) ~ x * y - β * z,
+    w ~ x + y + z + 2 * β]
+
+@mtkbuild sys = ODESystem(eqs, t)
+
+u0 = [D(x) => 2.0,
+    x => 1.0,
+    y => 0.0,
+    z => 0.0]
+
+p = [σ => 28.0,
+    ρ => 10.0,
+    β => 8 / 3]
+
+tspan = (0.0, 100.0)
+prob = ODEProblem(sys, u0, tspan, p, jac = true)
path: test/parameter_initialization.jl
user: login: ChrisRackauckas
body: Yeah there's an older version of the test that was overwritten, this one should be reverted.
created_at: 2025-04-22T14:33:15Z
author_association: MEMBER
diff_hunk: @@ -32,8 +34,98 @@ prob = ODEProblem(sys, u0, tspan, p, jac = true)
 sol = solve(prob, Tsit5())
 mtkparams = SciMLSensitivity.parameter_values(sol)
 
-gt = rand(5501)
+gt = rand(1495)
 dmtk, = Zygote.gradient(mtkparams) do p
-    new_sol = solve(prob, Rosenbrock23(), p = p)
+    new_sol = solve(prob, Tsit5(), p = p)
+    Zygote.ChainRules.ChainRulesCore.ignore_derivatives() do
+        @test all(isapprox.(new_sol[x + y + z + 2 * β - w], 0, atol = 1e-12))
+    end
     mean(abs.(new_sol[sys.x] .- gt))
 end
+
+# Force DAE handling for Initialization
+
+eqs = [D(D(x)) ~ σ * (y - x),
+    D(y) ~ x * (ρ - z) - y,
+    D(z) ~ x * y - β * z,
+    w ~ x + y + z + 2 * β,
+    0 ~ x^2 + y^2 - w2^2
+]
+
+@mtkbuild sys = ODESystem(eqs, t)
+
+u0_incorrect = [D(x) => 2.0,
+    x => 1.0,
+    y => 0.0,
+    z => 0.0,
+    w2 => 0.0,]
+
+p = [σ => 28.0,
+    ρ => 10.0,
+    β => 8 / 3]
+
+tspan = (0.0, 100.0)
+
+# Check that the gradients for the `solve` are the same both for an initialization
+# (for the algebraic variables) initialized poorly (therefore needs correction with BrownBasicInit)
+# and with the initialization corrected to satisfy the algebraic equation
+prob_incorrectu0 = ODEProblem(sys, u0_incorrect, tspan, p, jac = true)
+mtkparams_incorrectu0 = SciMLSensitivity.parameter_values(prob_incorrectu0)
+
+u0_timedep = [D(x) => 2.0,
+    x => 1.0,
+    y => t,
+    z => 0.0,
+    w2 => 0.0,]
+# this ensures that `y => t` is not applied in the adjoint equation
+# If the MTK init is called for the reverse, then `y0` in the backwards
+# pass will be extremely far off and cause an incorrect gradient
+prob_timedepu0 = ODEProblem(sys, u0_timedep, tspan, p, jac = true)
+mtkparams_timedepu0 = SciMLSensitivity.parameter_values(prob_incorrectu0)
+
+u0_correct = [D(x) => 2.0,
+    x => 1.0,
+    y => 0.0,
+    z => 0.0,
+    w2 => -1.0,]
+prob_correctu0 = remake(prob_incorrectu0, u0 = u0_correct)
+mtkparams_correctu0 = SciMLSensitivity.parameter_values(prob_correctu0)
+
+sensealg = GaussAdjoint(; autojacvec = SciMLSensitivity.ZygoteVJP())
+
+setups = [
+          (prob_correctu0, mtkparams_correctu0, CheckInit()), # Source of truth first
+    
+          (prob_incorrectu0, mtkparams_incorrectu0, BrownFullBasicInit()),
+          (prob_incorrectu0, mtkparams_incorrectu0, OrdinaryDiffEqCore.DefaultInit()),
+          (prob_incorrectu0, mtkparams_incorrectu0, nothing), 
+
+          (prob_timedepu0, mtkparams_timedepu0, BrownFullBasicInit()),
+          (prob_timedepu0, mtkparams_timedepu0, OrdinaryDiffEqCore.DefaultInit()),
+          (prob_timedepu0, mtkparams_timedepu0, nothing), 
+
+          (prob_correctu0, mtkparams_correctu0, BrownFullBasicInit()),
+          (prob_correctu0, mtkparams_correctu0, OrdinaryDiffEqCore.DefaultInit()),
+          
+          (prob_correctu0, mtkparams_correctu0, NoInit()), 
+          (prob_correctu0, mtkparams_correctu0, nothing), 
+]
+
+grads = map(setups) do setup
+    prob, ps, init = setup
+    @show init
+    Zygote.gradient(ps) do p
+        new_sol = solve(prob, Rodas5P(); p = p, initializealg = init, sensealg, abstol = 1e-6, reltol = 1e-3)
path: test/mtk.jl
user: login: ChrisRackauckas
body: ```suggestion
        if init === nothing
            new_sol = solve(prob, Rodas5P(); p = p, sensealg, abstol = 1e-6, reltol = 1e-3)
        else
            new_sol = solve(prob, Rodas5P(); p = p, initializealg = init, sensealg, abstol = 1e-6, reltol = 1e-3)
        end
```
created_at: 2025-04-23T18:05:48Z
author_association: MEMBER
diff_hunk: @@ -32,8 +34,116 @@ prob = ODEProblem(sys, u0, tspan, p, jac = true)
 sol = solve(prob, Tsit5())
 mtkparams = SciMLSensitivity.parameter_values(sol)
 
-gt = rand(5501)
+gt = rand(1495)
 dmtk, = Zygote.gradient(mtkparams) do p
-    new_sol = solve(prob, Rosenbrock23(), p = p)
+    new_sol = solve(prob, Tsit5(), p = p)
+    Zygote.ChainRules.ChainRulesCore.ignore_derivatives() do
+        @test all(isapprox.(new_sol[x + y + z + 2 * β - w], 0, atol = 1e-12))
+    end
     mean(abs.(new_sol[sys.x] .- gt))
 end
+
+# Force DAE handling for Initialization
+
+eqs = [D(D(x)) ~ σ * (y - x),
+    D(y) ~ x * (ρ - z) - y,
+    D(z) ~ x * y - β * z,
+    w ~ x + y + z + 2 * β,
+    0 ~ x^2 + y^2 - w2^2
+]
+
+@mtkbuild sys = ODESystem(eqs, t)
+
+u0_incorrect = [D(x) => 2.0,
+    x => 1.0,
+    y => 0.0,
+    z => 0.0]
+
+p = [σ => 28.0,
+    ρ => 10.0,
+    β => 8 / 3]
+
+tspan = (0.0, 100.0)
+
+# Check that the gradients for the `solve` are the same both for an initialization
+# (for the algebraic variables) initialized poorly (therefore needs correction with BrownBasicInit)
+# and with the initialization corrected to satisfy the algebraic equation
+prob_incorrectu0 = ODEProblem(sys, u0_incorrect, tspan, p, jac = true, guesses = [w2 => 0.0])
+mtkparams_incorrectu0 = SciMLSensitivity.parameter_values(prob_incorrectu0)
+
+u0_timedep = [D(x) => 2.0,
+    x => 1.0,
+    y => t,
+    z => 0.0,
+    w2 => 0.0,]
+# this ensures that `y => t` is not applied in the adjoint equation
+# If the MTK init is called for the reverse, then `y0` in the backwards
+# pass will be extremely far off and cause an incorrect gradient
+prob_timedepu0 = ODEProblem(sys, u0_timedep, tspan, p, jac = true, guesses = [w2 => 0.0])
+mtkparams_timedepu0 = SciMLSensitivity.parameter_values(prob_incorrectu0)
+
+u0_correct = [D(x) => 2.0,
+    x => 1.0,
+    y => 0.0,
+    z => 0.0,
+    w2 => -1.0,]
+prob_correctu0 = ODEProblem(sys, u0_correct, tspan, p, jac = true, guesses = [w2 => -1.0])
+mtkparams_correctu0 = SciMLSensitivity.parameter_values(prob_correctu0)
+prob_correctu0.u0[5] = -1.0
path: test/mtk.jl
user: login: ChrisRackauckas
body: @AayushSabharwal note that this is the issue of not passing over the initial values and guess values to u0. Without this `CheckInit` will fail. It's a minor issue but it is something we should handle over time.
created_at: 2025-04-23T21:02:04Z
author_association: MEMBER
diff_hunk: @@ -32,8 +34,116 @@ prob = ODEProblem(sys, u0, tspan, p, jac = true)
 sol = solve(prob, Tsit5())
 mtkparams = SciMLSensitivity.parameter_values(sol)
 
-gt = rand(5501)
+gt = rand(1495)
 dmtk, = Zygote.gradient(mtkparams) do p
-    new_sol = solve(prob, Rosenbrock23(), p = p)
+    new_sol = solve(prob, Tsit5(), p = p)
+    Zygote.ChainRules.ChainRulesCore.ignore_derivatives() do
+        @test all(isapprox.(new_sol[x + y + z + 2 * β - w], 0, atol = 1e-12))
+    end
     mean(abs.(new_sol[sys.x] .- gt))
 end
+
+# Force DAE handling for Initialization
+
+eqs = [D(D(x)) ~ σ * (y - x),
+    D(y) ~ x * (ρ - z) - y,
+    D(z) ~ x * y - β * z,
+    w ~ x + y + z + 2 * β,
+    0 ~ x^2 + y^2 - w2^2
+]
+
+@mtkbuild sys = ODESystem(eqs, t)
+
+u0_incorrect = [D(x) => 2.0,
+    x => 1.0,
+    y => 0.0,
+    z => 0.0]
+
+p = [σ => 28.0,
+    ρ => 10.0,
+    β => 8 / 3]
+
+tspan = (0.0, 100.0)
+
+# Check that the gradients for the `solve` are the same both for an initialization
+# (for the algebraic variables) initialized poorly (therefore needs correction with BrownBasicInit)
+# and with the initialization corrected to satisfy the algebraic equation
+prob_incorrectu0 = ODEProblem(sys, u0_incorrect, tspan, p, jac = true, guesses = [w2 => 0.0])
+mtkparams_incorrectu0 = SciMLSensitivity.parameter_values(prob_incorrectu0)
+
+u0_timedep = [D(x) => 2.0,
+    x => 1.0,
+    y => t,
+    z => 0.0,
+    w2 => 0.0,]
+# this ensures that `y => t` is not applied in the adjoint equation
+# If the MTK init is called for the reverse, then `y0` in the backwards
+# pass will be extremely far off and cause an incorrect gradient
+prob_timedepu0 = ODEProblem(sys, u0_timedep, tspan, p, jac = true, guesses = [w2 => 0.0])
+mtkparams_timedepu0 = SciMLSensitivity.parameter_values(prob_incorrectu0)
+
+u0_correct = [D(x) => 2.0,
+    x => 1.0,
+    y => 0.0,
+    z => 0.0,
+    w2 => -1.0,]
+prob_correctu0 = ODEProblem(sys, u0_correct, tspan, p, jac = true, guesses = [w2 => -1.0])
+mtkparams_correctu0 = SciMLSensitivity.parameter_values(prob_correctu0)
+prob_correctu0.u0[5] = -1.0
path: test/mtk.jl
user: login: AayushSabharwal
body: Yeah I'm working on using the hook in DiffEqBase. It's easy enough to implement, but _somehow_ the returned problem's u0 isn't being respected
created_at: 2025-04-24T04:49:08Z
author_association: MEMBER
diff_hunk: @@ -32,8 +34,119 @@ prob = ODEProblem(sys, u0, tspan, p, jac = true)
 sol = solve(prob, Tsit5())
 mtkparams = SciMLSensitivity.parameter_values(sol)
 
-gt = rand(5501)
+gt = rand(1495)
path: test/mtk.jl
user: login: ChrisRackauckas
body: ```suggestion
gt = rand(length(sol.u))
```
created_at: 2025-04-24T09:47:54Z
author_association: MEMBER
diff_hunk: @@ -737,16 +749,16 @@ function out_and_ts(_ts, duplicate_iterator_times, sol)
     return out, ts
 end
 
-if !hasmethod(Zygote.adjoint,
-    Tuple{Zygote.AContext, typeof(Zygote.literal_getproperty),
-        SciMLBase.AbstractTimeseriesSolution, Val{:u}})
-    Zygote.@adjoint function Zygote.literal_getproperty(sol::AbstractTimeseriesSolution,
-            ::Val{:u})
-        function solu_adjoint(Δ)
-            zerou = zero(sol.prob.u0)
-            _Δ = @. ifelse(Δ === nothing, (zerou,), Δ)
-            (SciMLBase.build_solution(sol.prob, sol.alg, sol.t, _Δ),)
-        end
-        sol.u, solu_adjoint
-    end
-end
+# if !hasmethod(Zygote.adjoint,
+#     Tuple{Zygote.AContext, typeof(Zygote.literal_getproperty),
+#         SciMLBase.AbstractTimeseriesSolution, Val{:u}})
+#     Zygote.@adjoint function Zygote.literal_getproperty(sol::AbstractTimeseriesSolution,
+#             ::Val{:u})
+#         function solu_adjoint(Δ)
+#             zerou = zero(sol.prob.u0)
+#             _Δ = @. ifelse(Δ === nothing, (zerou,), Δ)
+#             (SciMLBase.build_solution(sol.prob, sol.alg, sol.t, _Δ),)
+#         end
+#         sol.u, solu_adjoint
+#     end
+# end
path: src/adjoint_common.jl
user: login: ChrisRackauckas
body: dead code?
created_at: 2025-04-24T12:22:33Z
author_association: MEMBER
diff_hunk: @@ -737,16 +749,16 @@ function out_and_ts(_ts, duplicate_iterator_times, sol)
     return out, ts
 end
 
-if !hasmethod(Zygote.adjoint,
-    Tuple{Zygote.AContext, typeof(Zygote.literal_getproperty),
-        SciMLBase.AbstractTimeseriesSolution, Val{:u}})
-    Zygote.@adjoint function Zygote.literal_getproperty(sol::AbstractTimeseriesSolution,
-            ::Val{:u})
-        function solu_adjoint(Δ)
-            zerou = zero(sol.prob.u0)
-            _Δ = @. ifelse(Δ === nothing, (zerou,), Δ)
-            (SciMLBase.build_solution(sol.prob, sol.alg, sol.t, _Δ),)
-        end
-        sol.u, solu_adjoint
-    end
-end
+# if !hasmethod(Zygote.adjoint,
+#     Tuple{Zygote.AContext, typeof(Zygote.literal_getproperty),
+#         SciMLBase.AbstractTimeseriesSolution, Val{:u}})
+#     Zygote.@adjoint function Zygote.literal_getproperty(sol::AbstractTimeseriesSolution,
+#             ::Val{:u})
+#         function solu_adjoint(Δ)
+#             zerou = zero(sol.prob.u0)
+#             _Δ = @. ifelse(Δ === nothing, (zerou,), Δ)
+#             (SciMLBase.build_solution(sol.prob, sol.alg, sol.t, _Δ),)
+#         end
+#         sol.u, solu_adjoint
+#     end
+# end
path: src/adjoint_common.jl
user: login: DhairyaLGandhi
body: I made a mistake pushing https://github.com/SciML/SciMLSensitivity.jl/pull/1168/commits/cdaa2c70c7e89050f10cd3731b70311a4570b05c so have reverted it
created_at: 2025-04-24T12:40:11Z
author_association: MEMBER
diff_hunk: @@ -737,16 +749,16 @@ function out_and_ts(_ts, duplicate_iterator_times, sol)
     return out, ts
 end
 
-if !hasmethod(Zygote.adjoint,
-    Tuple{Zygote.AContext, typeof(Zygote.literal_getproperty),
-        SciMLBase.AbstractTimeseriesSolution, Val{:u}})
-    Zygote.@adjoint function Zygote.literal_getproperty(sol::AbstractTimeseriesSolution,
-            ::Val{:u})
-        function solu_adjoint(Δ)
-            zerou = zero(sol.prob.u0)
-            _Δ = @. ifelse(Δ === nothing, (zerou,), Δ)
-            (SciMLBase.build_solution(sol.prob, sol.alg, sol.t, _Δ),)
-        end
-        sol.u, solu_adjoint
-    end
-end
+# if !hasmethod(Zygote.adjoint,
+#     Tuple{Zygote.AContext, typeof(Zygote.literal_getproperty),
+#         SciMLBase.AbstractTimeseriesSolution, Val{:u}})
+#     Zygote.@adjoint function Zygote.literal_getproperty(sol::AbstractTimeseriesSolution,
+#             ::Val{:u})
+#         function solu_adjoint(Δ)
+#             zerou = zero(sol.prob.u0)
+#             _Δ = @. ifelse(Δ === nothing, (zerou,), Δ)
+#             (SciMLBase.build_solution(sol.prob, sol.alg, sol.t, _Δ),)
+#         end
+#         sol.u, solu_adjoint
+#     end
+# end
path: src/adjoint_common.jl
user: login: DhairyaLGandhi
body: Apologies
created_at: 2025-04-24T12:40:23Z
author_association: MEMBER
diff_hunk: @@ -1,7 +1,7 @@
 name = "SciMLSensitivity"
 uuid = "1ed8b502-d754-442c-8d5d-10ac956f44a1"
 authors = ["Christopher Rackauckas <accounts@chrisrackauckas.com>", "Yingbo Ma <mayingbo5@gmail.com>"]
-version = "7.77.0"
+version = "7.76.0"
path: Project.toml
user: login: ChrisRackauckas
body: ?
created_at: 2025-04-24T13:39:10Z
author_association: MEMBER
diff_hunk: @@ -1,7 +1,7 @@
 name = "SciMLSensitivity"
 uuid = "1ed8b502-d754-442c-8d5d-10ac956f44a1"
 authors = ["Christopher Rackauckas <accounts@chrisrackauckas.com>", "Yingbo Ma <mayingbo5@gmail.com>"]
-version = "7.77.0"
+version = "7.76.0"
path: Project.toml
user: login: DhairyaLGandhi
body: Ah this was picked from the history, updated. Thanks
created_at: 2025-04-24T13:50:12Z
author_association: MEMBER

PR comments:
user: login: DhairyaLGandhi
created_at: 2025-03-06T12:44:31Z
body: I wanted to ask whether it is preferred to retain `\` for the smaller problems, which would be slower with LinearSolve, and if so, we will need to deal with LAPACK errors.
author_association: MEMBER
user: login: ChrisRackauckas
created_at: 2025-03-08T12:45:46Z
body: > I wanted to ask whether it is preferred to retain \ for the smaller problems, which would be slower with LinearSolve, and if so, we will need to deal with LAPACK errors.

LinearSolve.jl should be faster across the board? It depends a bit on the CPU architecture since it depends on whether it guesses the right LU correctly, 


author_association: MEMBER
user: login: ChrisRackauckas
created_at: 2025-03-08T12:46:50Z
body: Note that with the latest MTK update, there is now an `Initials` section where the initial `u0` live. That should fix a few things.
author_association: MEMBER
user: login: DhairyaLGandhi
created_at: 2025-03-10T14:10:19Z
body: Trying to use the initialization end to end caused gradients against parameters to get dropped. https://github.com/DhairyaLGandhi/SciMLBase.jl/tree/dg/nonlinear is a WIP branch which adds adjoints to the `getindex` calls, which does capture the expected gradients, but we still end up dropping gradients somewhere in the chain. I am looking into whether we are doing so in the custom adjoints, because that was an issue I had identified for the ODE case. 


author_association: MEMBER
user: login: DhairyaLGandhi
created_at: 2025-03-13T10:57:16Z
body: I could use some understanding of how to handle initialization when MTK analytically solves the problem, and removes all the unknowns. In that case `u0`, `u` etc are empty, and we attempt to calculate the jacobian for it in `dgdgu_val` which seems weird, there are no unknowns to calculate the jacobian for. It would be equally incorrect to use the unknowns from the system itself. The best way to handle this might be to simply add a check for `isempty` and return. Is that reasonable?
author_association: MEMBER
user: login: DhairyaLGandhi
created_at: 2025-04-07T11:02:29Z
body: Core5 fails because we try to call `canonicalize` on parameters constructed by Lux models (ie a NamedTuple). These are not SciMLStructures, what's the best course of action here @AayushSabharwal 
author_association: MEMBER
user: login: AayushSabharwal
created_at: 2025-04-07T11:04:54Z
body: Use a `ComponentArray`?
author_association: MEMBER
user: login: ChrisRackauckas
created_at: 2025-04-21T18:15:59Z
body: I added the BrownBasic for the reverse and setup the tests so that there's a version that would capture that issue of the MTK init in the reverse pass. It almost certainly needs the fix to https://github.com/SciML/ModelingToolkit.jl/issues/3570 to pass though, so it will likely fail at first and @AayushSabharwal this is a reason to prioritize getting that one completed.

But when that is merged and this passes, then I think this is good to go.
author_association: MEMBER
user: login: DhairyaLGandhi
created_at: 2025-04-23T19:14:19Z
body: There seems to be an `InitialFailure` with `DefaultInit`
author_association: MEMBER
user: login: DhairyaLGandhi
created_at: 2025-04-23T19:23:09Z
body: ```julia
julia> prob, ps, init = setups[3];

julia> init
OrdinaryDiffEqCore.DefaultInit()

julia> new_sol = solve(prob, Rodas5P(); p = p, sensealg, initializealg = init, abstol = 1e-6, reltol = 1e-3)
┌ Warning: Initialization system is overdetermined. 1 equations for 0 unknowns. Initialization will default to using least squares. `SCCNonlinearProblem` can only be used for initialization of fully determined systems and hence will not be used here. To suppress this warning pass warn_initialize_determined = false. To make this warning into an error, pass fully_determined = true
└ @ ModelingToolkit ~/.julia/packages/ModelingToolkit/aau6A/src/systems/diffeqs/abstractodesystem.jl:1520
retcode: InitialFailure
Interpolation: specialized 4rd order "free" stiffness-aware interpolation
t: 1-element Vector{Float64
}:
 0.0
u: 1-element Vector{Vector{Float64}}:
 [2.0, 0.0, 0.0, 1.0, 0.0]
```
author_association: MEMBER