Repository: sunnypilot/sunnypilot
Pull request #667: Controls: Neural Network Lateral Control (NNLC) for Torque Lateral Accel Control
Author: login: sunnyhaibin
Labels: name: enhancement
description: New feature or request
name: ui
description: UI change
name: dev-c3
description: This label will flag the PR used on it to be brought into dev-c3 branch
Created: 2025-03-16T08:02:11Z | Merged: 2025-03-21T19:38:31Z

Huge thanks to @twilsonco for NNLC!

- Requires https://github.com/sunnypilot/opendbc/pull/85
- [x] split Custom Lateral Acceleration
- [x] UI

Closes https://github.com/sunnypilot/sunnypilot/issues/632

## Summary by Sourcery

Implements Neural Network Lateral Control (NNLC) to replace the existing lateral torque controller with a neural network-based controller. This enhancement aims to improve control accuracy by leveraging car-specific driving data. It also adds a toggle in the developer panel to enable/disable the feature.

New Features:
- Introduces a neural network-based lateral control system as an alternative to the existing torque controller.
- Adds a toggle in the developer panel to enable or disable the Neural Network Lateral Control (NNLC) feature.

## Summary by Sourcery

Implements Neural Network Lateral Control (NNLC) as an alternative to the existing lateral torque controller. It also adds a toggle in the developer panel to enable/disable the feature.

New Features:
- Introduces a neural network-based lateral control system as an alternative to the existing torque controller.
- Adds a toggle in the developer panel to enable or disable the Neural Network Lateral Control (NNLC) feature.

## Summary by Sourcery

Implements Neural Network Lateral Control (NNLC) as an alternative to the existing lateral torque controller. It also adds a toggle in the developer panel to enable/disable the feature.

New Features:
- Introduces a neural network-based lateral control system as an alternative to the existing torque controller.
- Adds a toggle in the developer panel to enable or disable the Neural Network Lateral Control (NNLC) feature.


Diff:
diff --git a/.gitmodules b/.gitmodules
index f1f7c400dbf..b9f0336a0e3 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -16,3 +16,6 @@
 [submodule "tinygrad"]
   path = tinygrad_repo
   url = https://github.com/commaai/tinygrad.git
+[submodule "sunnypilot/neural_network_data"]
+  path = sunnypilot/neural_network_data
+  url = https://github.com/sunnypilot/neural-network-data.git
diff --git a/cereal/custom.capnp b/cereal/custom.capnp
index 524ecdd6579..4d245c006bb 100644
--- a/cereal/custom.capnp
+++ b/cereal/custom.capnp
@@ -138,6 +138,18 @@ struct OnroadEventSP @0xda96579883444c35 {
 struct CarParamsSP @0x80ae746ee2596b11 {
   flags @0 :UInt32;        # flags for car specific quirks in sunnypilot
   safetyParam @1 : Int16;  # flags for sunnypilot's custom safety flags
+
+  neuralNetworkLateralControl @2 :NeuralNetworkLateralControl;
+
+  struct NeuralNetworkLateralControl {
+    model @0 :Model;
+    fuzzyFingerprint @1 :Bool;
+
+    struct Model {
+      path @0 :Text;
+      name @1 :Text;
+    }
+  }
 }
 
 struct CarControlSP @0xa5cd762cd951a455 {
diff --git a/common/params_keys.h b/common/params_keys.h
index d88189d4322..f95539963d7 100644
--- a/common/params_keys.h
+++ b/common/params_keys.h
@@ -141,6 +141,9 @@ inline static std::unordered_map<std::string, uint32_t> keys = {
     {"ModelManager_LastSyncTime", CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION},
     {"ModelManager_ModelsCache", PERSISTENT | BACKUP},
 
+    // Neural Network Lateral Control
+    {"NeuralNetworkLateralControl", PERSISTENT | BACKUP},
+
     // sunnylink params
     {"EnableSunnylinkUploader", PERSISTENT | BACKUP},
     {"LastSunnylinkPingTime", CLEAR_ON_MANAGER_START},
diff --git a/opendbc_repo b/opendbc_repo
index a26ea1e3ee9..c10bc5bd85a 160000
--- a/opendbc_repo
+++ b/opendbc_repo
@@ -1 +1 @@
-Subproject commit a26ea1e3ee982a4b2c377f5f75fdde63cba6a007
+Subproject commit c10bc5bd85a66b61e9f917a56154b6e3e2414288
diff --git a/selfdrive/ui/sunnypilot/SConscript b/selfdrive/ui/sunnypilot/SConscript
index 6866bfb7d8c..24c98c9868f 100644
--- a/selfdrive/ui/sunnypilot/SConscript
+++ b/selfdrive/ui/sunnypilot/SConscript
@@ -36,6 +36,7 @@ qt_src = [
 
 lateral_panel_qt_src = [
   "sunnypilot/qt/offroad/settings/lateral/mads_settings.cc",
+  "sunnypilot/qt/offroad/settings/lateral/neural_network_lateral_control.cc",
 ]
 
 network_src = [
diff --git a/selfdrive/ui/sunnypilot/qt/offroad/settings/lateral/neural_network_lateral_control.cc b/selfdrive/ui/sunnypilot/qt/offroad/settings/lateral/neural_network_lateral_control.cc
new file mode 100644
index 00000000000..d3bc0fffa7b
--- /dev/null
+++ b/selfdrive/ui/sunnypilot/qt/offroad/settings/lateral/neural_network_lateral_control.cc
@@ -0,0 +1,63 @@
+/**
+ * Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
+ *
+ * This file is part of sunnypilot and is licensed under the MIT License.
+ * See the LICENSE.md file in the root directory for more details.
+ */
+
+#include "selfdrive/ui/sunnypilot/qt/offroad/settings/lateral/neural_network_lateral_control.h"
+
+NeuralNetworkLateralControl::NeuralNetworkLateralControl() :
+  ParamControl("NeuralNetworkLateralControl", tr("Neural Network Lateral Control (NNLC)"),  "", "") {
+  setConfirmation(true, false);
+  updateToggle();
+}
+
+void NeuralNetworkLateralControl::showEvent(QShowEvent *event) {
+  updateToggle();
+}
+
+void NeuralNetworkLateralControl::updateToggle() {
+  QString statusInitText = "<font color='yellow'>" + STATUS_CHECK_COMPATIBILITY + "</font>";
+  QString notLoadedText = "<font color='yellow'>" + STATUS_NOT_LOADED + "</font>";
+  QString loadedText = "<font color=#00ff00>" + STATUS_LOADED + "</font>";
+
+  auto cp_bytes = params.get("CarParamsPersistent");
+  auto cp_sp_bytes = params.get("CarParamsSPPersistent");
+  if (!cp_bytes.empty() && !cp_sp_bytes.empty()) {
+    AlignedBuffer aligned_buf;
+    AlignedBuffer aligned_buf_sp;
+    capnp::FlatArrayMessageReader cmsg(aligned_buf.align(cp_bytes.data(), cp_bytes.size()));
+    capnp::FlatArrayMessageReader cmsg_sp(aligned_buf_sp.align(cp_sp_bytes.data(), cp_sp_bytes.size()));
+    cereal::CarParams::Reader CP = cmsg.getRoot<cereal::CarParams>();
+    cereal::CarParamsSP::Reader CP_SP = cmsg_sp.getRoot<cereal::CarParamsSP>();
+
+    if (CP.getSteerControlType() == cereal::CarParams::SteerControlType::ANGLE) {
+      params.remove("NeuralNetworkLateralControl");
+      setDescription(nnffDescriptionBuilder(STATUS_NOT_AVAILABLE));
+      setEnabled(false);
+    } else {
+      QString nn_model_name = QString::fromStdString(CP_SP.getNeuralNetworkLateralControl().getModel().getName());
+      QString nn_fuzzy = CP_SP.getNeuralNetworkLateralControl().getFuzzyFingerprint() ?
+                         STATUS_MATCH_FUZZY : STATUS_MATCH_EXACT;
+
+      if (nn_model_name.isEmpty()) {
+        setDescription(nnffDescriptionBuilder(statusInitText));
+      } else if (nn_model_name == "MOCK") {
+        setDescription(nnffDescriptionBuilder(
+          notLoadedText + "<br>" + buildSupportText(SUPPORT_DONATE_LOGS)
+        ));
+      } else {
+        QString statusText = loadedText + " | " + STATUS_MATCH + " = " + nn_fuzzy + " | " + nn_model_name;
+        QString explanationText = EXPLANATION_MATCH + " " + buildSupportText(SUPPORT_ISSUES);
+        setDescription(nnffDescriptionBuilder(statusText + "<br><br>" + explanationText));
+      }
+    }
+  } else {
+    setDescription(nnffDescriptionBuilder(statusInitText));
+  }
+
+  if (getDescription() != getBaseDescription()) {
+    showDescription();
+  }
+}
diff --git a/selfdrive/ui/sunnypilot/qt/offroad/settings/lateral/neural_network_lateral_control.h b/selfdrive/ui/sunnypilot/qt/offroad/settings/lateral/neural_network_lateral_control.h
new file mode 100644
index 00000000000..18a3fac1873
--- /dev/null
+++ b/selfdrive/ui/sunnypilot/qt/offroad/settings/lateral/neural_network_lateral_control.h
@@ -0,0 +1,64 @@
+/**
+ * Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
+ *
+ * This file is part of sunnypilot and is licensed under the MIT License.
+ * See the LICENSE.md file in the root directory for more details.
+ */
+
+#pragma once
+
+#include <map>
+
+#include "selfdrive/ui/sunnypilot/ui.h"
+#include "selfdrive/ui/qt/util.h"
+#include "selfdrive/ui/sunnypilot/qt/offroad/settings/settings.h"
+
+class NeuralNetworkLateralControl : public ParamControl {
+  Q_OBJECT
+
+public:
+  NeuralNetworkLateralControl();
+  void showEvent(QShowEvent *event) override;
+
+public slots:
+  void updateToggle();
+
+private:
+  Params params;
+
+  void refresh();
+
+  // Status messages
+  const QString STATUS_NOT_AVAILABLE = tr("NNLC is currently not available on this platform.");
+  const QString STATUS_CHECK_COMPATIBILITY = tr("Start the car to check car compatibility");
+  const QString STATUS_NOT_LOADED = tr("NNLC Not Loaded");
+  const QString STATUS_LOADED = tr("NNLC Loaded");
+  const QString STATUS_MATCH = tr("Match");
+  const QString STATUS_MATCH_EXACT = tr("Exact");
+  const QString STATUS_MATCH_FUZZY = tr("Fuzzy");
+
+  // Explanations
+  const QString EXPLANATION_MATCH = tr("Match: \"Exact\" is ideal, but \"Fuzzy\" is fine too.");
+  const QString EXPLANATION_FEATURE = tr("Formerly known as <b>\"NNFF\"</b>, this replaces the lateral <b>\"torque\"</b> controller, "
+                                            "with one using a neural network trained on each car's (actually, each separate EPS firmware) driving data for increased controls accuracy.");
+
+  // Support information
+  const QString SUPPORT_CHANNEL = "<font color='white'><b>#tuning-nnlc</b></font>";
+  const QString SUPPORT_REACH_OUT = tr("Reach out to the sunnypilot team in the following channel at the sunnypilot Discord server");
+  const QString SUPPORT_FEEDBACK = tr("with feedback, or to provide log data for your car if your car is currently unsupported:");
+  const QString SUPPORT_ISSUES = tr("if there are any issues:");
+  const QString SUPPORT_DONATE_LOGS = tr("and donate logs to get NNLC loaded for your car:");
+
+  // Description builders
+  QString buildSupportText(const QString& context) const {
+    return SUPPORT_REACH_OUT + " " + context + " " + SUPPORT_CHANNEL;
+  }
+
+  QString nnffDescriptionBuilder(const QString &custom_description) const {
+    return "<b>" + custom_description + "</b><br><br>" + getBaseDescription();
+  }
+
+  QString getBaseDescription() const {
+    return EXPLANATION_FEATURE + "<br><br>" + buildSupportText(SUPPORT_FEEDBACK);
+  }
+};
diff --git a/selfdrive/ui/sunnypilot/qt/offroad/settings/lateral_panel.cc b/selfdrive/ui/sunnypilot/qt/offroad/settings/lateral_panel.cc
index e604b03ba91..750e26c118f 100644
--- a/selfdrive/ui/sunnypilot/qt/offroad/settings/lateral_panel.cc
+++ b/selfdrive/ui/sunnypilot/qt/offroad/settings/lateral_panel.cc
@@ -42,8 +42,21 @@ LateralPanel::LateralPanel(SettingsWindowSP *parent) : QFrame(parent) {
   });
   list->addItem(madsSettingsButton);
 
+  nnlcToggle = new NeuralNetworkLateralControl();
+  list->addItem(nnlcToggle);
+
+  QObject::connect(nnlcToggle, &ParamControl::toggleFlipped, [=](bool state) {
+    if (state) {
+      nnlcToggle->showDescription();
+    } else {
+      nnlcToggle->hideDescription();
+    }
+
+    nnlcToggle->updateToggle();
+  });
+
   toggleOffroadOnly = {
-    madsToggle,
+    madsToggle, nnlcToggle,
   };
   QObject::connect(uiState(), &UIState::offroadTransition, this, &LateralPanel::updateToggles);
 
diff --git a/selfdrive/ui/sunnypilot/qt/offroad/settings/lateral_panel.h b/selfdrive/ui/sunnypilot/qt/offroad/settings/lateral_panel.h
index 7cc4b6f11d2..0857e6ebd54 100644
--- a/selfdrive/ui/sunnypilot/qt/offroad/settings/lateral_panel.h
+++ b/selfdrive/ui/sunnypilot/qt/offroad/settings/lateral_panel.h
@@ -12,6 +12,7 @@
 
 #include "selfdrive/ui/sunnypilot/ui.h"
 #include "selfdrive/ui/sunnypilot/qt/offroad/settings/lateral/mads_settings.h"
+#include "selfdrive/ui/sunnypilot/qt/offroad/settings/lateral/neural_network_lateral_control.h"
 #include "selfdrive/ui/qt/util.h"
 #include "selfdrive/ui/sunnypilot/qt/offroad/settings/settings.h"
 #include "selfdrive/ui/sunnypilot/qt/widgets/scrollview.h"
@@ -37,4 +38,5 @@ public slots:
   ParamControl *madsToggle;
   PushButtonSP *madsSettingsButton;
   MadsSettings *madsWidget = nullptr;
+  NeuralNetworkLateralControl *nnlcToggle = nullptr;
 };
diff --git a/sunnypilot/neural_network_data b/sunnypilot/neural_network_data
new file mode 160000
index 00000000000..1acc8f75fbd
--- /dev/null
+++ b/sunnypilot/neural_network_data
@@ -0,0 +1 @@
+Subproject commit 1acc8f75fbdf2ea38e0234e8f7410f85778fe487
diff --git a/sunnypilot/selfdrive/car/interfaces.py b/sunnypilot/selfdrive/car/interfaces.py
index 375ce7fa7fa..0f36d345731 100644
--- a/sunnypilot/selfdrive/car/interfaces.py
+++ b/sunnypilot/selfdrive/car/interfaces.py
@@ -8,9 +8,12 @@
 from opendbc.car import Bus, structs
 from opendbc.car.can_definitions import CanRecvCallable, CanSendCallable
 from opendbc.car.car_helpers import can_fingerprint
+from opendbc.car.interfaces import CarInterfaceBase
 from opendbc.car.hyundai.radar_interface import RADAR_START_ADDR
 from opendbc.car.hyundai.values import HyundaiFlags, DBC as HYUNDAI_DBC
 from opendbc.sunnypilot.car.hyundai.values import HyundaiFlagsSP
+from openpilot.common.swaglog import cloudlog
+from openpilot.sunnypilot.selfdrive.controls.lib.nnlc.helpers import get_nn_model_path
 
 import openpilot.system.sentry as sentry
 
@@ -22,6 +25,25 @@ def log_fingerprint(CP: structs.CarParams) -> None:
     sentry.capture_fingerprint(CP.carFingerprint, CP.brand)
 
 
+def initialize_neural_network_lateral_control(CP: structs.CarParams, CP_SP: structs.CarParamsSP, params, enabled: bool = False) -> None:
+  nnlc_model_path, nnlc_model_name, exact_match = get_nn_model_path(CP)
+
+  if nnlc_model_path is None:
+    cloudlog.error({"nnlc event": "car doesn't match any Neural Network model"})
+    nnlc_model_path = "MOCK"
+    nnlc_model_name = "MOCK"
+
+  if nnlc_model_path != "MOCK" and CP.steerControlType != structs.CarParams.SteerControlType.angle:
+    enabled = params.get_bool("NeuralNetworkLateralControl")
+
+  if enabled:
+    CarInterfaceBase.configure_torque_tune(CP.carFingerprint, CP.lateralTuning)
+
+  CP_SP.neuralNetworkLateralControl.model.path = nnlc_model_path
+  CP_SP.neuralNetworkLateralControl.model.name = nnlc_model_name
+  CP_SP.neuralNetworkLateralControl.fuzzyFingerprint = not exact_match
+
+
 def setup_car_interface_sp(CP: structs.CarParams, CP_SP: structs.CarParamsSP, params):
   if CP.brand == 'hyundai':
     if CP.flags & HyundaiFlags.MANDO_RADAR and CP.radarUnavailable:
@@ -32,6 +54,8 @@ def setup_car_interface_sp(CP: structs.CarParams, CP_SP: structs.CarParamsSP, pa
         if params.get_bool("HyundaiRadarTracks"):
           CP.radarUnavailable = False
 
+  initialize_neural_network_lateral_control(CP, CP_SP, params)
+
 
 def initialize_car_interface_sp(CP: structs.CarParams, CP_SP: structs.CarParamsSP, params, can_recv: CanRecvCallable,
                                 can_send: CanSendCallable):
diff --git a/sunnypilot/selfdrive/controls/lib/latcontrol_torque_ext.py b/sunnypilot/selfdrive/controls/lib/latcontrol_torque_ext.py
index 1166cb32d6b..ee8d64dcd07 100644
--- a/sunnypilot/selfdrive/controls/lib/latcontrol_torque_ext.py
+++ b/sunnypilot/selfdrive/controls/lib/latcontrol_torque_ext.py
@@ -4,10 +4,11 @@
 This file is part of sunnypilot and is licensed under the MIT License.
 See the LICENSE.md file in the root directory for more details.
 """
-from openpilot.sunnypilot.selfdrive.controls.lib.latcontrol_torque_ext_base import LatControlTorqueExtBase
 
+from openpilot.sunnypilot.selfdrive.controls.lib.nnlc.nnlc import NeuralNetworkLateralControl
 
-class LatControlTorqueExt(LatControlTorqueExtBase):
+
+class LatControlTorqueExt(NeuralNetworkLateralControl):
   def __init__(self, lac_torque, CP, CP_SP):
     super().__init__(lac_torque, CP, CP_SP)
 
@@ -22,5 +23,6 @@ def update(self, CS, VM, params, ff, pid_log, setpoint, measurement, calibrated_
     self._actual_lateral_accel = actual_lateral_accel
 
     self.update_calculations(CS, VM, desired_lateral_accel)
+    self.update_neural_network_feedforward(CS, params, calibrated_pose)
 
     return self._ff, self._pid_log
diff --git a/sunnypilot/selfdrive/controls/lib/nnlc/__init__.py b/sunnypilot/selfdrive/controls/lib/nnlc/__init__.py
new file mode 100644
index 00000000000..e69de29bb2d
diff --git a/sunnypilot/selfdrive/controls/lib/nnlc/helpers.py b/sunnypilot/selfdrive/controls/lib/nnlc/helpers.py
new file mode 100644
index 00000000000..5f68a0a28e2
--- /dev/null
+++ b/sunnypilot/selfdrive/controls/lib/nnlc/helpers.py
@@ -0,0 +1,58 @@
+"""
+Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
+
+This file is part of sunnypilot and is licensed under the MIT License.
+See the LICENSE.md file in the root directory for more details.
+"""
+import os
+from difflib import SequenceMatcher
+
+from opendbc.car import structs
+from openpilot.common.basedir import BASEDIR
+
+TORQUE_NN_MODEL_PATH = os.path.join(BASEDIR, "sunnypilot", "neural_network_data", "neural_network_lateral_control")
+
+
+def similarity(s1: str, s2: str) -> float:
+  return SequenceMatcher(None, s1, s2).ratio()
+
+
+def get_nn_model_path(CP: structs.CarParams) -> tuple[str | None, str, bool]:
+  exact_match = True
+  car_fingerprint = CP.carFingerprint
+  eps_fw = str(next((fw.fwVersion for fw in CP.carFw if fw.ecu == "eps"), ""))
+  model_name = ""
+
+  def check_nn_path(nn_candidate):
+    _model_path = None
+    _max_similarity = -1.0
+    for f in os.listdir(TORQUE_NN_MODEL_PATH):
+      if f.endswith(".json"):
+        model = os.path.splitext(f)[0]
+        similarity_score = similarity(model, nn_candidate)
+        if similarity_score > _max_similarity:
+          _max_similarity = similarity_score
+          _model_path = os.path.join(TORQUE_NN_MODEL_PATH, f)
+    return _model_path, _max_similarity
+
+  if len(eps_fw) > 3:
+    eps_fw = eps_fw.replace("\\", "")
+    nn_candidate = f"{car_fingerprint} {eps_fw}"
+    model_path, max_similarity = check_nn_path(nn_candidate)
+
+    if model_path is not None and car_fingerprint in model_path and max_similarity >= 0.9:
+      model_name = os.path.splitext(os.path.basename(model_path))[0]
+      exact_match = max_similarity >= 0.99
+      return model_path, model_name, exact_match
+
+  nn_candidate = car_fingerprint
+  model_path, max_similarity = check_nn_path(nn_candidate)
+
+  if model_path is None or car_fingerprint not in model_path or max_similarity < 0.9:
+    model_path = None
+
+  if model_path is not None:
+    model_name = os.path.splitext(os.path.basename(model_path))[0]
+    exact_match = max_similarity >= 0.99
+
+  return model_path, model_name, exact_match
diff --git a/sunnypilot/selfdrive/controls/lib/nnlc/model.py b/sunnypilot/selfdrive/controls/lib/nnlc/model.py
new file mode 100644
index 00000000000..1e527c73700
--- /dev/null
+++ b/sunnypilot/selfdrive/controls/lib/nnlc/model.py
@@ -0,0 +1,81 @@
+"""
+Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
+
+This file is part of sunnypilot and is licensed under the MIT License.
+See the LICENSE.md file in the root directory for more details.
+"""
+from json import load
+import numpy as np
+
+# dict used to rename activation functions whose names aren't valid python identifiers
+ACTIVATION_FUNCTION_NAMES = {'σ': 'sigmoid'}
+
+
+class NNTorqueModel:
+  def __init__(self, params_file, zero_bias=False):
+    with open(params_file) as f:
+      params = load(f)
+
+    self.input_size = params["input_size"]
+    self.output_size = params["output_size"]
+    self.input_mean = np.array(params["input_mean"], dtype=np.float32).T
+    self.input_std = np.array(params["input_std"], dtype=np.float32).T
+    self.layers = []
+    self.friction_override = False
+
+    for layer_params in params["layers"]:
+      W = np.array(layer_params[next(key for key in layer_params.keys() if key.endswith('_W'))], dtype=np.float32).T
+      b = np.array(layer_params[next(key for key in layer_params.keys() if key.endswith('_b'))], dtype=np.float32).T
+      if zero_bias:
+        b = np.zeros_like(b)
+      activation = layer_params["activation"]
+      for k, v in ACTIVATION_FUNCTION_NAMES.items():
+        activation = activation.replace(k, v)
+      self.layers.append((W, b, activation))
+
+    self.validate_layers()
+    self.check_for_friction_override()
+
+  # Begin activation functions.
+  # These are called by name using the keys in the model json file
+  @staticmethod
+  def sigmoid(x):
+    return 1 / (1 + np.exp(-x))
+
+  @staticmethod
+  def identity(x):
+    return x
+  # End activation functions
+
+  def forward(self, x):
+    for W, b, activation in self.layers:
+      x = getattr(self, activation)(x.dot(W) + b)
+    return x
+
+  def evaluate(self, input_array):
+    in_len = len(input_array)
+    if in_len != self.input_size:
+      # If the input is length 2-4, then it's a simplified evaluation.
+      # In that case, need to add on zeros to fill out the input array to match the correct length.
+      if 2 <= in_len:
+        input_array = input_array + [0] * (self.input_size - in_len)
+      else:
+        raise ValueError(f"Input array length {len(input_array)} must be length 2 or greater")
+
+    input_array = np.array(input_array, dtype=np.float32)
+
+    # Rescale the input array using the input_mean and input_std
+    input_array = (input_array - self.input_mean) / self.input_std
+
+    output_array = self.forward(input_array)
+
+    return float(output_array[0, 0])
+
+  def validate_layers(self):
+    for _, _, activation in self.layers:
+      if not hasattr(self, activation):
+        raise ValueError(f"Unknown activation: {activation}")
+
+  def check_for_friction_override(self):
+    y = self.evaluate([10.0, 0.0, 0.2])
+    self.friction_override = (y < 0.1)
diff --git a/sunnypilot/selfdrive/controls/lib/nnlc/nnlc.py b/sunnypilot/selfdrive/controls/lib/nnlc/nnlc.py
new file mode 100644
index 00000000000..e1208ab829a
--- /dev/null
+++ b/sunnypilot/selfdrive/controls/lib/nnlc/nnlc.py
@@ -0,0 +1,106 @@
+"""
+Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
+
+This file is part of sunnypilot and is licensed under the MIT License.
+See the LICENSE.md file in the root directory for more details.
+"""
+from collections import deque
+import math
+import numpy as np
+
+from opendbc.car.interfaces import LatControlInputs
+from openpilot.common.filter_simple import FirstOrderFilter
+from openpilot.common.params import Params
+from openpilot.selfdrive.modeld.constants import ModelConstants
+from openpilot.sunnypilot.selfdrive.controls.lib.latcontrol_torque_ext_base import LatControlTorqueExtBase
+from openpilot.sunnypilot.selfdrive.controls.lib.nnlc.model import NNTorqueModel
+
+
+# At a given roll, if pitch magnitude increases, the
+# gravitational acceleration component starts pointing
+# in the longitudinal direction, decreasing the lateral
+# acceleration component. Here we do the same thing
+# to the roll value itself, then passed to nnff.
+def roll_pitch_adjust(roll, pitch):
+  return roll * math.cos(pitch)
+
+
+class NeuralNetworkLateralControl(LatControlTorqueExtBase):
+  def __init__(self, lac_torque, CP, CP_SP):
+    super().__init__(lac_torque, CP, CP_SP)
+    self.params = Params()
+    self.enabled = self.params.get_bool("NeuralNetworkLateralControl")
+
+    # NN model takes current v_ego, lateral_accel, lat accel/jerk error, roll, and past/future/planned data
+    # of lat accel and roll
+    # Past value is computed using previous desired lat accel and observed roll
+    # Only initialize NNTorqueModel if enabled
+    self.model = NNTorqueModel(CP_SP.neuralNetworkLateralControl.model.path) if self.enabled else None
+
+    self.pitch = FirstOrderFilter(0.0, 0.5, 0.01)
+    self.pitch_last = 0.0
+
+    # setup future time offsets
+    self.nn_time_offset = CP.steerActuatorDelay + 0.2
+    future_times = [0.3, 0.6, 1.0, 1.5] # seconds in the future
+    self.nn_future_times = [i + self.nn_time_offset for i in future_times]
+
+    # setup past time offsets
+    self.past_times = [-0.3, -0.2, -0.1]
+    history_check_frames = [int(abs(i)*100) for i in self.past_times]
+    self.history_frame_offsets = [history_check_frames[0] - i for i in history_check_frames]
+    self.lateral_accel_desired_deque = deque(maxlen=history_check_frames[0])
+    self.roll_deque = deque(maxlen=history_check_frames[0])
+    self.error_deque = deque(maxlen=history_check_frames[0])
+    self.past_future_len = len(self.past_times) + len(self.nn_future_times)
+
+  def update_neural_network_feedforward(self, CS, params, calibrated_pose):
+    if not self.enabled or not self.model_valid:
+      return
+
+    # update past data
+    roll = params.roll
+    if calibrated_pose is not None:
+      pitch = self.pitch.update(calibrated_pose.orientation.pitch)
+      roll = roll_pitch_adjust(roll, pitch)
+      self.pitch_last = pitch
+    self.roll_deque.append(roll)
+    self.lateral_accel_desired_deque.append(self._desired_lateral_accel)
+
+    # prepare past and future values
+    # adjust future times to account for longitudinal acceleration
+    adjusted_future_times = [t + 0.5 * CS.aEgo * (t / max(CS.vEgo, 1.0)) for t in self.nn_future_times]
+    past_rolls = [self.roll_deque[min(len(self.roll_deque) - 1, i)] for i in self.history_frame_offsets]
+    future_rolls = [roll_pitch_adjust(np.interp(t, ModelConstants.T_IDXS, self.model_v2.orientation.x) + roll,
+                                      np.interp(t, ModelConstants.T_IDXS, self.model_v2.orientation.y) + self.pitch_last) for t in
+                    adjusted_future_times]
+    past_lateral_accels_desired = [self.lateral_accel_desired_deque[min(len(self.lateral_accel_desired_deque) - 1, i)]
+                                   for i in self.history_frame_offsets]
+    future_planned_lateral_accels = [np.interp(t, ModelConstants.T_IDXS, self.model_v2.acceleration.y) for t in
+                                     adjusted_future_times]
+
+    # compute NNFF error response
+    nnff_setpoint_input = [CS.vEgo, self._setpoint, self.lateral_jerk_setpoint, roll] \
+                          + [self._setpoint] * self.past_future_len \
+                          + past_rolls + future_rolls
+    # past lateral accel error shouldn't count, so use past desired like the setpoint input
+    nnff_measurement_input = [CS.vEgo, self._measurement, self.lateral_jerk_measurement, roll] \
+                             + [self._measurement] * self.past_future_len \
+                             + past_rolls + future_rolls
+    torque_from_setpoint = self.model.evaluate(nnff_setpoint_input)
+    torque_from_measurement = self.model.evaluate(nnff_measurement_input)
+    self._pid_log.error = torque_from_setpoint - torque_from_measurement
+
+    # compute feedforward (same as nn setpoint output)
+    friction_input = self.update_friction_input(self._setpoint, self._measurement)
+    nn_input = [CS.vEgo, self._desired_lateral_accel, friction_input, roll] \
+               + past_lateral_accels_desired + future_planned_lateral_accels \
+               + past_rolls + future_rolls
+    self._ff = self.model.evaluate(nn_input)
+
+    # apply friction override for cars with low NN friction response
+    if self.model.friction_override:
+      self._pid_log.error += self.torque_from_lateral_accel(LatControlInputs(0.0, 0.0, CS.vEgo, CS.aEgo), self.torque_params,
+                                                            friction_input,
+                                                            self._lateral_accel_deadzone, friction_compensation=True,
+                                                            gravity_adjusted=False)
diff --git a/system/manager/manager.py b/system/manager/manager.py
index d4595ec3275..9266a62aaec 100755
--- a/system/manager/manager.py
+++ b/system/manager/manager.py
@@ -49,7 +49,8 @@ def manager_init() -> None:
     ("MadsSteeringMode", "0"),
     ("MadsUnifiedEngagementMode", "1"),
     ("ModelManager_LastSyncTime", "0"),
-    ("ModelManager_ModelsCache", "")
+    ("ModelManager_ModelsCache", ""),
+    ("NeuralNetworkLateralControl", "0"),
   ]
 
   if params.get_bool("RecordFrontLock"):


Reviews:
user: login: sunnyhaibin
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-17T01:07:04Z
user: login: twilsonco
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T05:01:43Z
user: login: twilsonco
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T05:09:11Z
user: login: twilsonco
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T05:12:12Z
user: login: twilsonco
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T05:31:58Z
user: login: twilsonco
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T05:33:39Z
user: login: twilsonco
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T05:44:46Z
user: login: sunnyhaibin
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-17T18:16:46Z
user: login: twilsonco
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-20T16:18:04Z
user: login: sunnyhaibin
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-20T16:35:39Z
user: login: devtekve
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-20T17:26:25Z
user: login: devtekve
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-20T17:28:21Z
user: login: devtekve
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-20T17:29:00Z
user: login: devtekve
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-20T17:30:03Z
user: login: devtekve
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-20T17:30:48Z
user: login: devtekve
state: DISMISSED
author_association: CONTRIBUTOR
submitted_at: 2025-03-20T17:33:52Z
user: login: sunnyhaibin
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-20T17:37:58Z
user: login: sunnyhaibin
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-21T04:56:58Z

Review comments:
diff_hunk: @@ -196,7 +196,7 @@ def update_feed_forward(self, CS, params, setpoint, measurement, calibrated_pose
                     adjusted_future_times]
     past_lateral_accels_desired = [self.lateral_accel_desired_deque[min(len(self.lateral_accel_desired_deque) - 1, i)]
                                    for i in self.history_frame_offsets]
-    future_planned_lateral_accels = [np.interp(t, ModelConstants.T_IDXS[:CONTROL_N], self.model_v2.acceleration.y) for t in
+    future_planned_lateral_accels = [np.interp(t, ModelConstants.T_IDXS, self.model_v2.acceleration.y) for t in
path: sunnypilot/selfdrive/controls/lib/nnlc/nnlc.py
user: login: sunnyhaibin
body: @twilsonco I had to make this change as it was crashing with `fp and xp are not of the same length.`

xp
```
[
0.0,
0.009765625,
0.0390625,
0.087890625,
0.15625,
0.244140625,
0.3515625,
0.478515625,
0.625,
0.791015625
]
```

fp
```
array([-0.06699371, -0.06308746, -0.06019974, -0.06203461, -0.06251526,
       -0.06240082, -0.06358337, -0.05828857, -0.05441284, -0.05222988,
       -0.04789734, -0.0398407 , -0.02944183, -0.02194214, -0.01302958,
        0.00018311,  0.00842285,  0.01643372,  0.0282135 ,  0.03319931,
        0.03149223,  0.02767563,  0.02198029,  0.01364899,  0.0036087 ,
       -0.00476837, -0.01176453, -0.01628113, -0.0209465 , -0.02834702,
       -0.02967834, -0.02478027, -0.01730347])
```
created_at: 2025-03-17T01:07:03Z
author_association: COLLABORATOR
diff_hunk: @@ -196,7 +196,7 @@ def update_feed_forward(self, CS, params, setpoint, measurement, calibrated_pose
                     adjusted_future_times]
     past_lateral_accels_desired = [self.lateral_accel_desired_deque[min(len(self.lateral_accel_desired_deque) - 1, i)]
                                    for i in self.history_frame_offsets]
-    future_planned_lateral_accels = [np.interp(t, ModelConstants.T_IDXS[:CONTROL_N], self.model_v2.acceleration.y) for t in
+    future_planned_lateral_accels = [np.interp(t, ModelConstants.T_IDXS, self.model_v2.acceleration.y) for t in
path: sunnypilot/selfdrive/controls/lib/nnlc/nnlc.py
user: login: twilsonco
body: Yeah, the old "numpy fast" interp function could handle different lengths.  The intent was to limit the farthest future model lateral acceleration value that could be returned, so it's better to instead take [:N] from both arrays.
created_at: 2025-03-17T05:01:43Z
author_association: CONTRIBUTOR
diff_hunk: @@ -73,6 +77,11 @@ def update(self, active, CS, VM, params, steer_limited_by_controls, desired_curv
                                           desired_lateral_accel - actual_lateral_accel, lateral_accel_deadzone, friction_compensation=True,
                                           gravity_adjusted=True)
 
+      # Neural Network Lateral Control and custom stock lateral jerk updates
+      # Override stock ff and pid_log.error
+      ff, pid_log = self.nnlc.update(CS, VM, params, ff, pid_log, setpoint, measurement, calibrated_pose, roll_compensation,
path: selfdrive/controls/lib/latcontrol_torque.py
user: login: twilsonco
body: Very clean
created_at: 2025-03-17T05:09:11Z
author_association: CONTRIBUTOR
diff_hunk: @@ -22,6 +25,24 @@ def log_fingerprint(CP: structs.CarParams) -> None:
     sentry.capture_fingerprint(CP.carFingerprint, CP.brand)
 
 
+def initialize_neural_network_lateral_control(CP: structs.CarParams, CP_SP: structs.CarParamsSP, params) -> None:
+  nnlc_model_path, nnlc_model_name, fuzzy_fingerprint = get_nn_model_path(CP)
+
+  if nnlc_model_path is None:
+    cloudlog.error({"nnlc event": "car doesn't match any Neural Network model"})
+    nnlc_model_path = "MOCK"
+
+  if nnlc_model_path != "MOCK" and CP.steerControlType != structs.CarParams.SteerControlType.angle:
+    CP_SP.neuralNetworkLateralControl.enabled = params.get_bool("NeuralNetworkLateralControl")
path: sunnypilot/selfdrive/car/interfaces.py
user: login: twilsonco
body: I like this. Checking for model despite NNLC toggle setting. 
created_at: 2025-03-17T05:12:12Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,262 @@
+"""
+The MIT License
+
+Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
+
+Last updated: July 29, 2024
+"""
+from collections import deque
+import math
+import numpy as np
+
+from opendbc.car.interfaces import LatControlInputs
+from openpilot.common.params import Params
+from openpilot.common.filter_simple import FirstOrderFilter
+from openpilot.selfdrive.controls.lib.drive_helpers import CONTROL_N
+from openpilot.selfdrive.modeld.constants import ModelConstants
+from openpilot.sunnypilot.selfdrive.controls.lib.nnlc.flux_model import FluxModel
+
+LOW_SPEED_Y_NN = [12, 3, 1, 0]
+LAT_PLAN_MIN_IDX = 5
+
+
+def get_predicted_lateral_jerk(lat_accels, t_diffs):
+  # compute finite difference between subsequent model_v2.acceleration.y values
+  # this is just two calls of np.diff followed by an element-wise division
+  lat_accel_diffs = np.diff(lat_accels)
+  lat_jerk = lat_accel_diffs / t_diffs
+  # return as python list
+  return lat_jerk.tolist()
+
+
+def sign(x):
+  return 1.0 if x > 0.0 else (-1.0 if x < 0.0 else 0.0)
+
+
+def get_lookahead_value(future_vals, current_val):
+  if len(future_vals) == 0:
+    return current_val
+
+  same_sign_vals = [v for v in future_vals if sign(v) == sign(current_val)]
+
+  # if any future val has opposite sign of current val, return 0
+  if len(same_sign_vals) < len(future_vals):
+    return 0.0
+
+  # otherwise return the value with minimum absolute value
+  min_val = min(same_sign_vals + [current_val], key=lambda x: abs(x))
+  return min_val
+
+
+# At a given roll, if pitch magnitude increases, the
+# gravitational acceleration component starts pointing
+# in the longitudinal direction, decreasing the lateral
+# acceleration component. Here we do the same thing
+# to the roll value itself, then passed to nnff.
+def roll_pitch_adjust(roll, pitch):
+  return roll * math.cos(pitch)
+
+
+class NeuralNetworkLateralControl:
+  def __init__(self, lac_torque, CP, CP_SP):
+    self.lac_torque = lac_torque
+    self.params = Params()
+
+    self.enabled = CP_SP.neuralNetworkLateralControl.enabled
+
+    # NN model takes current v_ego, lateral_accel, lat accel/jerk error, roll, and past/future/planned data
+    # of lat accel and roll
+    # Past value is computed using previous desired lat accel and observed roll
+    # Only initialize FluxModel if enabled
+    self.flux_model = FluxModel(CP_SP.neuralNetworkLateralControl.modelPath) if self.enabled else None
+
+    self.torque_from_lateral_accel = lac_torque.torque_from_lateral_accel
+    self.torque_params = lac_torque.torque_params
+
+    self.model_v2 = None
+    self.model_valid = False
+    self.use_lateral_jerk: bool = self.params.get_bool("LateralTorqueControlLateralJerk")
+    self.use_steering_angle = lac_torque.use_steering_angle
+
+    self.actual_lateral_jerk: float = 0.0
+    self.lateral_jerk_setpoint: float = 0.0
+    self.lateral_jerk_measurement: float = 0.0
+    self.lookahead_lateral_jerk: float = 0.0
+
+    self._ff = 0.0
+    self._pid_log = None
+
+    # twilsonco's Lateral Neural Network Feedforward
+    # Instantaneous lateral jerk changes very rapidly, making it not useful on its own,
+    # however, we can "look ahead" to the future planned lateral jerk in order to gauge
+    # whether the current desired lateral jerk will persist into the future, i.e.
+    # whether it's "deliberate" or not. This allows us to simply ignore short-lived jerk.
+    # Note that LAT_PLAN_MIN_IDX is defined above and is used in order to prevent
+    # using a "future" value that is actually planned to occur before the "current" desired
+    # value, which is offset by the steerActuatorDelay.
+    self.friction_look_ahead_v = [1.4, 2.0] # how many seconds in the future to look ahead in [0, ~2.1] in 0.1 increments
+    self.friction_look_ahead_bp = [9.0, 30.0] # corresponding speeds in m/s in [0, ~40] in 1.0 increments
+
+    # Scaling the lateral acceleration "friction response" could be helpful for some.
+    # Increase for a stronger response, decrease for a weaker response.
+    self.lat_jerk_friction_factor = 0.4
+    self.lat_accel_friction_factor = 0.7 # in [0, 3], in 0.05 increments. 3 is arbitrary safety limit
+
+    # precompute time differences between ModelConstants.T_IDXS
+    self.t_diffs = np.diff(ModelConstants.T_IDXS)
+    self.desired_lat_jerk_time = CP.steerActuatorDelay + 0.3
+
+    self.pitch = FirstOrderFilter(0.0, 0.5, 0.01)
+    self.pitch_last = 0.0
+
+    # setup future time offsets
+    self.nn_time_offset = CP.steerActuatorDelay + 0.2
+    future_times = [0.3, 0.6, 1.0, 1.5] # seconds in the future
+    self.nn_future_times = [i + self.nn_time_offset for i in future_times]
+    self.nn_future_times_np = np.array(self.nn_future_times)
+
+    # setup past time offsets
+    self.past_times = [-0.3, -0.2, -0.1]
+    history_check_frames = [int(abs(i)*100) for i in self.past_times]
+    self.history_frame_offsets = [history_check_frames[0] - i for i in history_check_frames]
+    self.lateral_accel_desired_deque = deque(maxlen=history_check_frames[0])
+    self.roll_deque = deque(maxlen=history_check_frames[0])
+    self.error_deque = deque(maxlen=history_check_frames[0])
+    self.past_future_len = len(self.past_times) + len(self.nn_future_times)
+
+  def update_model_v2(self, model_v2):
+    self.model_v2 = model_v2
+    self.model_valid = self.model_v2 is not None and len(self.model_v2.orientation.x) >= CONTROL_N
+
+  def update_calculations(self, CS, VM, desired_lateral_accel):
+    self.actual_lateral_jerk = 0.0
+    self.lateral_jerk_setpoint = 0.0
+    self.lateral_jerk_measurement = 0.0
+    self.lookahead_lateral_jerk = 0.0
+
+    if self.use_steering_angle:
+      actual_curvature_rate = -VM.calc_curvature(math.radians(CS.steeringRateDeg), CS.vEgo, 0.0)
+      self.actual_lateral_jerk = actual_curvature_rate * CS.vEgo ** 2
+
+    if self.model_valid:
+      # prepare "look-ahead" desired lateral jerk
+      lookahead = np.interp(CS.vEgo, self.friction_look_ahead_bp, self.friction_look_ahead_v)
+      friction_upper_idx = next((i for i, val in enumerate(ModelConstants.T_IDXS) if val > lookahead), 16)
+      predicted_lateral_jerk = get_predicted_lateral_jerk(self.model_v2.acceleration.y, self.t_diffs)
+      desired_lateral_jerk = (np.interp(self.desired_lat_jerk_time, ModelConstants.T_IDXS,
+                                     self.model_v2.acceleration.y) - desired_lateral_accel) / self.desired_lat_jerk_time
+      self.lookahead_lateral_jerk = get_lookahead_value(predicted_lateral_jerk[LAT_PLAN_MIN_IDX:friction_upper_idx],
+                                                   desired_lateral_jerk)
+      if self.use_steering_angle or self.lookahead_lateral_jerk == 0.0:
+        self.lookahead_lateral_jerk = 0.0
+        self.actual_lateral_jerk = 0.0
+        self.lat_accel_friction_factor = 1.0
+      self.lateral_jerk_setpoint = self.lat_jerk_friction_factor * self.lookahead_lateral_jerk
+      self.lateral_jerk_measurement = self.lat_jerk_friction_factor * self.actual_lateral_jerk
+
+  def update_feed_forward(self, CS, params, setpoint, measurement, calibrated_pose,
+                            desired_lateral_accel, lateral_accel_deadzone):
+
+    if not self.enabled or not self.model_valid:
+      return
+
+    # update past data
+    roll = params.roll
+    if calibrated_pose is not None:
+      pitch = self.pitch.update(calibrated_pose.orientation.pitch)
+      roll = roll_pitch_adjust(roll, pitch)
+      self.pitch_last = pitch
+    self.roll_deque.append(roll)
+    self.lateral_accel_desired_deque.append(desired_lateral_accel)
+
+    # prepare past and future values
+    # adjust future times to account for longitudinal acceleration
+    adjusted_future_times = [t + 0.5 * CS.aEgo * (t / max(CS.vEgo, 1.0)) for t in self.nn_future_times]
+    past_rolls = [self.roll_deque[min(len(self.roll_deque) - 1, i)] for i in self.history_frame_offsets]
+    future_rolls = [roll_pitch_adjust(np.interp(t, ModelConstants.T_IDXS, self.model_v2.orientation.x) + roll,
+                                      np.interp(t, ModelConstants.T_IDXS, self.model_v2.orientation.y) + self.pitch_last) for t in
+                    adjusted_future_times]
+    past_lateral_accels_desired = [self.lateral_accel_desired_deque[min(len(self.lateral_accel_desired_deque) - 1, i)]
+                                   for i in self.history_frame_offsets]
+    future_planned_lateral_accels = [np.interp(t, ModelConstants.T_IDXS, self.model_v2.acceleration.y) for t in
+                                     adjusted_future_times]
+
+    # compute NNFF error response
+    nnff_setpoint_input = [CS.vEgo, setpoint, self.lateral_jerk_setpoint, roll] \
+                          + [setpoint] * self.past_future_len \
+                          + past_rolls + future_rolls
+    # past lateral accel error shouldn't count, so use past desired like the setpoint input
+    nnff_measurement_input = [CS.vEgo, measurement, self.lateral_jerk_measurement, roll] \
+                             + [measurement] * self.past_future_len \
+                             + past_rolls + future_rolls
+    torque_from_setpoint = self.flux_model.evaluate(nnff_setpoint_input)
+    torque_from_measurement = self.flux_model.evaluate(nnff_measurement_input)
+    self._pid_log.error = torque_from_setpoint - torque_from_measurement
+
+    # compute feedforward (same as nn setpoint output)
+    error = setpoint - measurement
+    friction_input = self.lat_accel_friction_factor * error + self.lat_jerk_friction_factor * self.lookahead_lateral_jerk
+    nn_input = [CS.vEgo, desired_lateral_accel, friction_input, roll] \
+               + past_lateral_accels_desired + future_planned_lateral_accels \
+               + past_rolls + future_rolls
+    self._ff = self.flux_model.evaluate(nn_input)
+
+    # apply friction override for cars with low NN friction response
+    if self.flux_model.friction_override:
+      self._pid_log.error += self.torque_from_lateral_accel(LatControlInputs(0.0, 0.0, CS.vEgo, CS.aEgo), self.torque_params,
+                                                      friction_input,
+                                                      lateral_accel_deadzone, friction_compensation=True,
+                                                      gravity_adjusted=False)
+
+  def update_stock_lateral_jerk(self, CS, setpoint, measurement, roll_compensation, desired_lateral_accel, actual_lateral_accel,
+                                lateral_accel_deadzone, gravity_adjusted_lateral_accel):
+    if (self.enabled and self.model_valid) or not self.use_lateral_jerk:
+      return
+
+    _error = desired_lateral_accel - actual_lateral_accel
+    friction_input = self.lat_accel_friction_factor * _error + self.lat_jerk_friction_factor * self.actual_lateral_jerk
path: sunnypilot/selfdrive/controls/lib/nnlc/nnlc.py
user: login: twilsonco
body: Here, it should be the look ahead lateral jerk (desired) that's used, not actual lateral jerk. 
created_at: 2025-03-17T05:31:58Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,262 @@
+"""
+The MIT License
+
+Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
+
+Last updated: July 29, 2024
+"""
+from collections import deque
+import math
+import numpy as np
+
+from opendbc.car.interfaces import LatControlInputs
+from openpilot.common.params import Params
+from openpilot.common.filter_simple import FirstOrderFilter
+from openpilot.selfdrive.controls.lib.drive_helpers import CONTROL_N
+from openpilot.selfdrive.modeld.constants import ModelConstants
+from openpilot.sunnypilot.selfdrive.controls.lib.nnlc.flux_model import FluxModel
+
+LOW_SPEED_Y_NN = [12, 3, 1, 0]
+LAT_PLAN_MIN_IDX = 5
+
+
+def get_predicted_lateral_jerk(lat_accels, t_diffs):
+  # compute finite difference between subsequent model_v2.acceleration.y values
+  # this is just two calls of np.diff followed by an element-wise division
+  lat_accel_diffs = np.diff(lat_accels)
+  lat_jerk = lat_accel_diffs / t_diffs
+  # return as python list
+  return lat_jerk.tolist()
+
+
+def sign(x):
+  return 1.0 if x > 0.0 else (-1.0 if x < 0.0 else 0.0)
+
+
+def get_lookahead_value(future_vals, current_val):
+  if len(future_vals) == 0:
+    return current_val
+
+  same_sign_vals = [v for v in future_vals if sign(v) == sign(current_val)]
+
+  # if any future val has opposite sign of current val, return 0
+  if len(same_sign_vals) < len(future_vals):
+    return 0.0
+
+  # otherwise return the value with minimum absolute value
+  min_val = min(same_sign_vals + [current_val], key=lambda x: abs(x))
+  return min_val
+
+
+# At a given roll, if pitch magnitude increases, the
+# gravitational acceleration component starts pointing
+# in the longitudinal direction, decreasing the lateral
+# acceleration component. Here we do the same thing
+# to the roll value itself, then passed to nnff.
+def roll_pitch_adjust(roll, pitch):
+  return roll * math.cos(pitch)
+
+
+class NeuralNetworkLateralControl:
+  def __init__(self, lac_torque, CP, CP_SP):
+    self.lac_torque = lac_torque
+    self.params = Params()
+
+    self.enabled = CP_SP.neuralNetworkLateralControl.enabled
+
+    # NN model takes current v_ego, lateral_accel, lat accel/jerk error, roll, and past/future/planned data
+    # of lat accel and roll
+    # Past value is computed using previous desired lat accel and observed roll
+    # Only initialize FluxModel if enabled
+    self.flux_model = FluxModel(CP_SP.neuralNetworkLateralControl.modelPath) if self.enabled else None
+
+    self.torque_from_lateral_accel = lac_torque.torque_from_lateral_accel
+    self.torque_params = lac_torque.torque_params
+
+    self.model_v2 = None
+    self.model_valid = False
+    self.use_lateral_jerk: bool = self.params.get_bool("LateralTorqueControlLateralJerk")
+    self.use_steering_angle = lac_torque.use_steering_angle
+
+    self.actual_lateral_jerk: float = 0.0
+    self.lateral_jerk_setpoint: float = 0.0
+    self.lateral_jerk_measurement: float = 0.0
+    self.lookahead_lateral_jerk: float = 0.0
+
+    self._ff = 0.0
+    self._pid_log = None
+
+    # twilsonco's Lateral Neural Network Feedforward
+    # Instantaneous lateral jerk changes very rapidly, making it not useful on its own,
+    # however, we can "look ahead" to the future planned lateral jerk in order to gauge
+    # whether the current desired lateral jerk will persist into the future, i.e.
+    # whether it's "deliberate" or not. This allows us to simply ignore short-lived jerk.
+    # Note that LAT_PLAN_MIN_IDX is defined above and is used in order to prevent
+    # using a "future" value that is actually planned to occur before the "current" desired
+    # value, which is offset by the steerActuatorDelay.
+    self.friction_look_ahead_v = [1.4, 2.0] # how many seconds in the future to look ahead in [0, ~2.1] in 0.1 increments
+    self.friction_look_ahead_bp = [9.0, 30.0] # corresponding speeds in m/s in [0, ~40] in 1.0 increments
+
+    # Scaling the lateral acceleration "friction response" could be helpful for some.
+    # Increase for a stronger response, decrease for a weaker response.
+    self.lat_jerk_friction_factor = 0.4
+    self.lat_accel_friction_factor = 0.7 # in [0, 3], in 0.05 increments. 3 is arbitrary safety limit
+
+    # precompute time differences between ModelConstants.T_IDXS
+    self.t_diffs = np.diff(ModelConstants.T_IDXS)
+    self.desired_lat_jerk_time = CP.steerActuatorDelay + 0.3
+
+    self.pitch = FirstOrderFilter(0.0, 0.5, 0.01)
+    self.pitch_last = 0.0
+
+    # setup future time offsets
+    self.nn_time_offset = CP.steerActuatorDelay + 0.2
+    future_times = [0.3, 0.6, 1.0, 1.5] # seconds in the future
+    self.nn_future_times = [i + self.nn_time_offset for i in future_times]
+    self.nn_future_times_np = np.array(self.nn_future_times)
+
+    # setup past time offsets
+    self.past_times = [-0.3, -0.2, -0.1]
+    history_check_frames = [int(abs(i)*100) for i in self.past_times]
+    self.history_frame_offsets = [history_check_frames[0] - i for i in history_check_frames]
+    self.lateral_accel_desired_deque = deque(maxlen=history_check_frames[0])
+    self.roll_deque = deque(maxlen=history_check_frames[0])
+    self.error_deque = deque(maxlen=history_check_frames[0])
+    self.past_future_len = len(self.past_times) + len(self.nn_future_times)
+
+  def update_model_v2(self, model_v2):
+    self.model_v2 = model_v2
+    self.model_valid = self.model_v2 is not None and len(self.model_v2.orientation.x) >= CONTROL_N
+
+  def update_calculations(self, CS, VM, desired_lateral_accel):
+    self.actual_lateral_jerk = 0.0
+    self.lateral_jerk_setpoint = 0.0
+    self.lateral_jerk_measurement = 0.0
+    self.lookahead_lateral_jerk = 0.0
+
+    if self.use_steering_angle:
+      actual_curvature_rate = -VM.calc_curvature(math.radians(CS.steeringRateDeg), CS.vEgo, 0.0)
+      self.actual_lateral_jerk = actual_curvature_rate * CS.vEgo ** 2
+
+    if self.model_valid:
+      # prepare "look-ahead" desired lateral jerk
+      lookahead = np.interp(CS.vEgo, self.friction_look_ahead_bp, self.friction_look_ahead_v)
+      friction_upper_idx = next((i for i, val in enumerate(ModelConstants.T_IDXS) if val > lookahead), 16)
+      predicted_lateral_jerk = get_predicted_lateral_jerk(self.model_v2.acceleration.y, self.t_diffs)
+      desired_lateral_jerk = (np.interp(self.desired_lat_jerk_time, ModelConstants.T_IDXS,
+                                     self.model_v2.acceleration.y) - desired_lateral_accel) / self.desired_lat_jerk_time
+      self.lookahead_lateral_jerk = get_lookahead_value(predicted_lateral_jerk[LAT_PLAN_MIN_IDX:friction_upper_idx],
+                                                   desired_lateral_jerk)
+      if self.use_steering_angle or self.lookahead_lateral_jerk == 0.0:
+        self.lookahead_lateral_jerk = 0.0
+        self.actual_lateral_jerk = 0.0
+        self.lat_accel_friction_factor = 1.0
+      self.lateral_jerk_setpoint = self.lat_jerk_friction_factor * self.lookahead_lateral_jerk
+      self.lateral_jerk_measurement = self.lat_jerk_friction_factor * self.actual_lateral_jerk
+
+  def update_feed_forward(self, CS, params, setpoint, measurement, calibrated_pose,
+                            desired_lateral_accel, lateral_accel_deadzone):
+
+    if not self.enabled or not self.model_valid:
+      return
+
+    # update past data
+    roll = params.roll
+    if calibrated_pose is not None:
+      pitch = self.pitch.update(calibrated_pose.orientation.pitch)
+      roll = roll_pitch_adjust(roll, pitch)
+      self.pitch_last = pitch
+    self.roll_deque.append(roll)
+    self.lateral_accel_desired_deque.append(desired_lateral_accel)
+
+    # prepare past and future values
+    # adjust future times to account for longitudinal acceleration
+    adjusted_future_times = [t + 0.5 * CS.aEgo * (t / max(CS.vEgo, 1.0)) for t in self.nn_future_times]
+    past_rolls = [self.roll_deque[min(len(self.roll_deque) - 1, i)] for i in self.history_frame_offsets]
+    future_rolls = [roll_pitch_adjust(np.interp(t, ModelConstants.T_IDXS, self.model_v2.orientation.x) + roll,
+                                      np.interp(t, ModelConstants.T_IDXS, self.model_v2.orientation.y) + self.pitch_last) for t in
+                    adjusted_future_times]
+    past_lateral_accels_desired = [self.lateral_accel_desired_deque[min(len(self.lateral_accel_desired_deque) - 1, i)]
+                                   for i in self.history_frame_offsets]
+    future_planned_lateral_accels = [np.interp(t, ModelConstants.T_IDXS, self.model_v2.acceleration.y) for t in
+                                     adjusted_future_times]
+
+    # compute NNFF error response
+    nnff_setpoint_input = [CS.vEgo, setpoint, self.lateral_jerk_setpoint, roll] \
+                          + [setpoint] * self.past_future_len \
+                          + past_rolls + future_rolls
+    # past lateral accel error shouldn't count, so use past desired like the setpoint input
+    nnff_measurement_input = [CS.vEgo, measurement, self.lateral_jerk_measurement, roll] \
+                             + [measurement] * self.past_future_len \
+                             + past_rolls + future_rolls
+    torque_from_setpoint = self.flux_model.evaluate(nnff_setpoint_input)
+    torque_from_measurement = self.flux_model.evaluate(nnff_measurement_input)
+    self._pid_log.error = torque_from_setpoint - torque_from_measurement
+
+    # compute feedforward (same as nn setpoint output)
+    error = setpoint - measurement
+    friction_input = self.lat_accel_friction_factor * error + self.lat_jerk_friction_factor * self.lookahead_lateral_jerk
+    nn_input = [CS.vEgo, desired_lateral_accel, friction_input, roll] \
+               + past_lateral_accels_desired + future_planned_lateral_accels \
+               + past_rolls + future_rolls
+    self._ff = self.flux_model.evaluate(nn_input)
+
+    # apply friction override for cars with low NN friction response
+    if self.flux_model.friction_override:
+      self._pid_log.error += self.torque_from_lateral_accel(LatControlInputs(0.0, 0.0, CS.vEgo, CS.aEgo), self.torque_params,
+                                                      friction_input,
+                                                      lateral_accel_deadzone, friction_compensation=True,
+                                                      gravity_adjusted=False)
+
+  def update_stock_lateral_jerk(self, CS, setpoint, measurement, roll_compensation, desired_lateral_accel, actual_lateral_accel,
+                                lateral_accel_deadzone, gravity_adjusted_lateral_accel):
+    if (self.enabled and self.model_valid) or not self.use_lateral_jerk:
+      return
+
+    _error = desired_lateral_accel - actual_lateral_accel
+    friction_input = self.lat_accel_friction_factor * _error + self.lat_jerk_friction_factor * self.actual_lateral_jerk
path: sunnypilot/selfdrive/controls/lib/nnlc/nnlc.py
user: login: twilsonco
body: Perhaps a better approach, that avoids redundant code, is to call this method from inside the update feedforward method. That way, look ahead lateral jerk can be computed once and passed to this method. 
created_at: 2025-03-17T05:33:39Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,262 @@
+"""
+The MIT License
+
+Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
+
+Last updated: July 29, 2024
+"""
+from collections import deque
+import math
+import numpy as np
+
+from opendbc.car.interfaces import LatControlInputs
+from openpilot.common.params import Params
+from openpilot.common.filter_simple import FirstOrderFilter
+from openpilot.selfdrive.controls.lib.drive_helpers import CONTROL_N
+from openpilot.selfdrive.modeld.constants import ModelConstants
+from openpilot.sunnypilot.selfdrive.controls.lib.nnlc.flux_model import FluxModel
+
+LOW_SPEED_Y_NN = [12, 3, 1, 0]
+LAT_PLAN_MIN_IDX = 5
+
+
+def get_predicted_lateral_jerk(lat_accels, t_diffs):
+  # compute finite difference between subsequent model_v2.acceleration.y values
+  # this is just two calls of np.diff followed by an element-wise division
+  lat_accel_diffs = np.diff(lat_accels)
+  lat_jerk = lat_accel_diffs / t_diffs
+  # return as python list
+  return lat_jerk.tolist()
+
+
+def sign(x):
+  return 1.0 if x > 0.0 else (-1.0 if x < 0.0 else 0.0)
+
+
+def get_lookahead_value(future_vals, current_val):
+  if len(future_vals) == 0:
+    return current_val
+
+  same_sign_vals = [v for v in future_vals if sign(v) == sign(current_val)]
+
+  # if any future val has opposite sign of current val, return 0
+  if len(same_sign_vals) < len(future_vals):
+    return 0.0
+
+  # otherwise return the value with minimum absolute value
+  min_val = min(same_sign_vals + [current_val], key=lambda x: abs(x))
+  return min_val
+
+
+# At a given roll, if pitch magnitude increases, the
+# gravitational acceleration component starts pointing
+# in the longitudinal direction, decreasing the lateral
+# acceleration component. Here we do the same thing
+# to the roll value itself, then passed to nnff.
+def roll_pitch_adjust(roll, pitch):
+  return roll * math.cos(pitch)
+
+
+class NeuralNetworkLateralControl:
+  def __init__(self, lac_torque, CP, CP_SP):
+    self.lac_torque = lac_torque
+    self.params = Params()
+
+    self.enabled = CP_SP.neuralNetworkLateralControl.enabled
+
+    # NN model takes current v_ego, lateral_accel, lat accel/jerk error, roll, and past/future/planned data
+    # of lat accel and roll
+    # Past value is computed using previous desired lat accel and observed roll
+    # Only initialize FluxModel if enabled
+    self.flux_model = FluxModel(CP_SP.neuralNetworkLateralControl.modelPath) if self.enabled else None
+
+    self.torque_from_lateral_accel = lac_torque.torque_from_lateral_accel
+    self.torque_params = lac_torque.torque_params
+
+    self.model_v2 = None
+    self.model_valid = False
+    self.use_lateral_jerk: bool = self.params.get_bool("LateralTorqueControlLateralJerk")
+    self.use_steering_angle = lac_torque.use_steering_angle
+
+    self.actual_lateral_jerk: float = 0.0
+    self.lateral_jerk_setpoint: float = 0.0
+    self.lateral_jerk_measurement: float = 0.0
+    self.lookahead_lateral_jerk: float = 0.0
+
+    self._ff = 0.0
+    self._pid_log = None
+
+    # twilsonco's Lateral Neural Network Feedforward
+    # Instantaneous lateral jerk changes very rapidly, making it not useful on its own,
+    # however, we can "look ahead" to the future planned lateral jerk in order to gauge
+    # whether the current desired lateral jerk will persist into the future, i.e.
+    # whether it's "deliberate" or not. This allows us to simply ignore short-lived jerk.
+    # Note that LAT_PLAN_MIN_IDX is defined above and is used in order to prevent
+    # using a "future" value that is actually planned to occur before the "current" desired
+    # value, which is offset by the steerActuatorDelay.
+    self.friction_look_ahead_v = [1.4, 2.0] # how many seconds in the future to look ahead in [0, ~2.1] in 0.1 increments
+    self.friction_look_ahead_bp = [9.0, 30.0] # corresponding speeds in m/s in [0, ~40] in 1.0 increments
+
+    # Scaling the lateral acceleration "friction response" could be helpful for some.
+    # Increase for a stronger response, decrease for a weaker response.
+    self.lat_jerk_friction_factor = 0.4
+    self.lat_accel_friction_factor = 0.7 # in [0, 3], in 0.05 increments. 3 is arbitrary safety limit
+
+    # precompute time differences between ModelConstants.T_IDXS
+    self.t_diffs = np.diff(ModelConstants.T_IDXS)
+    self.desired_lat_jerk_time = CP.steerActuatorDelay + 0.3
+
+    self.pitch = FirstOrderFilter(0.0, 0.5, 0.01)
+    self.pitch_last = 0.0
+
+    # setup future time offsets
+    self.nn_time_offset = CP.steerActuatorDelay + 0.2
+    future_times = [0.3, 0.6, 1.0, 1.5] # seconds in the future
+    self.nn_future_times = [i + self.nn_time_offset for i in future_times]
+    self.nn_future_times_np = np.array(self.nn_future_times)
+
+    # setup past time offsets
+    self.past_times = [-0.3, -0.2, -0.1]
+    history_check_frames = [int(abs(i)*100) for i in self.past_times]
+    self.history_frame_offsets = [history_check_frames[0] - i for i in history_check_frames]
+    self.lateral_accel_desired_deque = deque(maxlen=history_check_frames[0])
+    self.roll_deque = deque(maxlen=history_check_frames[0])
+    self.error_deque = deque(maxlen=history_check_frames[0])
+    self.past_future_len = len(self.past_times) + len(self.nn_future_times)
+
+  def update_model_v2(self, model_v2):
+    self.model_v2 = model_v2
+    self.model_valid = self.model_v2 is not None and len(self.model_v2.orientation.x) >= CONTROL_N
+
+  def update_calculations(self, CS, VM, desired_lateral_accel):
+    self.actual_lateral_jerk = 0.0
+    self.lateral_jerk_setpoint = 0.0
+    self.lateral_jerk_measurement = 0.0
+    self.lookahead_lateral_jerk = 0.0
+
+    if self.use_steering_angle:
+      actual_curvature_rate = -VM.calc_curvature(math.radians(CS.steeringRateDeg), CS.vEgo, 0.0)
+      self.actual_lateral_jerk = actual_curvature_rate * CS.vEgo ** 2
+
+    if self.model_valid:
+      # prepare "look-ahead" desired lateral jerk
+      lookahead = np.interp(CS.vEgo, self.friction_look_ahead_bp, self.friction_look_ahead_v)
+      friction_upper_idx = next((i for i, val in enumerate(ModelConstants.T_IDXS) if val > lookahead), 16)
+      predicted_lateral_jerk = get_predicted_lateral_jerk(self.model_v2.acceleration.y, self.t_diffs)
+      desired_lateral_jerk = (np.interp(self.desired_lat_jerk_time, ModelConstants.T_IDXS,
+                                     self.model_v2.acceleration.y) - desired_lateral_accel) / self.desired_lat_jerk_time
+      self.lookahead_lateral_jerk = get_lookahead_value(predicted_lateral_jerk[LAT_PLAN_MIN_IDX:friction_upper_idx],
+                                                   desired_lateral_jerk)
+      if self.use_steering_angle or self.lookahead_lateral_jerk == 0.0:
+        self.lookahead_lateral_jerk = 0.0
+        self.actual_lateral_jerk = 0.0
+        self.lat_accel_friction_factor = 1.0
+      self.lateral_jerk_setpoint = self.lat_jerk_friction_factor * self.lookahead_lateral_jerk
+      self.lateral_jerk_measurement = self.lat_jerk_friction_factor * self.actual_lateral_jerk
+
+  def update_feed_forward(self, CS, params, setpoint, measurement, calibrated_pose,
+                            desired_lateral_accel, lateral_accel_deadzone):
+
+    if not self.enabled or not self.model_valid:
+      return
+
+    # update past data
+    roll = params.roll
+    if calibrated_pose is not None:
+      pitch = self.pitch.update(calibrated_pose.orientation.pitch)
+      roll = roll_pitch_adjust(roll, pitch)
+      self.pitch_last = pitch
+    self.roll_deque.append(roll)
+    self.lateral_accel_desired_deque.append(desired_lateral_accel)
+
+    # prepare past and future values
+    # adjust future times to account for longitudinal acceleration
+    adjusted_future_times = [t + 0.5 * CS.aEgo * (t / max(CS.vEgo, 1.0)) for t in self.nn_future_times]
+    past_rolls = [self.roll_deque[min(len(self.roll_deque) - 1, i)] for i in self.history_frame_offsets]
+    future_rolls = [roll_pitch_adjust(np.interp(t, ModelConstants.T_IDXS, self.model_v2.orientation.x) + roll,
+                                      np.interp(t, ModelConstants.T_IDXS, self.model_v2.orientation.y) + self.pitch_last) for t in
+                    adjusted_future_times]
+    past_lateral_accels_desired = [self.lateral_accel_desired_deque[min(len(self.lateral_accel_desired_deque) - 1, i)]
+                                   for i in self.history_frame_offsets]
+    future_planned_lateral_accels = [np.interp(t, ModelConstants.T_IDXS, self.model_v2.acceleration.y) for t in
+                                     adjusted_future_times]
+
+    # compute NNFF error response
+    nnff_setpoint_input = [CS.vEgo, setpoint, self.lateral_jerk_setpoint, roll] \
+                          + [setpoint] * self.past_future_len \
+                          + past_rolls + future_rolls
+    # past lateral accel error shouldn't count, so use past desired like the setpoint input
+    nnff_measurement_input = [CS.vEgo, measurement, self.lateral_jerk_measurement, roll] \
+                             + [measurement] * self.past_future_len \
+                             + past_rolls + future_rolls
+    torque_from_setpoint = self.flux_model.evaluate(nnff_setpoint_input)
+    torque_from_measurement = self.flux_model.evaluate(nnff_measurement_input)
+    self._pid_log.error = torque_from_setpoint - torque_from_measurement
+
+    # compute feedforward (same as nn setpoint output)
+    error = setpoint - measurement
+    friction_input = self.lat_accel_friction_factor * error + self.lat_jerk_friction_factor * self.lookahead_lateral_jerk
+    nn_input = [CS.vEgo, desired_lateral_accel, friction_input, roll] \
+               + past_lateral_accels_desired + future_planned_lateral_accels \
+               + past_rolls + future_rolls
+    self._ff = self.flux_model.evaluate(nn_input)
+
+    # apply friction override for cars with low NN friction response
+    if self.flux_model.friction_override:
+      self._pid_log.error += self.torque_from_lateral_accel(LatControlInputs(0.0, 0.0, CS.vEgo, CS.aEgo), self.torque_params,
+                                                      friction_input,
+                                                      lateral_accel_deadzone, friction_compensation=True,
+                                                      gravity_adjusted=False)
+
+  def update_stock_lateral_jerk(self, CS, setpoint, measurement, roll_compensation, desired_lateral_accel, actual_lateral_accel,
+                                lateral_accel_deadzone, gravity_adjusted_lateral_accel):
+    if (self.enabled and self.model_valid) or not self.use_lateral_jerk:
+      return
+
+    _error = desired_lateral_accel - actual_lateral_accel
+    friction_input = self.lat_accel_friction_factor * _error + self.lat_jerk_friction_factor * self.actual_lateral_jerk
path: sunnypilot/selfdrive/controls/lib/nnlc/nnlc.py
user: login: twilsonco
body: Or save the lookahead jerk to an attribute in the feedforward method so it can be used in the lateral jerk method. 
created_at: 2025-03-17T05:44:46Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,262 @@
+"""
+The MIT License
+
+Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
+
+Last updated: July 29, 2024
+"""
+from collections import deque
+import math
+import numpy as np
+
+from opendbc.car.interfaces import LatControlInputs
+from openpilot.common.params import Params
+from openpilot.common.filter_simple import FirstOrderFilter
+from openpilot.selfdrive.controls.lib.drive_helpers import CONTROL_N
+from openpilot.selfdrive.modeld.constants import ModelConstants
+from openpilot.sunnypilot.selfdrive.controls.lib.nnlc.flux_model import FluxModel
+
+LOW_SPEED_Y_NN = [12, 3, 1, 0]
+LAT_PLAN_MIN_IDX = 5
+
+
+def get_predicted_lateral_jerk(lat_accels, t_diffs):
+  # compute finite difference between subsequent model_v2.acceleration.y values
+  # this is just two calls of np.diff followed by an element-wise division
+  lat_accel_diffs = np.diff(lat_accels)
+  lat_jerk = lat_accel_diffs / t_diffs
+  # return as python list
+  return lat_jerk.tolist()
+
+
+def sign(x):
+  return 1.0 if x > 0.0 else (-1.0 if x < 0.0 else 0.0)
+
+
+def get_lookahead_value(future_vals, current_val):
+  if len(future_vals) == 0:
+    return current_val
+
+  same_sign_vals = [v for v in future_vals if sign(v) == sign(current_val)]
+
+  # if any future val has opposite sign of current val, return 0
+  if len(same_sign_vals) < len(future_vals):
+    return 0.0
+
+  # otherwise return the value with minimum absolute value
+  min_val = min(same_sign_vals + [current_val], key=lambda x: abs(x))
+  return min_val
+
+
+# At a given roll, if pitch magnitude increases, the
+# gravitational acceleration component starts pointing
+# in the longitudinal direction, decreasing the lateral
+# acceleration component. Here we do the same thing
+# to the roll value itself, then passed to nnff.
+def roll_pitch_adjust(roll, pitch):
+  return roll * math.cos(pitch)
+
+
+class NeuralNetworkLateralControl:
+  def __init__(self, lac_torque, CP, CP_SP):
+    self.lac_torque = lac_torque
+    self.params = Params()
+
+    self.enabled = CP_SP.neuralNetworkLateralControl.enabled
+
+    # NN model takes current v_ego, lateral_accel, lat accel/jerk error, roll, and past/future/planned data
+    # of lat accel and roll
+    # Past value is computed using previous desired lat accel and observed roll
+    # Only initialize FluxModel if enabled
+    self.flux_model = FluxModel(CP_SP.neuralNetworkLateralControl.modelPath) if self.enabled else None
+
+    self.torque_from_lateral_accel = lac_torque.torque_from_lateral_accel
+    self.torque_params = lac_torque.torque_params
+
+    self.model_v2 = None
+    self.model_valid = False
+    self.use_lateral_jerk: bool = self.params.get_bool("LateralTorqueControlLateralJerk")
+    self.use_steering_angle = lac_torque.use_steering_angle
+
+    self.actual_lateral_jerk: float = 0.0
+    self.lateral_jerk_setpoint: float = 0.0
+    self.lateral_jerk_measurement: float = 0.0
+    self.lookahead_lateral_jerk: float = 0.0
+
+    self._ff = 0.0
+    self._pid_log = None
+
+    # twilsonco's Lateral Neural Network Feedforward
+    # Instantaneous lateral jerk changes very rapidly, making it not useful on its own,
+    # however, we can "look ahead" to the future planned lateral jerk in order to gauge
+    # whether the current desired lateral jerk will persist into the future, i.e.
+    # whether it's "deliberate" or not. This allows us to simply ignore short-lived jerk.
+    # Note that LAT_PLAN_MIN_IDX is defined above and is used in order to prevent
+    # using a "future" value that is actually planned to occur before the "current" desired
+    # value, which is offset by the steerActuatorDelay.
+    self.friction_look_ahead_v = [1.4, 2.0] # how many seconds in the future to look ahead in [0, ~2.1] in 0.1 increments
+    self.friction_look_ahead_bp = [9.0, 30.0] # corresponding speeds in m/s in [0, ~40] in 1.0 increments
+
+    # Scaling the lateral acceleration "friction response" could be helpful for some.
+    # Increase for a stronger response, decrease for a weaker response.
+    self.lat_jerk_friction_factor = 0.4
+    self.lat_accel_friction_factor = 0.7 # in [0, 3], in 0.05 increments. 3 is arbitrary safety limit
+
+    # precompute time differences between ModelConstants.T_IDXS
+    self.t_diffs = np.diff(ModelConstants.T_IDXS)
+    self.desired_lat_jerk_time = CP.steerActuatorDelay + 0.3
+
+    self.pitch = FirstOrderFilter(0.0, 0.5, 0.01)
+    self.pitch_last = 0.0
+
+    # setup future time offsets
+    self.nn_time_offset = CP.steerActuatorDelay + 0.2
+    future_times = [0.3, 0.6, 1.0, 1.5] # seconds in the future
+    self.nn_future_times = [i + self.nn_time_offset for i in future_times]
+    self.nn_future_times_np = np.array(self.nn_future_times)
+
+    # setup past time offsets
+    self.past_times = [-0.3, -0.2, -0.1]
+    history_check_frames = [int(abs(i)*100) for i in self.past_times]
+    self.history_frame_offsets = [history_check_frames[0] - i for i in history_check_frames]
+    self.lateral_accel_desired_deque = deque(maxlen=history_check_frames[0])
+    self.roll_deque = deque(maxlen=history_check_frames[0])
+    self.error_deque = deque(maxlen=history_check_frames[0])
+    self.past_future_len = len(self.past_times) + len(self.nn_future_times)
+
+  def update_model_v2(self, model_v2):
+    self.model_v2 = model_v2
+    self.model_valid = self.model_v2 is not None and len(self.model_v2.orientation.x) >= CONTROL_N
+
+  def update_calculations(self, CS, VM, desired_lateral_accel):
+    self.actual_lateral_jerk = 0.0
+    self.lateral_jerk_setpoint = 0.0
+    self.lateral_jerk_measurement = 0.0
+    self.lookahead_lateral_jerk = 0.0
+
+    if self.use_steering_angle:
+      actual_curvature_rate = -VM.calc_curvature(math.radians(CS.steeringRateDeg), CS.vEgo, 0.0)
+      self.actual_lateral_jerk = actual_curvature_rate * CS.vEgo ** 2
+
+    if self.model_valid:
+      # prepare "look-ahead" desired lateral jerk
+      lookahead = np.interp(CS.vEgo, self.friction_look_ahead_bp, self.friction_look_ahead_v)
+      friction_upper_idx = next((i for i, val in enumerate(ModelConstants.T_IDXS) if val > lookahead), 16)
+      predicted_lateral_jerk = get_predicted_lateral_jerk(self.model_v2.acceleration.y, self.t_diffs)
+      desired_lateral_jerk = (np.interp(self.desired_lat_jerk_time, ModelConstants.T_IDXS,
+                                     self.model_v2.acceleration.y) - desired_lateral_accel) / self.desired_lat_jerk_time
+      self.lookahead_lateral_jerk = get_lookahead_value(predicted_lateral_jerk[LAT_PLAN_MIN_IDX:friction_upper_idx],
+                                                   desired_lateral_jerk)
+      if self.use_steering_angle or self.lookahead_lateral_jerk == 0.0:
+        self.lookahead_lateral_jerk = 0.0
+        self.actual_lateral_jerk = 0.0
+        self.lat_accel_friction_factor = 1.0
+      self.lateral_jerk_setpoint = self.lat_jerk_friction_factor * self.lookahead_lateral_jerk
+      self.lateral_jerk_measurement = self.lat_jerk_friction_factor * self.actual_lateral_jerk
+
+  def update_feed_forward(self, CS, params, setpoint, measurement, calibrated_pose,
+                            desired_lateral_accel, lateral_accel_deadzone):
+
+    if not self.enabled or not self.model_valid:
+      return
+
+    # update past data
+    roll = params.roll
+    if calibrated_pose is not None:
+      pitch = self.pitch.update(calibrated_pose.orientation.pitch)
+      roll = roll_pitch_adjust(roll, pitch)
+      self.pitch_last = pitch
+    self.roll_deque.append(roll)
+    self.lateral_accel_desired_deque.append(desired_lateral_accel)
+
+    # prepare past and future values
+    # adjust future times to account for longitudinal acceleration
+    adjusted_future_times = [t + 0.5 * CS.aEgo * (t / max(CS.vEgo, 1.0)) for t in self.nn_future_times]
+    past_rolls = [self.roll_deque[min(len(self.roll_deque) - 1, i)] for i in self.history_frame_offsets]
+    future_rolls = [roll_pitch_adjust(np.interp(t, ModelConstants.T_IDXS, self.model_v2.orientation.x) + roll,
+                                      np.interp(t, ModelConstants.T_IDXS, self.model_v2.orientation.y) + self.pitch_last) for t in
+                    adjusted_future_times]
+    past_lateral_accels_desired = [self.lateral_accel_desired_deque[min(len(self.lateral_accel_desired_deque) - 1, i)]
+                                   for i in self.history_frame_offsets]
+    future_planned_lateral_accels = [np.interp(t, ModelConstants.T_IDXS, self.model_v2.acceleration.y) for t in
+                                     adjusted_future_times]
+
+    # compute NNFF error response
+    nnff_setpoint_input = [CS.vEgo, setpoint, self.lateral_jerk_setpoint, roll] \
+                          + [setpoint] * self.past_future_len \
+                          + past_rolls + future_rolls
+    # past lateral accel error shouldn't count, so use past desired like the setpoint input
+    nnff_measurement_input = [CS.vEgo, measurement, self.lateral_jerk_measurement, roll] \
+                             + [measurement] * self.past_future_len \
+                             + past_rolls + future_rolls
+    torque_from_setpoint = self.flux_model.evaluate(nnff_setpoint_input)
+    torque_from_measurement = self.flux_model.evaluate(nnff_measurement_input)
+    self._pid_log.error = torque_from_setpoint - torque_from_measurement
+
+    # compute feedforward (same as nn setpoint output)
+    error = setpoint - measurement
+    friction_input = self.lat_accel_friction_factor * error + self.lat_jerk_friction_factor * self.lookahead_lateral_jerk
+    nn_input = [CS.vEgo, desired_lateral_accel, friction_input, roll] \
+               + past_lateral_accels_desired + future_planned_lateral_accels \
+               + past_rolls + future_rolls
+    self._ff = self.flux_model.evaluate(nn_input)
+
+    # apply friction override for cars with low NN friction response
+    if self.flux_model.friction_override:
+      self._pid_log.error += self.torque_from_lateral_accel(LatControlInputs(0.0, 0.0, CS.vEgo, CS.aEgo), self.torque_params,
+                                                      friction_input,
+                                                      lateral_accel_deadzone, friction_compensation=True,
+                                                      gravity_adjusted=False)
+
+  def update_stock_lateral_jerk(self, CS, setpoint, measurement, roll_compensation, desired_lateral_accel, actual_lateral_accel,
+                                lateral_accel_deadzone, gravity_adjusted_lateral_accel):
+    if (self.enabled and self.model_valid) or not self.use_lateral_jerk:
+      return
+
+    _error = desired_lateral_accel - actual_lateral_accel
+    friction_input = self.lat_accel_friction_factor * _error + self.lat_jerk_friction_factor * self.actual_lateral_jerk
path: sunnypilot/selfdrive/controls/lib/nnlc/nnlc.py
user: login: sunnyhaibin
body: @twilsonco https://github.com/sunnypilot/sunnypilot/pull/667/commits/6b4ac127df0776f334af9dadee7396309606c869
created_at: 2025-03-17T18:16:46Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,113 @@
+import math
+import numpy as np
+
+from openpilot.selfdrive.controls.lib.drive_helpers import CONTROL_N
+from openpilot.selfdrive.modeld.constants import ModelConstants
+
+LAT_PLAN_MIN_IDX = 5
+
+
+def get_predicted_lateral_jerk(lat_accels, t_diffs):
+  # compute finite difference between subsequent model_v2.acceleration.y values
+  # this is just two calls of np.diff followed by an element-wise division
+  lat_accel_diffs = np.diff(lat_accels)
+  lat_jerk = lat_accel_diffs / t_diffs
+  # return as python list
+  return lat_jerk.tolist()
+
+
+def sign(x):
+  return 1.0 if x > 0.0 else (-1.0 if x < 0.0 else 0.0)
+
+
+def get_lookahead_value(future_vals, current_val):
+  if len(future_vals) == 0:
+    return current_val
+
+  same_sign_vals = [v for v in future_vals if sign(v) == sign(current_val)]
+
+  # if any future val has opposite sign of current val, return 0
+  if len(same_sign_vals) < len(future_vals):
+    return 0.0
+
+  # otherwise return the value with minimum absolute value
+  min_val = min(same_sign_vals + [current_val], key=lambda x: abs(x))
+  return min_val
+
+
+class LatControlTorqueExtBase:
+  def __init__(self, lac_torque, CP, CP_SP):
+    self.model_v2 = None
+    self.model_valid = False
+    self.use_steering_angle = lac_torque.use_steering_angle
+
+    self.actual_lateral_jerk: float = 0.0
+    self.lateral_jerk_setpoint: float = 0.0
+    self.lateral_jerk_measurement: float = 0.0
+    self.lookahead_lateral_jerk: float = 0.0
+
+    self.torque_from_lateral_accel = lac_torque.torque_from_lateral_accel
+    self.torque_params = lac_torque.torque_params
+
+    self._ff = 0.0
+    self._pid_log = None
+    self._setpoint = 0.0
+    self._measurement = 0.0
+    self._lateral_accel_deadzone = 0.0
+    self._desired_lateral_accel = 0.0
+    self._actual_lateral_accel = 0.0
+
+    # twilsonco's Lateral Neural Network Feedforward
+    # Instantaneous lateral jerk changes very rapidly, making it not useful on its own,
+    # however, we can "look ahead" to the future planned lateral jerk in order to gauge
+    # whether the current desired lateral jerk will persist into the future, i.e.
+    # whether it's "deliberate" or not. This allows us to simply ignore short-lived jerk.
+    # Note that LAT_PLAN_MIN_IDX is defined above and is used in order to prevent
+    # using a "future" value that is actually planned to occur before the "current" desired
+    # value, which is offset by the steerActuatorDelay.
+    self.friction_look_ahead_v = [1.4, 2.0]  # how many seconds in the future to look ahead in [0, ~2.1] in 0.1 increments
+    self.friction_look_ahead_bp = [9.0, 30.0]  # corresponding speeds in m/s in [0, ~40] in 1.0 increments
+
+    # Scaling the lateral acceleration "friction response" could be helpful for some.
+    # Increase for a stronger response, decrease for a weaker response.
+    self.lat_jerk_friction_factor = 0.4
+    self.lat_accel_friction_factor = 0.7  # in [0, 3], in 0.05 increments. 3 is arbitrary safety limit
+
+    # precompute time differences between ModelConstants.T_IDXS
+    self.t_diffs = np.diff(ModelConstants.T_IDXS)
+    self.desired_lat_jerk_time = CP.steerActuatorDelay + 0.3
+
+  def update_model_v2(self, model_v2):
+    self.model_v2 = model_v2
+    self.model_valid = self.model_v2 is not None and len(self.model_v2.orientation.x) >= CONTROL_N
+
+  def update_friction_input(self, val_1, val_2):
+    _error = val_1 - val_2
+    _value = self.lat_accel_friction_factor * _error + self.lat_jerk_friction_factor * self.lookahead_lateral_jerk
+
+    return _value
+
+  def update_calculations(self, CS, VM, desired_lateral_accel):
+    self.actual_lateral_jerk = 0.0
+    self.lateral_jerk_setpoint = 0.0
+    self.lateral_jerk_measurement = 0.0
+    self.lookahead_lateral_jerk = 0.0
+
+    if self.use_steering_angle:
+      actual_curvature_rate = -VM.calc_curvature(math.radians(CS.steeringRateDeg), CS.vEgo, 0.0)
+      self.actual_lateral_jerk = actual_curvature_rate * CS.vEgo ** 2
+
+    if self.model_valid:
+      # prepare "look-ahead" desired lateral jerk
+      lookahead = np.interp(CS.vEgo, self.friction_look_ahead_bp, self.friction_look_ahead_v)
+      friction_upper_idx = next((i for i, val in enumerate(ModelConstants.T_IDXS) if val > lookahead), 16)
+      predicted_lateral_jerk = get_predicted_lateral_jerk(self.model_v2.acceleration.y, self.t_diffs)
+      desired_lateral_jerk = (np.interp(self.desired_lat_jerk_time, ModelConstants.T_IDXS,
+                              self.model_v2.acceleration.y) - desired_lateral_accel) / self.desired_lat_jerk_time
+      self.lookahead_lateral_jerk = get_lookahead_value(predicted_lateral_jerk[LAT_PLAN_MIN_IDX:friction_upper_idx], desired_lateral_jerk)
+      if self.use_steering_angle or self.lookahead_lateral_jerk == 0.0:
path: sunnypilot/selfdrive/controls/lib/latcontrol_torque_ext_base.py
user: login: twilsonco
body: Oh my.... realizing this way too late. This should be `if not self.use_steering_angle`. Same change in `dev-c3` and my own fork.
created_at: 2025-03-20T16:18:03Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,113 @@
+import math
+import numpy as np
+
+from openpilot.selfdrive.controls.lib.drive_helpers import CONTROL_N
+from openpilot.selfdrive.modeld.constants import ModelConstants
+
+LAT_PLAN_MIN_IDX = 5
+
+
+def get_predicted_lateral_jerk(lat_accels, t_diffs):
+  # compute finite difference between subsequent model_v2.acceleration.y values
+  # this is just two calls of np.diff followed by an element-wise division
+  lat_accel_diffs = np.diff(lat_accels)
+  lat_jerk = lat_accel_diffs / t_diffs
+  # return as python list
+  return lat_jerk.tolist()
+
+
+def sign(x):
+  return 1.0 if x > 0.0 else (-1.0 if x < 0.0 else 0.0)
+
+
+def get_lookahead_value(future_vals, current_val):
+  if len(future_vals) == 0:
+    return current_val
+
+  same_sign_vals = [v for v in future_vals if sign(v) == sign(current_val)]
+
+  # if any future val has opposite sign of current val, return 0
+  if len(same_sign_vals) < len(future_vals):
+    return 0.0
+
+  # otherwise return the value with minimum absolute value
+  min_val = min(same_sign_vals + [current_val], key=lambda x: abs(x))
+  return min_val
+
+
+class LatControlTorqueExtBase:
+  def __init__(self, lac_torque, CP, CP_SP):
+    self.model_v2 = None
+    self.model_valid = False
+    self.use_steering_angle = lac_torque.use_steering_angle
+
+    self.actual_lateral_jerk: float = 0.0
+    self.lateral_jerk_setpoint: float = 0.0
+    self.lateral_jerk_measurement: float = 0.0
+    self.lookahead_lateral_jerk: float = 0.0
+
+    self.torque_from_lateral_accel = lac_torque.torque_from_lateral_accel
+    self.torque_params = lac_torque.torque_params
+
+    self._ff = 0.0
+    self._pid_log = None
+    self._setpoint = 0.0
+    self._measurement = 0.0
+    self._lateral_accel_deadzone = 0.0
+    self._desired_lateral_accel = 0.0
+    self._actual_lateral_accel = 0.0
+
+    # twilsonco's Lateral Neural Network Feedforward
+    # Instantaneous lateral jerk changes very rapidly, making it not useful on its own,
+    # however, we can "look ahead" to the future planned lateral jerk in order to gauge
+    # whether the current desired lateral jerk will persist into the future, i.e.
+    # whether it's "deliberate" or not. This allows us to simply ignore short-lived jerk.
+    # Note that LAT_PLAN_MIN_IDX is defined above and is used in order to prevent
+    # using a "future" value that is actually planned to occur before the "current" desired
+    # value, which is offset by the steerActuatorDelay.
+    self.friction_look_ahead_v = [1.4, 2.0]  # how many seconds in the future to look ahead in [0, ~2.1] in 0.1 increments
+    self.friction_look_ahead_bp = [9.0, 30.0]  # corresponding speeds in m/s in [0, ~40] in 1.0 increments
+
+    # Scaling the lateral acceleration "friction response" could be helpful for some.
+    # Increase for a stronger response, decrease for a weaker response.
+    self.lat_jerk_friction_factor = 0.4
+    self.lat_accel_friction_factor = 0.7  # in [0, 3], in 0.05 increments. 3 is arbitrary safety limit
+
+    # precompute time differences between ModelConstants.T_IDXS
+    self.t_diffs = np.diff(ModelConstants.T_IDXS)
+    self.desired_lat_jerk_time = CP.steerActuatorDelay + 0.3
+
+  def update_model_v2(self, model_v2):
+    self.model_v2 = model_v2
+    self.model_valid = self.model_v2 is not None and len(self.model_v2.orientation.x) >= CONTROL_N
+
+  def update_friction_input(self, val_1, val_2):
+    _error = val_1 - val_2
+    _value = self.lat_accel_friction_factor * _error + self.lat_jerk_friction_factor * self.lookahead_lateral_jerk
+
+    return _value
+
+  def update_calculations(self, CS, VM, desired_lateral_accel):
+    self.actual_lateral_jerk = 0.0
+    self.lateral_jerk_setpoint = 0.0
+    self.lateral_jerk_measurement = 0.0
+    self.lookahead_lateral_jerk = 0.0
+
+    if self.use_steering_angle:
+      actual_curvature_rate = -VM.calc_curvature(math.radians(CS.steeringRateDeg), CS.vEgo, 0.0)
+      self.actual_lateral_jerk = actual_curvature_rate * CS.vEgo ** 2
+
+    if self.model_valid:
+      # prepare "look-ahead" desired lateral jerk
+      lookahead = np.interp(CS.vEgo, self.friction_look_ahead_bp, self.friction_look_ahead_v)
+      friction_upper_idx = next((i for i, val in enumerate(ModelConstants.T_IDXS) if val > lookahead), 16)
+      predicted_lateral_jerk = get_predicted_lateral_jerk(self.model_v2.acceleration.y, self.t_diffs)
+      desired_lateral_jerk = (np.interp(self.desired_lat_jerk_time, ModelConstants.T_IDXS,
+                              self.model_v2.acceleration.y) - desired_lateral_accel) / self.desired_lat_jerk_time
+      self.lookahead_lateral_jerk = get_lookahead_value(predicted_lateral_jerk[LAT_PLAN_MIN_IDX:friction_upper_idx], desired_lateral_jerk)
+      if self.use_steering_angle or self.lookahead_lateral_jerk == 0.0:
path: sunnypilot/selfdrive/controls/lib/latcontrol_torque_ext_base.py
user: login: sunnyhaibin
body: d15a5ce300064251edc876ffb13a518a97df9b88
created_at: 2025-03-20T16:35:39Z
author_association: COLLABORATOR
diff_hunk: @@ -73,6 +77,11 @@ def update(self, active, CS, VM, params, steer_limited_by_controls, desired_curv
                                           desired_lateral_accel - actual_lateral_accel, lateral_accel_deadzone, friction_compensation=True,
                                           gravity_adjusted=True)
 
+      # Neural Network Lateral Control and custom stock lateral jerk updates
+      # Override stock ff and pid_log.error
+      ff, pid_log = self.extension.update(CS, VM, params, ff, pid_log, setpoint, measurement, calibrated_pose, roll_compensation,
path: selfdrive/controls/lib/latcontrol_torque.py
user: login: devtekve
body: what's ff? do we need it obscure? 
created_at: 2025-03-20T17:26:25Z
author_association: CONTRIBUTOR
diff_hunk: @@ -111,6 +116,7 @@ void DeveloperPanel::updateToggles(bool _offroad) {
     hyundaiRadarTracksToggle->setVisible(false);
   }
   experimentalLongitudinalToggle->refresh();
+  neuralNetworkLateralControl.refresh();
path: selfdrive/ui/qt/offroad/developer_panel.cc
user: login: devtekve
body: if not SP, this might blow up. Do we really need the if sunnypilot above? (likely the answer is yes, but still had to ask)
created_at: 2025-03-20T17:28:21Z
author_association: CONTRIBUTOR
diff_hunk: @@ -111,6 +116,7 @@ void DeveloperPanel::updateToggles(bool _offroad) {
     hyundaiRadarTracksToggle->setVisible(false);
   }
   experimentalLongitudinalToggle->refresh();
+  neuralNetworkLateralControl.refresh();
path: selfdrive/ui/qt/offroad/developer_panel.cc
user: login: devtekve
body: nvm, yeah, the only way it wouldn't be with that IF is if we make our own sunnypilot_developer panel and inherit and yada yada yada 
created_at: 2025-03-20T17:29:00Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,67 @@
+/**
+* Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
+ *
+ * This file is part of sunnypilot and is licensed under the MIT License.
+ * See the LICENSE.md file in the root directory for more details.
+ */
+
+#include "selfdrive/ui/sunnypilot/qt/offroad/settings/sunnypilot/neural_network_lateral_control.h"
+
+NeuralNetworkLateralcontrol::NeuralNetworkLateralcontrol(QWidget *parent) {
+    toggle = new ParamControl(
+      "NeuralNetworkLateralControl",
+      tr("Neural Network Lateral Control (NNLC)"),
+      nnff_description,
+      "");
+    toggle->setConfirmation(true, false);
+}
+
+void NeuralNetworkLateralcontrol::refresh() {
+    QString nnff_available_desc = tr("NNLC is currently not available on this platform.");
+    QString nnff_fuzzy_desc = tr("Match: \"Exact\" is ideal, but \"Fuzzy\" is fine too. Reach out to the sunnypilot team in the following channel at the sunnypilot Discord server if there are any issues: ")
+                                 + "<font color='white'><b>#tuning-nnlc</b></font>";
+    QString nnff_status_init = "<font color='yellow'>⚠️ " + tr("Start the car to check car compatibility") + "</font>";
+    QString nnff_not_loaded = "<font color='yellow'>⚠️ " + tr("NNLC Not Loaded") + "</font>";
+    QString nnff_loaded = "<font color=#00ff00>✅ " + tr("NNLC Loaded") + "</font>";
+    auto _car_model = QString::fromStdString(params.get("NNFFCarModel"));
+
+    auto cp_bytes = params.get("CarParamsPersistent");
+    auto cp_sp_bytes = params.get("CarParamsSPPersistent");
+    if (!cp_bytes.empty() && !cp_sp_bytes.empty()) {
+        AlignedBuffer aligned_buf;
+        AlignedBuffer aligned_buf_sp;
+        capnp::FlatArrayMessageReader cmsg(aligned_buf.align(cp_bytes.data(), cp_bytes.size()));
+        capnp::FlatArrayMessageReader cmsg_sp(aligned_buf_sp.align(cp_sp_bytes.data(), cp_sp_bytes.size()));
+        cereal::CarParams::Reader CP = cmsg.getRoot<cereal::CarParams>();
+        cereal::CarParamsSP::Reader CP_SP = cmsg_sp.getRoot<cereal::CarParamsSP>();
+
+        if (CP.getSteerControlType() == cereal::CarParams::SteerControlType::ANGLE) {
+            params.remove("EnforceTorqueLateral");
+
+            toggle->setDescription(nnffDescriptionBuilder(nnff_available_desc));
+            toggle->setEnabled(false);
+            params.remove("NNFF");
+        } else if (toggle->isToggled()) {
+            if (CP.getLateralTuning().which() == cereal::CarParams::LateralTuning::TORQUE) {
+                QString nn_model_name = QString::fromStdString(CP_SP.getNeuralNetworkLateralControl().getModelName());
+                QString nn_fuzzy = CP_SP.getNeuralNetworkLateralControl().getFuzzyFingerprint() ? tr("Fuzzy") : tr("Exact");
+
+                toggle->setDescription(nnffDescriptionBuilder(
+                    (nn_model_name == "")     ? nnff_status_init :
+                        (nn_model_name == "mock") ? (nnff_not_loaded + "<br>" + tr("Reach out to the sunnypilot team in the following channel at the sunnypilot Discord server and donate logs to get NNLC loaded for your car: ")
+                                                     + "<font color='white'><b>#tuning-nnlc</b></font>") :
+                            (nnff_loaded + " | " + tr("Match") + " = " + nn_fuzzy + " | " + _car_model + "<br><br>" + nnff_fuzzy_desc)));
+            } else {
+                toggle->setDescription(nnffDescriptionBuilder(nnff_status_init));
+            }
+        } else {
+            toggle->setDescription(nnff_description);
+        }
+    }
+}
+
+
+// ParamControlSP *NeuralNetworkLateralcontrol::configure_settings_interaction(ListWidgetSP *list) {
+//     list->addItem(neuralNetworkLateralControlToggle);
+//     return neuralNetworkLateralControlToggle;
+// }
path: selfdrive/ui/sunnypilot/qt/offroad/settings/sunnypilot/neural_network_lateral_control.cc
user: login: devtekve
body: this looked better :( it's abstraction. Also maybe add  line ending at the end of file
created_at: 2025-03-20T17:30:02Z
author_association: CONTRIBUTOR
diff_hunk: @@ -22,6 +25,25 @@ def log_fingerprint(CP: structs.CarParams) -> None:
     sentry.capture_fingerprint(CP.carFingerprint, CP.brand)
 
 
+def initialize_neural_network_lateral_control(CP: structs.CarParams, CP_SP: structs.CarParamsSP, params) -> None:
+  nnlc_model_path, nnlc_model_name, exact_match = get_nn_model_path(CP)
+
+  if nnlc_model_path is None:
+    cloudlog.error({"nnlc event": "car doesn't match any Neural Network model"})
+    nnlc_model_path = "MOCK"
+    nnlc_model_name = "MOCK"
+
+  if nnlc_model_path != "MOCK" and CP.steerControlType != structs.CarParams.SteerControlType.angle:
+    CP_SP.neuralNetworkLateralControl.enabled = params.get_bool("NeuralNetworkLateralControl")
+
+  if CP_SP.neuralNetworkLateralControl.enabled:
+    CarInterfaceBase.configure_torque_tune(CP.carFingerprint, CP.lateralTuning)
+
+  CP_SP.neuralNetworkLateralControl.modelPath = nnlc_model_path
+  CP_SP.neuralNetworkLateralControl.modelName = nnlc_model_name
+  CP_SP.neuralNetworkLateralControl.fuzzyFingerprint = not exact_match
path: sunnypilot/selfdrive/car/interfaces.py
user: login: devtekve
body: @sunnyhaibin if you change the cereal, remember this will be slightly diff
created_at: 2025-03-20T17:30:48Z
author_association: CONTRIBUTOR
diff_hunk: @@ -111,6 +116,7 @@ void DeveloperPanel::updateToggles(bool _offroad) {
     hyundaiRadarTracksToggle->setVisible(false);
   }
   experimentalLongitudinalToggle->refresh();
+  neuralNetworkLateralControl.refresh();
path: selfdrive/ui/qt/offroad/developer_panel.cc
user: login: sunnyhaibin
body: We'll have a dedicated panel for this. `Developer` is a temporary home for them :)
created_at: 2025-03-20T17:37:58Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,75 @@
+/**
+ * Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
+ *
+ * This file is part of sunnypilot and is licensed under the MIT License.
+ * See the LICENSE.md file in the root directory for more details.
+ */
+
+#include "selfdrive/ui/sunnypilot/qt/offroad/settings/lateral/neural_network_lateral_control.h"
+
+NeuralNetworkLateralControl::NeuralNetworkLateralControl() :
+  ParamControl("NeuralNetworkLateralControl", tr("Neural Network Lateral Control (NNLC)"),  "", "") {
+  setConfirmation(true, false);
+  updateToggle();
+}
+
+void NeuralNetworkLateralControl::showEvent(QShowEvent *event) {
+  updateToggle();
+}
+
+void NeuralNetworkLateralControl::updateToggle() {
+  QString nnff_available_desc = tr("NNLC is currently not available on this platform.");
+  QString nnff_fuzzy_desc =
+    tr("Match: \"Exact\" is ideal, but \"Fuzzy\" is fine too. Reach out to the sunnypilot team in the following channel at the sunnypilot Discord server if there are any issues:")
+    + " <font color='white'><b>#tuning-nnlc</b></font>";
+  QString nnff_status_init = "<font color='yellow'>" + tr("Start the car to check car compatibility") + "</font>";
+  QString nnff_not_loaded = "<font color='yellow'>" + tr("NNLC Not Loaded") + "</font>";
+  QString nnff_loaded = "<font color=#00ff00>" + tr("NNLC Loaded") + "</font>";
+
+  auto cp_bytes = params.get("CarParamsPersistent");
+  auto cp_sp_bytes = params.get("CarParamsSPPersistent");
+  if (!cp_bytes.empty() && !cp_sp_bytes.empty()) {
+    AlignedBuffer aligned_buf;
+    AlignedBuffer aligned_buf_sp;
+    capnp::FlatArrayMessageReader cmsg(aligned_buf.align(cp_bytes.data(), cp_bytes.size()));
+    capnp::FlatArrayMessageReader cmsg_sp(aligned_buf_sp.align(cp_sp_bytes.data(), cp_sp_bytes.size()));
+    cereal::CarParams::Reader CP = cmsg.getRoot<cereal::CarParams>();
+    cereal::CarParamsSP::Reader CP_SP = cmsg_sp.getRoot<cereal::CarParamsSP>();
+
+    /*** NNLC ***/
+    {
+      if (CP.getSteerControlType() == cereal::CarParams::SteerControlType::ANGLE) {
+        params.remove("NeuralNetworkLateralControl");
+
+        setDescription(nnffDescriptionBuilder(nnff_available_desc));
+        setEnabled(false);
+      } else if (isToggled()) {
+        if (CP.getLateralTuning().which() == cereal::CarParams::LateralTuning::TORQUE) {
+          QString nn_model_name = QString::fromStdString(CP_SP.getNeuralNetworkLateralControl().getModel().getName());
+          QString nn_fuzzy = CP_SP.getNeuralNetworkLateralControl().getFuzzyFingerprint() ? tr("Fuzzy") : tr("Exact");
+
+          setDescription(nnffDescriptionBuilder(
+            (nn_model_name == "")
+              ? nnff_status_init
+              : (nn_model_name == "MOCK")
+                  ? (nnff_not_loaded + "<br>" + tr(
+                       "Reach out to the sunnypilot team in the following channel at the sunnypilot Discord server and donate logs to get NNLC loaded for your car: ")
+                     + "<font color='white'><b>#tuning-nnlc</b></font>")
+                  : (nnff_loaded + " | " + tr("Match") + " = " + nn_fuzzy + " | " + nn_model_name + "<br><br>" +
+                     nnff_fuzzy_desc)
+          ));
+        } else {
+          setDescription(nnffDescriptionBuilder(nnff_status_init));
+        }
+      } else {
+        setDescription(nnff_description);
+      }
+    }
+  } else {
+    setDescription(isToggled() ? nnffDescriptionBuilder(nnff_status_init) : nnff_description);
+  }
+
+  if (getDescription() != nnff_description) {
+    showDescription();
+  }
+}
path: selfdrive/ui/sunnypilot/qt/offroad/settings/lateral/neural_network_lateral_control.cc
user: login: sunnyhaibin
body: This whole thing needs a refactor.
created_at: 2025-03-21T04:56:58Z
author_association: COLLABORATOR

PR comments:
user: login: github-actions[bot]
created_at: 2025-03-16T11:25:39Z
body: <!-- _(run_id_screenshots **13999595642**)_ -->
## UI Preview
<details open><summary>settings_sunnypilot : $${\color{red}\text{DIFFERENT}}$$</summary><table><tr>  <td> master <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/settings_sunnypilot_master_ref.png"> </td>  <td> proposed <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/settings_sunnypilot.png"> </td></tr><tr>  <td> diff <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/settings_sunnypilot_diff.png"> </td>  <td> composite diff <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/settings_sunnypilot_diff.gif"> </td></tr></table></details><details open><summary>settings_sunnypilot_mads : $${\color{red}\text{DIFFERENT}}$$</summary><table><tr>  <td> master <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/settings_sunnypilot_mads_master_ref.png"> </td>  <td> proposed <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/settings_sunnypilot_mads.png"> </td></tr><tr>  <td> diff <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/settings_sunnypilot_mads_diff.png"> </td>  <td> composite diff <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/settings_sunnypilot_mads_diff.gif"> </td></tr></table></details><details><summary>All Screenshots</summary><table><tr>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/body.png"> </td>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/driver_camera.png"> </td></tr><tr>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/homescreen.png"> </td>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/keyboard.png"> </td></tr><tr>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/keyboard_uppercase.png"> </td>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/offroad_alert.png"> </td></tr><tr>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/onroad.png"> </td>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/onroad_alert_full.png"> </td></tr><tr>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/onroad_alert_mid.png"> </td>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/onroad_alert_small.png"> </td></tr><tr>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/onroad_disengaged.png"> </td>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/onroad_override.png"> </td></tr><tr>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/onroad_sidebar.png"> </td>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/onroad_wide.png"> </td></tr><tr>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/onroad_wide_sidebar.png"> </td>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/prime.png"> </td></tr><tr>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/settings_developer.png"> </td>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/settings_device.png"> </td></tr><tr>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/settings_firehose.png"> </td>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/settings_network.png"> </td></tr><tr>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/settings_network_advanced.png"> </td>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/settings_software.png"> </td></tr><tr>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/settings_sunnylink.png"> </td>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/settings_sunnylink_sponsor_button.png"> </td></tr><tr>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/settings_sunnypilot.png"> </td>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/settings_sunnypilot_mads.png"> </td></tr><tr>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/settings_toggles.png"> </td>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/settings_trips.png"> </td></tr><tr>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/settings_vehicle.png"> </td>  <td> <img src="https://raw.githubusercontent.com/sunnypilot/ci-artifacts/openpilot/pr-667/update_available.png"> </td></tr></table></details>
<!-- thollander/actions-comment-pull-request "run_id_screenshots" -->
author_association: CONTRIBUTOR
user: login: sunnyhaibin
created_at: 2025-03-16T15:57:49Z
body: @twilsonco can you take a look and see if there's anything I missed? 😁
author_association: COLLABORATOR
user: login: sunnyhaibin
created_at: 2025-03-16T23:43:40Z
body: @sourcery-ai guide
author_association: COLLABORATOR
user: login: sourcery-ai[bot]
created_at: 2025-03-17T00:52:44Z
body: <!-- Generated by sourcery-ai[bot]: start review_guide -->

## Reviewer's Guide by Sourcery

This pull request implements Neural Network Lateral Control (NNLC) as an alternative to the existing lateral torque controller. It includes a toggle in the developer panel to enable/disable the feature. The implementation involves changes to car parameters, UI elements, control logic, and the addition of new files for the neural network model and related functionalities.

#### Updated class diagram for CarParamsSP

```mermaid
classDiagram
  class CarParamsSP {
    UInt32 flags
    Int16 safetyParam
    NeuralNetworkLateralControl neuralNetworkLateralControl
  }
  class NeuralNetworkLateralControl {
    Model model
    Bool fuzzyFingerprint
  }
  class Model {
    Text path
    Text name
  }
  CarParamsSP -- NeuralNetworkLateralControl : contains
  NeuralNetworkLateralControl -- Model : contains
```

### File-Level Changes

| Change | Details | Files |
| ------ | ------- | ----- |
| Introduces a new function to initialize and configure Neural Network Lateral Control (NNLC) based on car parameters and settings. | <ul><li>Added `initialize_neural_network_lateral_control` function to determine the NNLC model path and name.</li><li>Configures torque tune based on car fingerprint and lateral tuning if NNLC is enabled.</li><li>Sets the model path, name, and fuzzy fingerprint status in `CarParamsSP`.</li></ul> | `sunnypilot/selfdrive/car/interfaces.py` |
| Adds a toggle in the lateral panel to enable or disable Neural Network Lateral Control (NNLC). | <ul><li>Added `NeuralNetworkLateralControl` class.</li><li>Added `nnlcToggle` to the `LateralPanel`.</li><li>Connects the toggle to a function that shows/hides the description based on the toggle state.</li><li>Adds `nnlcToggle` to the list of offroad only toggles.</li></ul> | `selfdrive/ui/sunnypilot/qt/offroad/settings/lateral_panel.cc`<br/>`selfdrive/ui/sunnypilot/qt/offroad/settings/lateral_panel.h` |
| Adds a new struct to cereal to store Neural Network Lateral Control (NNLC) parameters. | <ul><li>Added `neuralNetworkLateralControl` field to `CarParamsSP`.</li><li>Defined nested structs `NeuralNetworkLateralControl` and `Model` to hold model path, name, and fuzzy fingerprint status.</li></ul> | `cereal/custom.capnp` |
| Refactors the lateral control to use a neural network-based controller. | <ul><li>Replaces the base class of `LatControlTorqueExt` with `NeuralNetworkLateralControl`.</li><li>Calls `update_neural_network_feedforward` in the `update` method.</li></ul> | `sunnypilot/selfdrive/controls/lib/latcontrol_torque_ext.py` |
| Adds a new parameter to the parameter store to control Neural Network Lateral Control (NNLC). | <ul><li>Added `NeuralNetworkLateralControl` to the list of keys in `params_keys.h`.</li></ul> | `common/params_keys.h` |
| Initializes the NeuralNetworkLateralControl param to 0. | <ul><li>Added `NeuralNetworkLateralControl` to the list of initial params in `manager.py`.</li></ul> | `system/manager/manager.py` |
| Adds source file for the Neural Network Lateral Control UI element. | <ul><li>Added `neural_network_lateral_control.cc` to the list of source files in `SConscript`.</li></ul> | `selfdrive/ui/sunnypilot/SConscript` |
| Introduces new files for the Neural Network Lateral Control implementation. | <ul><li>Added `nnlc.py` which contains the `NeuralNetworkLateralControl` class.</li><li>Added `model.py` which contains the `NNTorqueModel` class.</li><li>Added `helpers.py` which contains helper functions for NNLC.</li></ul> | `sunnypilot/selfdrive/controls/lib/nnlc/nnlc.py`<br/>`sunnypilot/selfdrive/controls/lib/nnlc/model.py`<br/>`sunnypilot/selfdrive/controls/lib/nnlc/helpers.py` |
| Adds a directory to store neural network data. | <ul><li>Added an empty directory `sunnypilot/neural_network_data`.</li></ul> | `sunnypilot/neural_network_data` |

---

<details>
<summary>Tips and commands</summary>

#### Interacting with Sourcery

- **Trigger a new review:** Comment `@sourcery-ai review` on the pull request.
- **Continue discussions:** Reply directly to Sourcery's review comments.
- **Generate a GitHub issue from a review comment:** Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with `@sourcery-ai issue` to create an issue from it.
- **Generate a pull request title:** Write `@sourcery-ai` anywhere in the pull
  request title to generate a title at any time. You can also comment
  `@sourcery-ai title` on the pull request to (re-)generate the title at any time.
- **Generate a pull request summary:** Write `@sourcery-ai summary` anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment `@sourcery-ai summary` on the pull request to
  (re-)generate the summary at any time.
- **Generate reviewer's guide:** Comment `@sourcery-ai guide` on the pull
  request to (re-)generate the reviewer's guide at any time.
- **Resolve all Sourcery comments:** Comment `@sourcery-ai resolve` on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
- **Dismiss all Sourcery reviews:** Comment `@sourcery-ai dismiss` on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  `@sourcery-ai review` to trigger a new review!
- **Generate a plan of action for an issue:** Comment `@sourcery-ai plan` on
  an issue to generate a plan of action for it.

#### Customizing Your Experience

Access your [dashboard](https://app.sourcery.ai) to:
- Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
- Change the review language.
- Add, remove or edit custom review instructions.
- Adjust other review settings.

#### Getting Help

- [Contact our support team](mailto:support@sourcery.ai) for questions or feedback.
- Visit our [documentation](https://docs.sourcery.ai) for detailed guides and information.
- Keep in touch with the Sourcery team by following us on [X/Twitter](https://x.com/SourceryAI), [LinkedIn](https://www.linkedin.com/company/sourcery-ai/) or [GitHub](https://github.com/sourcery-ai).

</details>

<!-- Generated by sourcery-ai[bot]: end review_guide -->
author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-03-17T03:10:37Z
body: ⚠️ Error during automated master-dev-c3-new squash:
```
None
```
author_association: CONTRIBUTOR
user: login: twilsonco
created_at: 2025-03-17T05:35:25Z
body: Looks good except for what I mentioned in the comments above. Great job refactoring. Keeps the stock code much cleaner. 
author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-03-17T15:21:15Z
body: ## Squash and Merge

⚠️ Error during automated `master-dev-c3-new` squash:
```
None
```
author_association: CONTRIBUTOR
user: login: sunnyhaibin
created_at: 2025-03-21T18:52:25Z
body: - [x] split Custom Lateral Acceleration
author_association: COLLABORATOR