Repository: kubernetes
Issue: Can't mount iscsi volume with openiscsi from target that does not automatically inform your initiator about changes in that session.
Labels: kind/bug,sig/storage,needs-triage

### What happened?

Can't mount iscsi volume in pod. 

I'm creating pod with iscsi volume from NAS using config:
```
---
apiVersion: v1
kind: Pod
metadata:
  name: iscsipd2
spec:
  containers:
  - name: iscsipd-ro
    image: nginx
    volumeMounts:
    - mountPath: "/mnt/iscsipd"
      name: iscsipd-rw
  volumes:
  - name: iscsipd-rw
    iscsi:
      targetPortal: <portalhost/port>
      iqn: <IQN>
      lun: 1
      fsType: ext4
      readOnly: false
```

Pod fails to start reporting:
```
  Warning  FailedMount             2s (x7 over 3m28s)  kubelet                  MountVolume.WaitForAttach failed for volume "iscsipd-rw" : failed to get any path for iscsi disk, last err seen:
Timed out waiting for device at path /dev/disk/by-path/<portal-IQN-Lun> after 30s
```

iSCSI server is Synology NAS which uses open iscsi 2.0-876.

It is possible to mount LUNs onto the host itself manually.

### What did you expect to happen?

Volume mounted.

### How can we reproduce it (as minimally and precisely as possible)?

Reproducing could be tricky if you don't have version of iscsi that shows similar behaviour.


When kubelet mounts iSCSI volume it executes the following command sequence
(starts here https://github.com/kubernetes/kubernetes/blob/642efbb595df18bcac54e1e53ab3d1d4df1569aa/pkg/volume/iscsi/iscsi_util.go#L339)

```
iscsiadm -m discoverydb -t sendtargets -p <portal> -I default -o new
iscsiadm -m discoverydb -t sendtargets -p <portal> -I default --discover
iscsiadm -m node -p <portal> -T <target> -I default -o update -n node.session.scan -v manual
iscsiadm -m node -p <portal> -T <target> -I default --login
iscsiadm -m node -p <portal> -T <target> -o update -n node.startup -v manual
```
After this it will wait for device path to appear once iscsid service on the node attaches to LUN. This unfortunately doesn't happen. The reason is `node.session.scan` being set to `manual` which was introduced in #90982 to address different issue. It is happening because my server doesn't notify initiator about new luns in session when `manual` option is set.

The solution is to rescan active iSCSI session to discover LUN which would trigger device file creation.
The test was to just execute
```
sudo iscsiadm -m node -p <...> -T <...> -I default -R
```
while `kubelet` was waiting for device file.
I then added
```
diff --git a/pkg/volume/iscsi/iscsi_util.go b/pkg/volume/iscsi/iscsi_util.go
index b51127aa92f..239cb075aef 100644
--- a/pkg/volume/iscsi/iscsi_util.go
+++ b/pkg/volume/iscsi/iscsi_util.go
@@ -376,6 +376,12 @@ func (util *ISCSIUtil) AttachDisk(b iscsiDiskMounter) (string, error) {
                                        klog.Warningf("Warning: Failed to set iSCSI login mode to manual. Error: %v", err)
                                }
 
+                               // For targets that don't notify initiator about LUNs added to session we need to trigger recans
+                               _, err = execWithLog(b, "iscsiadm", "-m", "node", "-p", tp, "-T", b.Iqn, "-I", b.Iface, "-R")
+                               if err != nil {
+                                       klog.Warningf("Warning: Failed to trigger rescan of LUNs in session. Error: %v", err)
+                               }
+
                                // Rebuild the host map after logging in
                                portalHostMap, err := b.deviceUtil.GetISCSIPortalHostMapForTarget(b.Iqn)
                                if err != nil {

```
to `kubelet` unconditionally after `--login` command to fix the issue. While it works, I'm sure there might be cases where this behaviour is undesirable, some internet people mentioned rescan being slow. Nevertheless it would be nice to have an option to do rescan either based on config or when mount can't be found within certain time threshold.

### Anything else we need to know?

_No response_

### Kubernetes version

<details>

```console
$ kubectl version
Client Version: v1.31.3
Kustomize Version: v5.4.2
Server Version: v1.32.0
```

</details>


### Cloud provider

<details>
Self hosted
</details>


### OS version

<details>

```console
# On Linux:
$ cat /etc/os-release
PRETTY_NAME="Ubuntu 24.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04.1 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo
$ uname -a
Linux xxx 6.8.0-51-generic #52-Ubuntu SMP PREEMPT_DYNAMIC Thu Dec  5 13:09:44 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
```

</details>


### Install tools

<details>

</details>


### Container runtime (CRI) and version (if applicable)

<details>

</details>


### Related plugins (CNI, CSI, ...) and versions (if applicable)

<details>

</details>