EBS Volume stuck in "Attaching"
Tags: aws, cloud, devops, devjournal
Published: 2021-12-06 00:29:17

Came across this error when I was doing labs for storage management in Linux. I was having issues on not being able to format the disks, so I finally decided,

```bash
"Ah, f*ck it. Imma just delete this volume and slap a new, 
fresh one in its place."
```
---

### Detaching and Force Detaching

So there I went. Entered the AWS console and then to the EBS menu, selected my volume and deleted it. After detaching the volume, it still showed **"In-use"** volume state. Refreshed the menu a couple of times and it still showed the same. If the detachment was successful, it should have instead show **"Available"**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/eo6o18ql29wejyvjspn8.png)

Another f*ck-it moment.
```bash
"Ah, f*ck it. Imma force delete this sh*t."
```

Selected the volume, hit **Actions**, and chose **Force detach volume**.
Note that it will prompt you to confirm by typing in *detach* on the prompt.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/i70s5eklfoi70a3dbaoi.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ufciuyufdajjqgywxwak.png)

Afterwards, I deleted the volume so I can create a new one with the same device name.

---

### Creating and Attaching the new EBS volume

Created a new EBS volume and gave it the same **Name**. Attached it to the same EC2 instance and reused the same device name **/dev/sdc**
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cjkdsfno39twr2nbbwv0.png)
 

---

### EBS Menu vs. EC2 Instance Menu

From the EBS volume menu, the new volume is immediately transitioned to **"In-use"** state.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x9we5a54hiireghvtkpv.png)

However, it still showing as **Attaching** from the EC2 instance menu. Refreshed the menu a couple of times and it still shows the same.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pw8owqm4gxhphcz67do2.png)
 

From the terminal, I also can't see the new */dev/xvdc*.
```bash
eden@tst-rhel:~ $ sudo fdisk /dev/xvdc -l
fdisk: cannot open /dev/xvdc: Input/output error
```
---

### Google Time
Went online and immediately found this helpful article from AWS themselves. It carefully explained what may have caused the issue. Breezed through it and found the culprit:

> **The block device driver didn't release the device name**
> If a user has initiated a forced detach of an Amazon EBS volume, the block device driver of the Amazon EC2 instance might not immediately release the device name for reuse. Attempting to use that device name when attaching a volume causes the volume to be stuck in the attaching state. You must either choose a different device name or reboot the instance.

#### Reboot
So I went and rebooted the instance. After it went back, i checked again the **Storage** tab in the **Instance Summary** panel. But now it showed a new error at the bottom. It also still shows the specific volume stuck in "attaching" state

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h9x0f0s3rx7yj8jgvl0t.png) 

#### Restart
Going back to the AWS articles, it suggested to do a restart.
> If these steps don’t resolve the issue, or if you must use the device name that isn't working, try the following procedures:
> - Reboot the instance.
> - Stop and start the instance to migrate it to new underlying hardware. Keep in mind that instance store data is lost when you stop and start an instance. If your instance is instance store-backed or has instance store volumes containing data, the data is lost when you stop the instance.

#### Wait, hold up!
After re-reading the AWS article, I went back to the AWS Console to restart the instance. And what do you know. The new error is gone, the volume has also changed to **"Attached"** state.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1vi6c8gndgcvvzyn1r5o.png)

Also can see it from the terminal
```bash
eden@tst-rhel:~ $ sudo fdisk /dev/xvdc -l
Disk /dev/xvdc: 100 GiB, 107374182400 bytes, 209715200 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
```

That's it.

---

### References

- [Why is my Amazon EBS volume stuck in the "attaching" state?](https://aws.amazon.com/premiumsupport/knowledge-center/ebs-stuck-attaching/)

---