How To Recover Access To EC2 Instance After Losing Pem File(SSH Keys) 2022
Tags: aws, devops, cloud, webdev
Published: 2022-07-08 18:00:42

Losing credentials or log-in details to any platform or Infrastructure can be frustrating and can have a lot of consequences which is why we must protect our security credentials with all intentions. 

Imagine losing credentials to your vault and you can't access your money 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5r3rfgl9zkcp7pxiga9q.jpg)

Never lost my ssh keys tho. So this article is a request from someone and I'm writing just to show a walk-through of how to recover missing Pem file to an AWS instance. let's go

## RECOVERING ACCESS TO MY EC2 INSTANCE AFTER LOSING PEM FILE

For Pem file recovery to be possible, the lunched instance with the missing Pem file must be an EBS-Backed Instance because it is not possible on an instance backed by an instant store.

Assume that your instance with lost Pem file is "Lost pem server"

I. To confirm the backed-storage type

- Click/Check the instance.

- Select the “storage” tab.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/87722r17tfjf1fd4l7mw.png)

II. Launch a Recovery Instance

- Launch the “Recovery Instance” in the same AZ as the “Lost pem server”

- To Launch a New Instance in the same Availability Zone as the instance(Lost pem server) with the missing Pem file….Make sure to check which AZ the instance is running(The above is running in us-east-2a).

- Select the same network(VPC) and subnet as the old instance(Lost pem server).

III. Create key pair for the new instance(Recovery server) and connect to it using ssh. Be sure you can connect to the instance.

IV. Now stop the old instance(Lost pem server). 

V. Detach the EBS volume connected to the “Lost pem server”.

- Click on the instance with lost Pem and select the “Storage” Tab.

- Scroll down to blocked devices and click on the ID for the Root Device. In my case, the root device is **dev/sda1**.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/se6vbf6pr2d4uai0nyhn.png)

- Make sure the Root Storage is still selected, then click on Action

- Finally, select “Detach volume”

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0r6rmc6fgtxlin6h43im.png)

- Go to “volumes” under EBS(Elastic Block Store) and select the volume you previously detached from the “Lost pem server”


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/435afhvm1eiatf5p0glb.png)

- Select the volume and click “Action” 
- Select “Attach Volume”.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kcb91lwxcft7mgjbp0x7.png)
- Select the recovery server(Recovery Server).

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/je0wpzubcz3yin3i2vlt.png)

VI. Go back to the shh terminal to the “Recovery server” and check the EBS volume 
**NOTE:** The attached EBS volume won’t mount automatically, so you have to mount it.

- Go to your terminal and ssh into the Recovery server. In my case, I’m using **WSL** in **visual studio**.

## Enter the following commands

I. Optionally: you might want to copy the pem file from where it is to the .ssh folder

```
cp /mnt/c/Users/YourName/desktop/pem/recovery.pem ~/.ssh/recovery.pem
```

II. Connect to the instance using SSH

III. To check the list of storage on the device to see if the attached EBS was mounted or not. Though it doesn’t mount automatically.

```
lsblk
```

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/am01o3miupb3etf44ftf.png)

NOTE: The attached EBS(XVDF) has not been mounted to any path as you can see in the screenshot above.

IV. Also, confirm that the attached EBS is not empty because u can’t mount an empty EBS.

```
sudo file -s /dev/xvdf
```

Output: /dev/xvdf: DOS/MBR boot sector, extended partition table (last)

Output like this means the drive is not empty, so you can go ahead and mount.  

V. Create a temporary directory to mount the EBS volume(for the Lost pem server) in the Recovery server. 

```
sudo mkdir /mnt/tempvolume 
```
The path = /mnt/tempvolume

VI. Mount the storage on path /mnt/tempvolume. Mount the drive with a number at the end **“/dev/xvdf1”**

**/dev/xvdf1** is the disk to be mounted.

```
sudo mount /dev/xvdf1 /mnt/tempvolume 
```

VII. Check if the EBS has mounted

```
lsblk  
```

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8lqdl3btye5zwseqw0jb.png)

To check the list of storage on the device again.

VIII. Copy the SSH key of your “Recovery server” into the attached drive belonging to the “Lost pem server”
	
```
cp .ssh/authorized_keys /mnt/tempvolume/home/ubuntu/.ssh/
```

NOTE: Ubuntu in path /mnt/tempvolume/home/ubuntu/.ssh/ is the server name, it can be ec2-user if u launched linux server.

IX. Check the list of contents in the temporary location on the storage you copied the key to.

```
ls -lah /mnt/tempvolume/home/ubuntu/.ssh/ 
```

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/midqsz9xbemjkf8qkdgk.png)

The key “Authozied_keys” is now in the folder you copied it to.

X. Unmount the attached storage from the “Recovery server so you can attach it back to the “Lost pem server”.

```
sudo umount /mnt/tempvolume/
```

XI. Check if the disk has unmounted.

```
lsblk
```

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s3c1dchgx1kz0fezcd58.png)

The mount point of /dev/xvdf1 is no longer  /mnt/tempvolume

XII. Now detach the attached “Lost pem EBS” from the “Recovery server” to attach it back to its original server “Lost pem server”.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ei665dwjmsv4cpn4vmx3.png)

XIII. Attached the EBS to the  “Lost pem server” and edit the name to **/dev/sda1** and save.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4sps7gbj221yanuicuxv.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/serhxvuvqso4cmdnwoha.png)

To confirm the time the EBS was attached, check “Attachment time”

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/g4l075raz2ejudr9nige.png)
XIV. Connect to the “Lost pem server” with the key of the “Recovery Server”.

```
ssh -i "ggfgvfv.pem" ubuntu@ec2-3-145-xxx-92.us-east-2.compute.amazonaws.com
```

XV. Connect to the “Lost pem server” with the key of the “Recovery Server”.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3idu4kgkxi1rb44nruuj.png)

NOTE: Don’t forget to terminate the instance if it’s not a free tier instance to avoid extra cost.

Thank you and next time don't misplace your key 😄