Tutorial: Creating Dataset The Elder Scroll: Skyrim Armor and Sending to Kaggle Datasets
Tags: python, jupyter, kaggle, pandas
Published: 2024-01-13 18:37:17


One of the games that made it into my top 10 was **The Elder Scroll Skyrim**, making it nostalgic to be able to do a project related to the game. These days I was thinking about doing a project and I ended up having an idea. But I needed to have a set of data, I searched the internet and found some, but I lacked armor.

I searched [The Elder Scrolls Wiki](https://elderscrolls.fandom.com/wiki/The_Elder_Scrolls_Wiki]) for this table of [armors](https://elderscrolls.fandom.com/wiki/Armor_(Skyrim)) . I need to carry out the information in the table. In this simple tutorial, you will extract the table, perform pre-processing and finally add it to Kaggle Datasets.


## Extract Dataset Tabular

To perform this step you will need the [Pandas](https://pandas.pydata.org/pandas-docs/stable/index.html) library. Pandas has a huge set of features to perform processing, extraction, processing and much more. On the Wiki website there are several tables with armor data, for this I need to use [pandas.read_html](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_html.html ).

With **pd.read_html** I can:

- Read an HTML, where I pass the link as a string to the function.
- Use Regular expression
- Extract Links

That and some options that I'm going through, I recommend looking in the documentation to find out others.

Finally, it returns to me all the tables it found, already being Dataframes.

```python
# Import Library Pandas
import pandas as pd

# Read HTML
tables_on_page = pd.read_html("https://elderscrolls.fandom.com/wiki/Armor_(Skyrim)")

# Amount Tables
len(tables_on_page) # output: 16

```


Each table is a type of armor, so I separated them by the names I found on the website. I won't include all types so as not to accumulate something repetitive in the post. But at the end I will leave the project link so you can check it out and test it too.

```python
headgear_light_armor = tables_on_page[4]
headgear_light_armor
```

![Create By Author - Notebook Skyrim Dataset](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/knjx0nxppzfhp0qwynaz.png)


## Preprocessing

I did this step individually for each table to add a new column. Where I needed to put the type of armor as a class. At the end I merge the tables, to get just one. As there was no difference between the columns, it was only necessary to use [pd.concat](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.concat.html). Then I renamed some columns, due to some functionality of the link in these columns it was not possible to return the text. Then I dropped the ID column because I didn't think it was necessary for what I'm creating. Finally I save the file in [.csv ](https://www.lifewire.com/csv-file-2622708).

```python
# New Column
headgear_light_armor['type_armor'] = 'Headgear Light Armor'
headgear_light_armor

# Unfiy Datasets
df = pd.concat([gauntlets_heavy_armor, gauntlets_light_armor, boots_heavy_armor, boots_light_armor, cuirasses_heavy_armor, cuirasses_light_armor, headgear_heavy_armor, headgear_light_armor], axis=0)

# Rename Columns
df.rename(columns={'Unnamed: 1':'Armor','Unnamed: 2':'Encumbrance','Unnamed: 3':'Gold'}, inplace=True)

# Drop ID
df.drop(['Item ID'], axis=1, inplace=True)

# Save File
df.to_csv('dataset_armor_skyrim_1.csv', index=False)
```

![Create By Author](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b40iwavch21ov4jzn2ic.png)


## [Kaggle Datasets](https://www.kaggle.com)

When I was developing this notebook I didn't think about adding it to Kaggle. I wanted to use the result of the set for my project. From this point I thought why not share, I can create a post explaining the process and finally post the dataset for other users to use on their notebooks. So make a contribution. And here we are, let's go to post this dataset.

If you don't know much about Kaggle, there's a post of mine where I talk a little about this incredible tool.

{% embed https://dev.to/sc0v0ne/no-excuses-to-start-working-with-python-cli %}

After you have created your account on Kaggle, you will be on the dashboard. Search for **Datasets**.


![Create By Authot](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/j9f1vp7qycvmuabhb759.png)


Then click on **New Dataset**.


![Create By author- Kaggle Add new Dataset](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x7b02q5berfnc9xf0128.png)

Now you will get the created dataset, which was created in the notebook. You can drag or open the window to add the file.


![Create By Author - Add dataset Kaggle](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8w95hwi53bgygcxcisst.png)

Now you need to add a title to your dataset, you can add more files if you need. Choose between the public or private options, finally create the dataset. At this stage it will load your file and then direct you to the dataset page.


![Create by autho - Add file Kaggle Datasets](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6aw7mqyagvttb8ntf236.png)

Finally, your dataset will be available on Kaggle for you to use in your projects or, if the dataset is public, other users can use it.

**[The Elder Scrolls Skyrim - Armor](https://www.kaggle.com/datasets/sc0v1n0/the-elder-scroll-skyrim-armor)**
![Create By Author - Dataset Kaggle example Skyrim](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5bakt4vgi8wjohv505z0.gif)


----

## Repository

You can find the project via the link below.

{% embed https://gitlab.com/public-dev-projects-1/dataset-armor-the-elder-scroll-skyrim %}

-----

## About the author:

{% embed https://dev.to/sc0v0ne %}

**A little more about me...**

Graduated in Bachelor of Information Systems, in college I had contact with different technologies. Along the way, I took the Artificial Intelligence course, where I had my first contact with machine learning and Python. From this it became my passion to learn about this area. Today I work with machine learning and deep learning developing communication software. Along the way, I created a blog where I create some posts about subjects that I am studying and share them to help other users.

I'm currently learning TensorFlow and Computer Vision

Curiosity: I love coffee

- [Kaggle](https://www.kaggle.com/sc0v1n0)
- [Gitlab](https://gitlab.com/sc0v0n3)
- [Github](https://github.com/sc0v0ne)
- [Mastodon](https://mastodon.social/@sc0v0ne)

--------

## My Latest Posts

{% embed https://dev.to/sc0v0ne/my-super-powers-as-a-software-developer-2024-1h17 %}

{% embed https://dev.to/sc0v0ne/docker-communicate-between-containers-5ge5 %}

{% embed https://dev.to/sc0v0ne/blueflix-idea-build-deploy-ji5 %}

-------

### Resources

- [Image Dragonborn](https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2Fstaticdelivery.nexusmods.com%2Fmods%2F110%2Fimages%2F83754-0-1494296578.png&f=1&nofb=1&ipt=c26bf9c8016074776e013cd088c81053e68e1af915c1f0b806f849b178fffa10&ipo=images)
- [Image Dragon](https://external-content.duckduckgo.com/iu/?u=http%3A%2F%2Fwallpapercave.com%2Fwp%2FZA7iKmq.jpg&f=1&nofb=1&ipt=7cab68be4a0a876512220c01c8dce32381a6ac6c2225dc90769c61cd3328ea00&ipo=images)