Python Lists vs Sets. When to use lists and when to use sets? | by Jacob Toftgaard Rasmussen | Towards Data Science

You will learn about the differences between lists and sets in this article. You will also see a practical example where a set performs far better than a list does.
This post has the following structure:
Introduction
Why lists are cool and also uncool
Why sets are uncool and also cool
A practical example showing differences
Summary and goodbyes
The python list is a pretty cool data structure that has a lot of functionality. You might already be using lists, but do you know what makes python lists special? Compared to other programming languages Python lists allow give you a lot of freedom. You can put all different kinds of data types into a python list.  Take a look at this totally valid (but also very weird) python list:
valid_list = [    77,    4.2,    'a string',    ('a tuple', 'inside a list'),    {'secret_password': 123},    ['another list', 'why not?']]
No problemo.
This list contains an integer, a float, a string, a tuple, a dictionary and even another list. When compared to other languages this almost seems like magic. This can be a benefit as you have extreme flexibility when it comes to what you want to include in your lists, yet it can also become a trap if you are not careful. Other languages like C# for example require you to specify what data type your list will consist of. Because C# is more strict it can save you from making mistakes. An example of this could be if you try to do some arithmetic change to every item in a list. Lets say that you want to multiply all items by 2. That works fine as long as the items in the list are all number types. But what if you somewhere somehow added a dictionary to the list. You probably would not notice that it had happened because python will simply smile and allow it. But then when you try do multiplication with a dictionary, python throws up:
a_list_of_numbers = [1, 77, 4.2]# somehow a dictionary sneaks in to ruin the funa_list_of_numbers.append({'rude': 'intruder'})for number in a_list_of_numbers:    print(number * 2)#->21548.4Traceback (most recent call last):  File "c:\secret\directory\your_file.py", line 18, in <module>    print(number * 2)TypeError: unsupported operand type(s) for *: 'dict' and 'int'
Mucho problemo!
Another downside of python’s list’s flexibility is the computation time and memory that they use. For most programs or scripts that you write this will not be a problem, and you do not have to worry too much about it. However, if you are writing the software for a tiny device that has limited computational power and storage capacity, or if you are working with very large amounts of data, then you should start considering if python lists are the best solution. You will see a practical example of this further down.
Alright, now we have covered that python lists are cool since they allow you to do almost anything, but remember that this flexibility comes at a price. Next lets have a look at sets.
Sets work similarly to lists, in the fact that they can store multiple elements. Apart from that though, they are almost completely different. Sets only allow you to store unique values. In other words, a set cannot store duplicates of the same value and it will simply remove any duplicates:
valid_list = ['Apple', 'Banana', 'Apple', 'Orange']# Only one 'Apple' can stay in the setnot_valid_set = {'Apple', 'Banana', 'Apple', 'Orange'}valid_set = {'Apple', 'Banana', 'Orange'}
You probably also noticed that the syntax for creating a set is different than for lists. A set is initiated with curly brackets instead of square brackets.
You can easily convert from a list to a set like this:
fruit_list = ['Apple', 'Banana', 'Apple', 'Orange']fruit_set = set(fruit_list)print(fruit_set)#-> {'Orange', 'Banana', 'Apple'}
Great! Now you know how to convert from a list to a set. As you can see in the print output we have lost an ‘Apple’ entry when converting to a list. You therefore have to be careful, as you cannot restore the lost ‘Apple’ by converting back to a list.
A small trick is that you can use conversion to a set if you have a list where you want to remove duplicates. This is quicker than writing a function or loop that looks for duplicates and removes them. However, you can only do this if the order of the elements does not matter. Because...
Did you notice that the order of the fruits was also changed?If not, then take a look again, or copy the code into your own editor and try it out. You might have to run the script a few times, because the order is not necessarily the same each time.
Why does this happen? It is because python sets store their values via hashing.Hashing is a topic for another day. For now you just need to know that when you use a set you cannot be sure what order the elements appear in. This might seem annoying, yet it also makes look ups in sets very efficient. You will see this in the upcoming practical example.
First I would like to give credit to the book Impractical Python Projects written by Lee Vaughan and published by no starch press, as this book gave me the inspiration to write this example and post.
The book has a chapter where you learn to find palindromes and palingrams via python code. Palindromes are words that are spelled the same forwards and backwards and palingrams are whole phrases spelled the same forwards and backwards. For this example I will try to keep it short and not go into details about finding palindromes, as the point is just to demonstrate how powerful sets can be.
Imagine that you have a list of words from the English dictionary, you can get one here. Lets say you want to look through all the words and for each word you want to know if the first 3 letters form another word from the dictionary, (it is a silly example but I hope you can work with that. Maybe you can come up with something more relevant?). The code could look like this:
def find_words_list():         with open('dictionary.txt') as f:        word_list = f.read().strip().split('\n')        word_list = [x.lower() for x in word_list]    result_list = []    for word in word_list:        if len(word) > 3:            if word[:3] in word_list:                result_list.append(word)    print(len(result_list))
If I run the above function on my laptop it takes about 47 seconds. That is actually a long time if you think about it in computer time. The collection of words I have used (and linked to) has 45,333 words, but there are other collections available with more than 100,000 words, it would take much longer to look through all those words. Using sets we can reduce this computation time by quite a lot:
def find_words_set():         with open('dictionary.txt') as f:        word_list = f.read().strip().split('\n')        word_list = [x.lower() for x in word_list]        word_set = set(word_list)       result_list = []    for word in word_set:        if len(word) > 3:            if word[:3] in word_set:                result_list.append(word)    print(len(result_list))
This code where we have converted the list of words into a set takes roughly 0.08 seconds to run on my machine.
Wow! That is quite a difference.
As I mentioned previously, the inspiration for this example came from the excellent book Impractical Python Projects. When I discovered this difference between lists and sets it was quite an Aha moment for me. Which is why I really wanted to share it with you!
You have now learned some critical differences between lists and sets. These are the key takeaways:
Lists can contain all sorts of data types (be careful with this)
Lists can contain duplicates
Lists maintain the order of elements
Lists use a lot of computational time and takes up a lot of space
Sets can contain all sorts of data types (be careful with this)
Sets cannot contain duplicates, and they will simply disappear.
Sets cannot maintain the order of elements
Sets use hashing to perform look ups which makes them way faster than lists in this regard. (In the practical example the code using lists took about 45 seconds to run, whereas the code with sets took less than a tenth of a second!)
Thank you for taking the time to read this post. I hope that you learned something useful!
Keep learning! — Jacob Toftgaard Rasmussen