Complexity Theory 101: Problems Classification
By: Sara A. Metwalli
Tags: Technology, Women In Tech, Programming, Data Science, Science

As data scientists — or developers — we work daily to build and develop new solutions to the problems that face us. We create algorithms, write code, and test it for different instances of the problem we have at hand.

One important step in this process is defining the type of the problem and classify it. This step leads us to know whether the problem we are trying to solve is solvable or not before going further into the development process.

Complexity theory is a subfield of computer science that deals with classifying problems into a set of categories that specify the solvability of these problems. I am certain that along your journey as a data scientist, a programmer, or a computer science student/enthusiast, you have come across terms like, “this is an NP problem,” or “this problem is solvable in P time,” or “NP doesn’t equal P!”.

But, what exactly do these categories mean? And how can we understand what problems are solvable and what aren’t? If a problem is not solvable, what is the best that we can do to obtain some kind of answer to that problem?

Hopefully, this article will help you answer the above questions and deeper your understanding of the different complexity categorizations. To explain the different classifications, let’s stick to a very common problem, you and your friend want to go out for dinner. Now, you need to decide what to eat.

This problem can be easily translated into programming terms. Assume you’re working on a project and you’re asked to add new features to it. How easy will it be to add a specific feature? If you think about it, it’s fundamentally the same as deciding what to eat.

Easy Problems (P)

You’re sitting with your friend, and you’re both hungry. You suggest going out for pizza, but your friend isn’t feeling pizza today. So, they suggest to go for sushi, but you can’t handle raw fish. That goes on for a while, and you’re both getting hungrier.

Eventually, you say, “let's go with the most convenient option.” That is the closest one to where you are. You start sorting the top options based on their distance from your current location.

This problem can be described mathematically using an equation like Y(time to get to the restaurant) = 10 X( the distance to the resultant). Where 10 is the number of minutes you need to walk 1 km. This type of equation in math is called a polynomial.

Here where the classification Polynomial-time comes from. P problems are not often interesting, but there are problems that we come across daily such as sorting and searching algorithms.

Hard Problems (EXP)

Let’s take our problem up a notch. Now, it’s not just you and your friend; your parents and siblings are joining you for dinner as well. You can still follow the same approach you did when there were only two people and go to the nearest place, or you can think of better solutions.

You ask each one to say what they want to eat and try to convince others to choose the same thing. In the P solution, each person chose one place, and then the nearest one was the winner.

But, now, each will have to make more than one decision, each will have to choose a place and make a decision about the other’s suggestions.

This solution now got more complex. To describe it in math-terms, we would say Y(time needed to decide) = X² where X is the number of people we have. This now becomes an exponential (EXP) problem, and solving it will take longer than following the P approach.

Solvable Problems (R)

So far, despite how long it took us to decide what to eat, we eventually did decide and did eat. Both approaches to decide eventually led to a solution. That’s what solvable problems are. R problems are problems that can be solved in finite time. Whether the time is seconds, hours, or days, as long as it’s finite, then the problem is solvable.

You might be wondering now, is there a problem that would take an infinite time to solve?

The answer is yes—the halting problem. The problem statement goes, “Can we design a general algorithm that decides if a given program will halt or not if given a specific input?”. The term general makes this problem unsolvable because we can’t generalize when a program will halt or not unless we try and see if it does.

This problem is called undecidable and hence, unsolvable in infinite time. The halting problem is not the only undecidable problem in computer science; you can find more problems here.

Determenistic and nondeterministic

Image by the author (made using Canva)

When we write code to solve a specific problem, we often do so in a deterministic way. We write a series of loops, conditional statements to get us to where we want to go. Another approach to solve problems is doing so in a nondeterministic way.

In a nondeterministic way, we solve problems by asking questions and/ or making guesses until we reach the desired answer.

Going back to our eating dilemma, let’s say, instead of sorting options based on their relative distance to us, or having each person debate and justify their choice, you decide to write the options down and let your cat decide which the winner. So, you set the papers in front of the cat and wait for it to place its paw on one of them.

Following this approach, you managed to solve the problem nondeteminstacally. This type of solution takes P type problems and solves them faster with a magical nondeterministic approach. These types of problems are classified as NP.

Does P = NP?

Finding a nondeterministic solution to a problem is often a shot of luck. You were lucky that you had a cat to choose what to eat easier for you, but not all people have cats.

In terms of computer science, people have been divide on whether P = NP or not. Some say it’s a matter of time until we develop a nondeterministic way to solve all P problems. While others think some problems can’t be solved nondetemnistacally. Who is correct? No one knows; only time will prove that.

NP problems

Perhaps the most interesting problems we have in computer science are NP problems. In 1970, a mathematician called Richard Karp proved that we could reduce all SAT (Satisfiability) problems to NP and made a list of 21 NP-complete problems.

If you have gone through this ist before, you might have noticed that although these problems are unsolvable, you can still find solutions to some of them in textbooks. An example is the traveling salesman problem. Although that problem is defined as NP-hard, some approximate solutions solve it in some special cases.

Image by the author (made using Canva)

NP-hard vs. NP-complete

Any NP problem that can be solved in P time nondeterministically is called an NP-complete problem as long as it is a decision problem. NP-hard is the next level. NP-hard problems can be reduced to NP-complete in P time and are not decision problems such as the halting problem.

Conclusion

Complexity theory is one of the important subfields of computer science that deals with categorizing problems based on the time they need to be solved. One important thing to mention is, complexity theory doesn’t care much about the algorithm you use to solve a specific problem, rather the problem itself.

We face NP-hard and NP-complete problems more often than we might initially think. Having the required knowledge to categorize and understand the type of these problems can help us come up with better answers or reduce the problems to P type problems to match our needs.

Having a deeper understanding of complexity theory and the different problem categorization will definitely benefit your work and someday maybe save it.