# Greedy Algorithms

## Summary


A greedy algorithm is a type of algorithm that follows the problem-solving heuristic of making the locally optimal choice at each stage with the hope of finding a global optimum. While it may not find the global optimum, greedy algorithms are often simpler and faster while being not too far from the global optimum.

Greedy algorithms are being used in many areas of computer science. They're also used in operations research, engineering, finance, machine learning, bioinformatics, game theory, etc. They've become a well-established technique for solving optimization problems.

The key to developing an effective greedy algorithm is to identify the problem's specific structure and design a heuristic that makes the optimal local choice at each step. While they do have some limitations, there's ongoing research to address these limitations.

## Discussion

### Could you explain greedy algorithms with a simple example?

Suppose you are given an array of positive integers and you want to find the subset of those integers whose sum is as large as possible, but does not exceed a given value. For instance, if you were given the array `[3, 4, 5, 6, 8]` and the limit 13, the optimal subset would be `[8, 5]`, with a sum of 13.

A greedy algorithm for this problem would be to sort the array in decreasing order, then start selecting the largest numbers that don't exceed the limit, until the limit is reached. So for the example above, the steps of the algorithm would be:

    + Sort the array in decreasing order: `[8, 6, 5, 4, 3]`
    + Initialize an empty subset and a variable to track the current sum: subset = `[]`, sum = 0
    + Start the iterations: (i) subset = `[8]`, sum = 8; (ii) subset = `[8, 5]`, sum = 13.
    + Return the subset: `[8, 5]`

### Why should I use greedy algorithms when the solution is not guaranteed to be optimal?

While greedy algorithms don't guarantee an optimal solution, they have their benefits:

    + **Simplicity**: Greedy algorithms are often simple to implement and understand, making them a good choice for problems with large inputs or when efficiency is a concern.
    + **Speed**: Greedy algorithms are often very fast, especially when compared to more complex algorithms. This makes them a good choice for problems with large inputs.
    + **Approximation**: Even though the greedy algorithm does not always guarantee the optimal solution, it can often give a very good approximation of the optimal solution. In many cases, the difference between the optimal solution and the solution found by the greedy algorithm is not significant.
    + **Starting Point**: Greedy algorithms can be a good starting point for more complex algorithms. By using a greedy algorithm to quickly find a good solution, you can then refine the solution using other techniques.

### What are some applications of greedy algorithms?

Here are some examples of greedy algorithms:

    + **Minimum Spanning Tree (MST)**: MST is useful in network design and transportation planning. Kruskal's and Prim's algorithms are greedy approaches to this problem.
    + **Huffman Coding**: This is applied in data compression and transmission. It assigns shorter codes to more frequently occurring characters.
    + **Knapsack Problem**: This is considered a classic optimization problem. It asks what's the best way to fill a bag of fixed capacity with items of different sizes and values. A greedy algorithm selects items based on their value-to-size ratio.
    + **Activity Selection**: In scheduling problems, there's a need to select the maximum number of non-overlapping activities. A simple greedy algorithm solves this by selecting activities based on their finish time.
    + **Shortest Path Algorithms**: Dijkstra's algorithm is an example. It selects the shortest path from a given vertex to all other vertices in a graph.
    + **Coin Changing Problem**: This asks what's the minimum number of coins needed to make change for a given amount of money. This is solved by selecting the largest coin possible at each step.

### How are greedy algorithms different from dynamic programming?

Greedy algorithms and dynamic programming are two popular techniques for solving optimization problems, but they differ in several key ways.

Greedy algorithms make the **locally optimal choice** at each step in the hope of finding a global optimal solution. Dynamic programming breaks down a problem into smaller subproblems and then solves them in a bottom-up fashion. It stores the solutions to the subproblems and reuses them to solve the larger problem.

Greedy algorithms may give a **sub-optimal solution**, whereas dynamic programming always finds the optimal solution. Greedy algorithms typically make choices based only on the current state of the problem, while dynamic programming considers all possible subproblems and their solutions.

Greedy algorithms typically require **less memory** because they don't need to store the solutions to all possible subproblems.

Greedy algorithms are typically **faster** due to fewer calculations. However, the time complexity of greedy algorithms can be higher in certain cases.

Greedy algorithms are useful when there is a **unique optimal solution**. Dynamic programming can solve a wider range of problems, including problems with multiple optimal solutions or overlapping subproblems.


### What are some practical limitations of greedy algorithms?

Because greedy algorithms make the locally optimal choice at each step, this may not lead to the global optimal solution. In some cases, making a suboptimal choice at an early stage can lead to a better global solution. The figure shows an example in which the objective is to maximize the sum of the nodes on a top-to-bottom path. Greedy algorithm leads to a sub-optimal solution. 

Where the objective function has multiple conflicting objectives, or changes non-monotonically over time, greedy algorithms will not work well. The same can be said of problems with complex constraints or a large search space. For these cases, greedy algorithms would incur a larger time complexity as well.

Many optimization problems are NP-hard, which means that finding the optimal solution is computationally intractable. Greedy algorithms are not suitable for solving them, as they can't guarantee the optimal solution in a reasonable amount of time.


### What are some recent advances with greedy algorithms?

**Adaptive greedy algorithms** adjust their choices dynamically based on the problem's structure and input data. They have outperformed traditional greedy algorithms in many real-world optimization problems.

**Hybrid algorithms** combine greedy techniques with other optimization techniques, such as dynamic programming or local search. These hybrid algorithms can often provide better results than either technique used alone.

There are greedy algorithms capable of **multi-objective optimization**. They can be used to find a Pareto-optimal set of solutions.

In many real-world applications, input data is received incrementally over time. **Online optimization algorithms** must make decisions in real-time, without having access to all the input data in advance. Greedy algorithms have been shown to be effective in this setting because they can make quick decisions based on the available data.

Researchers have also been working on developing new methods for analysing the performance of greedy algorithms. There are new theoretical frameworks for understanding the behaviour of greedy algorithms in different types of optimization problems.

## Milestones

500  
AD

Indian mathematician Aryabhata uses a greedy algorithm to solve the problem of finding the largest number that can be written as the sum of two squares.

1930

The modern concept of greedy algorithms was first introduced in the 1930s by mathematicians and computer scientists working on the design of algorithms for optimization problems.

1952

The term **greedy algorithm** is coined by mathematician and computer scientist David A. Huffman in reference to the Huffman coding algorithm. The term "greedy" refers to the algorithm's strategy of always choosing the option that seems best at the time, without considering the possible consequences of that choice on future steps.

1956

Computer scientist Joseph Kruskal develops a greedy algorithm for finding the **minimum spanning tree** of a graph. In 1957, Robert Prim develop's his own greedy algorithm for the solving the same problem. Also in 1956, Edsger Dijkstra develops a greedy algorithm for finding the **shortest path** in a graph. The same year L. R. Ford and D. R. Fulkerson develop a greedy algorithm for solving the **maximum flow problem** in a network.

1980

In the 1980s, researchers develop a variety of **approximation algorithms** based on greedy techniques, which provide near-optimal solutions to NP-hard problems. This research continues into the 1990s.