People often do a double-take when hearing the term "Gnome Sort" because it's not that common. The Gnome Sort is extremely simple, and is very similar to the principle behind the Insertion Sort (which we'll be covering soon).

The Gnome Sort is yet another comparison and exchange sort which has elements that are similar to the Bubble Sort. If you haven't read up on the Bubble Sort, be sure to do that before reading this article as it may help with your understanding.

Let's get straight into it!

Common Terms

• Data set - the array or list of items that is to be sorted.
• n - the number of elements in the data set.
• Bubble - perform a Bubble Sort-like shuffle of an item in a given direction.

The Algorithm

The first principle to bear in mind when doing a Gnome Sort is that as the algorithm progresses, all items to the left of the current item are always in order. Each iteration of the algorithm moves the current item to the correct spot with respect to the previously sorted items.

The first part of the algorithm looks at the first two items in the data set and swaps them around if they are in the wrong order. Then, for each iteration thereafter, the next item is bubbled backwards towards to head of the list until a value is encountered which is less than the current item (or the head of the list is reached). At this point, the current item is left alone, and the sort continues from the next item.

The Example

As per usual we will use the same initial data set that we used for the Bubble Sort and Cocktail Sort to aid in highlighting the differences.

The initial set looks like this:

33 98 74 13 55 20 77 45 64 83

The first step is to compare the firs two values:

33 98 74 13 55 20 77 45 64 83

Given that they are in the correct order we leave this two values alone. We can say that two items in the data set are in order.

Next we look at the item immediately after the in order items and compare that to the last sorted item:

33 98 74 13 55 20 77 45 64 83

We can see that these two are not in the correct order, so we swap them:

33 74 98 13 55 20 77 45 64 83

Given that we had to swap the values in the previous step, we can't assume that the current item is in the right location with respect to the other in order items, and hence we need to continue to work backwards. We compare the current item to the preceeding item in the data set:

33 74 98 13 55 20 77 45 64 83

These two values are in the correct order, so we can now stop this iteration. At this point, the first three items in the set are sorted with respect to each other. Next, we move to the fourth item, and compare that with the last of the in order items:

33 74 98 13 55 20 77 45 64 83

They're out of order, so we swap them:

33 74 13 98 55 20 77 45 64 83

... and we keep bubbling backwards until we reach either a value that is smaller or the head of the list:

33 74 13 98 55 20 77 45 64 83
33 13 74 98 55 20 77 45 64 83
33 13 74 98 55 20 77 45 64 83
13 33 74 98 55 20 77 45 64 83

Here we can see that 13 was the smallest number, so it was bubbled all the way to the head of the list. We then move on to the next iteration where we again compare the next item with the head of the list, and again bubble backwards until we reach the correct location. Here is a summary of the steps for the next iteration:

13 33 74 98 55 20 77 45 64 83
13 33 74 55 98 20 77 45 64 83
13 33 74 55 98 20 77 45 64 83
13 33 55 74 98 20 77 45 64 83
13 33 55 74 98 20 77 45 64 83

55 is now in the correct spot with regards to the other pre-sorted items, so we move onto the next item. At this point it should be fairly obvious what's going on, so I shall show a summary for the rest of the example:

13 33 55 74 98 20 77 45 64 83
13 33 55 74 20 98 77 45 64 83
13 33 55 74 20 98 77 45 64 83
13 33 55 20 74 98 77 45 64 83
13 33 55 20 74 98 77 45 64 83
13 33 20 55 74 98 77 45 64 83
13 33 20 55 74 98 77 45 64 83
13 20 33 55 74 98 77 45 64 83
13 20 33 55 74 98 77 45 64 83
13 20 33 55 74 98 77 45 64 83
13 20 33 55 74 77 98 45 64 83
13 20 33 55 74 77 98 45 64 83
13 20 33 55 74 77 98 45 64 83
13 20 33 55 74 77 45 98 64 83
13 20 33 55 74 77 45 98 64 83
13 20 33 55 74 45 77 98 64 83
13 20 33 55 74 45 77 98 64 83
13 20 33 55 45 74 77 98 64 83
13 20 33 55 45 74 77 98 64 83
13 20 33 45 55 74 77 98 64 83
13 20 33 45 55 74 77 98 64 83
13 20 33 45 55 74 77 98 64 83
13 20 33 45 55 74 77 64 98 83
13 20 33 45 55 74 77 64 98 83
13 20 33 45 55 74 64 77 98 83
13 20 33 45 55 74 64 77 98 83
13 20 33 45 55 64 74 77 98 83
13 20 33 45 55 64 74 77 98 83
13 20 33 45 55 64 74 77 98 83
13 20 33 45 55 64 74 77 83 98
13 20 33 45 55 64 74 77 83 98
13 20 33 45 55 64 74 77 83 98

Note that we do not know for sure that an item is in the correct location until the entire set has been sorted.

The Implementation

Here is a sample implementation written in C#. It is heavily commented in the hope that it will aid in understanding how the algorithm works:

/// <summary>
/// Example implementation of the Gnome Sort algorithm.
/// </summary>
/// <param name="dataSet">Array of items to be sorted.</param>
static void GnomeSort(int[] dataSet)
{
    // start at the second item
    int i = 1;
 
    // keep track of the 'next' item so we
    // can jump to it when the current iteration
    // is finished.
    int j = 2;
 
    // loop until we reach the end of the data set
    while (i < dataSet.Length)
    {
        // swap if required
        if (dataSet[i - 1] > dataSet[i])
        {
            Swap(dataSet, i - 1, i);
 
            // move backwards through the sorted items
            // until we find the rightful spot.
            --i;
 
            // stop at the head of the list
            if (i == 0)
            {
                // jump to the next item in the set
                i = j++;
            }
        }
        else
        {
            // jump to the next item in the set
            i = j++;
        }
    }
}

The Complexity

This is not a very efficient algorithm. It's time and space complexity are exactly that of the Bubble Sort. That is, the time complexity is O(n²), and space complexity for in-place sorting is O(1).

Other Implementations

I'm slowly gathering a collection of implementations of all the sorting algorithms, including the ones listed above, that I'm covering in this series and posting them up on BitBucket. The Gnome Sort implementations can be found here.

Note: You'll need Mercurial if you want to pull directly from the repository, otherwise you'll have to copy and paste from the web.

References and Other Information

1. Wikipedia - Gnome sort

Next up, we'll be looking at the first of the mind-bending algorithms: the Quick Sort.

Disclaimer

I'm no expert, nor an authority. The post above is based on my experience and a bit of research. If you find something wrong with anything I've said please let me know. Comments and feedback are greatly appreciated.

Note: For those reading this article in an RSS reader, you may find the colours do not appear in the examples properly. For some reason the feed is stripping out some of the formatting. I will do my best to fix this up soon.

Like the Cocktail Sort, the Comb Sort isn't particularly well known. Most people manage to make their way through tertiary studies without ever hearing of it. This post is designed to change that!

The Comb Sort is another comparison and exchange sort which builds on the idea of the Bubble Sort and adds a potential optimisation or two.

Make sure you read the articles on the Bubble Sort and the Cocktail Sort before you read this article. Doing so will make it much easier to understand.

Right, enough with the introductions, let's break it down!

Common Terms

• Data set - the array or list of items that is to be sorted.
• n - the number of elements in the data set.
• Turtle - a name given to small numbers that appear towards the end of a data set. When used in a Bubble Sort, small numbers at the end of the set take a very long time to get to the front of the list, and hence are called turtles because of the lack of speed.
• Hare - a name given to large numbers that appear towards the start of a data set. When used in a Bubble Sort, large numbers move to the end of the set very quickly, and hence are called hares due to their high speed.
• Killing the turtles - a phrase that implies quickly moving turtles to the front of the data set.
• Gap - the size of the gap between areas of the data set that are being sorted during a given iteration of the sort.
• Shrink Factor - a factor which is applied to the gap prior to each iteration to reduce its size.

A Bit of Background Information

The Comb Sort, along with the Cocktail Sort, was developed as an improvement to the Bubble Sort using the idea of killing the turtles. Turtles drastically reduce the efficiency of the Bubble Sort and hence are an obvious place to attempt optimisation.

The Bubble Sort algorithm always compares values that are adjacent to each other in the data set. The Comb Sort improves on this by adding a gap which allows non-adjacent numbers to be compared. After each iteration, the gap is reduced by a shrink factor until it reaches the value of 1. Hence, at the very end, the Comb Sort behaves exactly like the Bubble Sort.

One interesting thing to note about this algorithm is that it's almost as fast as a Quick Sort!

The Algorithm

Each iteration of the algorithm consists of three stages:

1. Calculation of the gap value.
2. Iterating over the data set comparing each item with the item that is "gap" elements further down the list and swapping them if required.
3. Checking to see if the gap value has reached one and no swaps have occurred. If so, then the set has been sorted.

1. Calculating the "gap"

The gap, which is essentially an offset used when comparing elements, is something that changes for each iteration. The value needs to change in a uniform manner in such a way as to provide optimum comparisons during the sorting phases.

Stephen Lacey and Richard Box, who popularised the algorithm, suggested that in each iteration the gap should be reduced by a factor of approximately 1.3 (see the wikipedia article for more information). Hence, calculation of the gap is as simple as starting with the size of the data set and dividing by a shrink factor of 1.3 each iteration.

2. Iterating and Swapping

This phase is the same as the Bubble Sort. The only difference in the Comb Sort is that the items which are compared are i and i + gap as opposed to i and i + 1.

For each iteration, we start at the beginning of the data set and loop until i + gap is greater-than or equal to the size of the data . For each sub-iteration in this loop, we compare i and i + gap and swap the values if required.

At this point the gap is recalculated and we go again.

3. Terminating the Loop

When the gap value reaches one, we know that we're now behaving the same as the Bubble Sort algorithm (since we're comparing i and i + 1), so we know that if we don't perform any swaps during the iteration then the set must be sorted.

The Example

As per usual we will use the same initial data set that we used for the Bubble Sort and Cocktail Sort to aid in highlighting the differences.

The initial set looks like this:

33 98 74 13 55 20 77 45 64 83

We start by setting our gap value to 10, which is the size of the set. We then shrink the gap by the shrink factor, 1.3, which results in the value of 7 (when rounded down).

The first iteration starts at the beginning of the set, comparing the 1st item with the item that is "gap" elements further up the set (the 8th item). These two items are shown in red:

33 98 74 13 55 20 77 45 64 83

33 is less than 45, so no swap is required. We then move to the 2nd item, and compare that item with the 9th item:

33 98 74 13 55 20 77 45 64 83

We perform a swap at this point because 98 is bigger than 64. The swap is shown in blue:

33 64 74 13 55 20 77 45 98 83

We then move on to compare the 3rd and 10th elements:

33 64 74 13 55 20 77 45 98 83

Again, no swap is required here as the numbers are in the right order.

At this point we have completed our first iteration as we've reached the end of the set. Given that the gap value is not 1, we know we still have more iterations to go. So we need to recalculate our gap value by dividing it again by the shrink factor, resulting in a value of 5 (after rounding down).

We then start another iteration from the beginning of the set, using the gap value of 5 as the offset. We start with the 1st and 6th items:

33 64 74 13 55 20 77 45 98 83

Given 33 is bigger than 20, we swap:

20 64 74 13 55 33 77 45 98 83

Then we compare the 2nd and 7th items:

20 64 74 13 55 33 77 45 98 83

No swap required, move to the 3rd and 8th items:

20 64 74 13 55 33 77 45 98 83

Perform the swap since 74 is bigger than 45:

20 64 45 13 55 33 77 74 98 83

We then compare the 4th and 9th items:

20 64 45 13 55 33 77 74 98 83

No swap required, compare the 5th and 10th items:

20 64 45 13 55 33 77 74 98 83

Again, no swap required. We've hit the end of this next iteration, and since gap doesn't equal 1, we recalculate (resulting in the value of 3) and go again.

This time I'll just show a summary of the steps:

20 64 45 13 55 33 77 74 98 83
13 64 45 20 55 33 77 74 98 83
13 64 45 20 55 33 77 74 98 83
13 55 45 20 64 33 77 74 98 83
13 55 45 20 64 33 77 74 98 83
13 55 33 20 64 45 77 74 98 83
13 55 33 20 64 45 77 74 98 83
13 55 33 20 64 45 77 74 98 83
13 55 33 20 64 45 77 74 98 83
13 55 33 20 64 45 77 74 98 83

Again, we iterate the value of gap, which becomes 2, and we go again:

13 55 33 20 64 45 77 74 98 83
13 55 33 20 64 45 77 74 98 83
13 20 33 55 64 45 77 74 98 83
13 20 33 55 64 45 77 74 98 83
13 20 33 55 64 45 77 74 98 83
13 20 33 45 64 55 77 74 98 83
13 20 33 45 64 55 77 74 98 83
13 20 33 45 64 55 77 74 98 83
13 20 33 45 64 55 77 74 98 83
13 20 33 45 64 55 77 74 98 83

Finally, we reduce the value of gap one more time, resulting in the value of 1. We're finally at the Bubble Sort stage, and hence the last iteration looks like this:

13 20 33 45 64 55 77 74 98 83
13 20 33 45 64 55 77 74 98 83
13 20 33 45 64 55 77 74 98 83
13 20 33 45 64 55 77 74 98 83
13 20 33 45 64 55 77 74 98 83
13 20 33 45 55 64 77 74 98 83
13 20 33 45 55 64 77 74 98 83
13 20 33 45 55 64 77 74 98 83
13 20 33 45 55 64 74 77 98 83
13 20 33 45 55 64 74 77 98 83
13 20 33 45 55 64 74 77 83 98
13 20 33 45 55 64 74 77 83 98

At this point we know, due to observation, that the list is sorted. Unfortunately, the algorithm is yet to know. It looks at the value of gap and sees that it is set to 1, but during the last iteration there were swaps -- so the code will iterate again.

At the end of this iteration, given that there were no swaps, it will know that the list is sorted and the code will terminate.

The Implementation

Here is a sample implementation written in C#. It is heavily commented in the hope that it will aid in understanding how the algorithm works:

/// <summary>
/// The implementation of the standard comb sort algorithm which sorts
/// an array of integers.
/// </summary>
/// <param name="dataSet">Reference to the dataset that is to be soroted.</param>
static void CombSort(int[] dataSet)
{
    // start by using the length/size of the set as the gap.
    int gap = dataSet.Length;
 
    // loop indefinately.
    while (true)
    {
        // update the gap value so that it shrinks
        // towards 1.
        if (gap > 1)
        {
            gap = (int)((double)gap / SHRINK_FACTOR);
        }
 
        // do the comb sort with the current gap
        bool swapped = false;
        for (int i = 0; i + gap < dataSet.Length; ++i)
        {
            if (dataSet[i] > dataSet[i + gap])
            {
                Swap(dataSet, i, i + gap);
                swapped = true;
            }
        }
 
        // if we're down to a gap of 1, and we haven't swapped
        // anything, then we're sorted.
        if (gap == 1 && !swapped)
        {
            break;
        }
    }
}

The Complexity

This is quite surprising. Despite being based on the idea of a Bubble Sort the time complexity is just O(n log n), and space complexity for in-place sorting is O(1). Amazing for such a simple sorting algorithm!

A Note on Performance

What I'm about to say is almost a direct rip-off of Wikipedia But at least I'm being honest!

When you use a shrink factor of 1.3, as recommended, it turns out that there are only 3 ways for the gap sizes to reduce to 1. They are:

1. - 9, 6, 4, 3, 2, 1
2. - 10, 7, 5, 3, 2, 1
3. - 11, 8, 6, 4, 3, 2, 1

According to Wikipedia, it turns out that the 3rd option is the only one which manages to kill all turtles before the gap becomes 1. I haven't been able to find anything that backs this up though.

Assuming that this is true, we can create a modified version of the Comb Sort which has this extra little check:

if(gap == 9 || gap == 10)
{
    gap = 11;
}

This is a little hack which makes sure that we get the third option before we reduce the gap to 1. Of course, this would only have an effect on sets of data which have more than 11 elements.

Other Implementations

I'm slowly gathering a collection of implementations of all the sorting algorithms, including the ones listed above, that I'm covering in this series and posting them up on BitBucket. The Comb Sort implementations can be found here.

Note: You'll need Mercurial if you want to pull directly from the repository, otherwise you'll have to copy and paste from the web.

References and Other Information

1. Wikipedia - Comb sort

Next up, we'll be looking at the Gnome Sort.

Disclaimer

I'm no expert, nor an authority. The post above is based on my experience and a bit of research. If you find something wrong with anything I've said please let me know. Comments and feedback are greatly appreciated.

Note: For those reading this article in an RSS reader, you may find the colours do not appear in the examples properly. For some reason the feed is stripping out some of the formatting. I will do my best to fix this up soon.

This algorithm was the next logical choice in the series because it is very similar to the Bubble Sort in the way that it operates. If you're yet to read the first in the series, head over there now as it will make this algorithm easier to understand.

Fundamentally the algorithm is the same. The difference is that the Cocktail Sort iterates through a given data set in both directions when sorting. So let's break it down.

Common Terms

• Data set - the array or list of items that is to be sorted.
• n - the number of elements in the data set

The Algorithm

Each iteration of the algorithm is broken up into two stages.

The first stage loops through the data set from bottom to top, just like the Bubble Sort. During the loop, adjacent items are compared. If at any point the value on the left is greater than the value on the right, the items are swapped. At the end of the first iteration, the largest number will reside at the end of the set.

The second stage loops through the data set in the opposite direction - starting from the item just before the most recently sorted item, and moving back towards the start of the list. Again, adjacent items are swapped if required.

The Cocktail Sort also fits in the category of Exchange Sorts due to the manner in which elements are moved inside the data set during the sorting process.

The Example

We will use the same initial data set that we used for the Bubble Sort to aid in highlighting the differences.

The initial set looks like this:

33 98 74 13 55 20 77 45 64 83

The first iteration starts at the beginning of the list, comparing the first two items (marked in red):

33 98 74 13 55 20 77 45 64 83

Since 33 is less than 98, no swapping needs to be done as they're already in the correct order. So we move on to the next comparison:

33 98 74 13 55 20 77 45 64 83

This time a swap is required as 98 is greater than 74:

33 74 98 13 55 20 77 45 64 83

We do the same again, this time starting at the third item:

33 74 98 13 55 20 77 45 64 83

Again, we need to swap the items since they're not in order:

33 74 13 98 55 20 77 45 64 83

We repeat this process until we get to the end of the list (marked in green):

33 74 13 55 20 77 45 64 83 98

As was mentioned earlier, the result is that the largest number, 98 is placed at the end of the list. The next stage of the first iteration requires us to loop in the opposite direction. Since we know that 98 is in its rightful place, we start at the items immediately to the left:

33 74 13 55 20 77 45 64 83 98

We perform the same comparison as normal, and in this case we can see that we don't have to swap the items because 64 is less than 83. We move on to the next pair:

33 74 13 55 20 77 45 64 83 98

Again, we find that a swap is not necessary because 45 is less than 64. Moving down the list again we compare the previous pair:

33 74 13 55 20 77 45 64 83 98

This time we do want to swap the items, because 45 is less than 77, and hence the items are in the wrong order.

33 74 13 55 20 45 77 64 83 98

With the swap complete we again move to the previous pair:

33 74 13 55 20 45 77 64 83 98

Again, no swap needed, look at the previous pair:

33 74 13 55 20 45 77 64 83 98

These two are not in the right order, so swap them:

33 74 13 20 55 45 77 64 83 98

With the swap performed, we again move to the previous pair:

33 74 13 20 55 45 77 64 83 98

No swap needed, go to the previous pair:

33 74 13 20 55 45 77 64 83 98

These are out of order, so swap:

33 13 74 20 55 45 77 64 83 98

Finally we look at the last pair in this stage:

33 13 74 20 55 45 77 64 83 98

Again a swap is required:

13 33 74 20 55 45 77 64 83 98

We're now done with the second stage, and as we can see we have the highest and lowest values at the end and start of the set (respectively):

13 33 74 20 55 45 77 64 83 98

So at this point we're ready to iterate again, but we don't want to include the items that have already been sorted because we know they're in the right spot. Here's a short hand demo of both stages of the next iteration (remember, comparisons are in red, swaps are in blue, stores are in green):

13 33 74 20 55 45 77 64 83 98
13 33 74 20 55 45 77 64 83 98
13 33 20 74 55 45 77 64 83 98
13 33 20 74 55 45 77 64 83 98
13 33 20 55 74 45 77 64 83 98
13 33 20 55 74 45 77 64 83 98
13 33 20 55 45 74 77 64 83 98
13 33 20 55 45 74 77 64 83 98
13 33 20 55 45 74 77 64 83 98
13 33 20 55 45 74 77 64 83 98
13 33 20 55 45 74 77 64 83 98
13 33 20 55 45 74 77 64 83 98
13 33 20 55 45 74 64 77 83 98
13 33 20 55 45 74 64 77 83 98
13 33 20 55 45 64 74 77 83 98
13 33 20 55 45 64 74 77 83 98
13 33 20 55 45 64 74 77 83 98
13 33 20 45 55 64 74 77 83 98
13 33 20 45 55 64 74 77 83 98
13 33 20 45 55 64 74 77 83 98
13 20 33 45 55 64 74 77 83 98
13 20 33 45 55 64 74 77 83 98

The process repeats again. But this time as we iterate through, we can see that no swaps are required. The result after the next iterate is simple:

13 20 33 45 55 64 74 77 83 98

As we can see, using this algorithm to sort this particular data set results in less work than when using the Bubble Sort.

The Implementation

Here is a sample implementation written in C#. It is heavily commented in the hope that it will aid in understanding how the algorithm works:

/// <summary>
/// Performs a cocktail sort on an array of integers.
/// </summary>
/// <param name="dataSet">An array of ints to be sorted.</param>
static void CocktailSortBasic(int[] dataSet)
{
    bool swapped = false;
    int start = 0;
    int end = dataSet.Length - 1;
 
    do
    {
        // make sure we reset the swapped flag on entering
        // the loop, because it might be true from a previous
        // iteration.
        swapped = false;
 
        // loop from bottom to top just like we do with
        // the bubble sort
        for (int i = start; i < end; ++i)
        {
            if (dataSet[i] > dataSet[i + 1])
            {
                Swap(dataSet, i, i + 1);
                swapped = true;
            }
        }
 
        // if nothing moved, then we're sorted.
        if (!swapped)
        {
            break;
        }
 
        // otherwise, reset the swapped flag so that it
        // can be used in the next stage
        swapped = false;
 
        // move the end point back by one, because we know
        // that the item at the end is in its rightful spot
        --end;
 
        // this time we loop from top to bottom, doing the
        // same comparison as in the previous stage
        for (int i = end - 1; i >= start; --i)
        {
            if (dataSet[i] > dataSet[i + 1])
            {
                Swap(dataSet, i, i + 1);
                swapped = true;
            }
        }
 
        // this time we increase the starting point, because
        // the last stage would have moved the next smallest
        // number to its rightful spot.
        ++start;
    } while (swapped);
}

The Complexity

Both space and time complexity are the same as that of the Bubble Sort for exactly the same reasons. That is, time complexity is O(n²), and space complexity for in-place sorting is O(1).

A Note on Performance

The Cocktail Sort can actually prove to be faster than the Bubble Sort in a fair few cases. This is due to the fact that we sort in both directions each iteration instead of just one.

Here's an example data set which would require 9 iterations with a Bubble Sort, but only 1 iteration (of two stages) with a Cocktail Sort:

20 33 45 55 64 74 77 83 98 13

The second stage of the Cocktail Sort would simply move the number 13 all the way down to the start of the list, at which point the list is then sorted. The Bubble Sort would move the number 13 left one place for each iteration.

In general, the Cocktail Sort will perform, at worst, the same as the Bubble Sort.

Other Implementations

I'm slowly gathering a collection of implementations of all the sorting algorithms, including the ones listed above, that I'm covering in this series and posting them up on BitBucket. The Cocktail Sort implementations can be found here.

Note: You'll need Mercurial if you want to pull directly from the repository, otherwise you'll have to copy and paste from the web.

References and Other Information

1. Wikipedia - Cocktail sort

Next up, we'll be looking at the Comb Sort.

Disclaimer

I'm no expert, nor an authority. The post above is based on my experience and a bit of research. If you find something wrong with anything I've said please let me know. Comments and feedback are greatly appreciated.

Note: For those reading this article in an RSS reader, you may find the colours do not appear in the examples properly. For some reason the feed is stripping out some of the formatting. I will do my best to fix this up soon.

I chose the Bubble Sort algorithm as the first to cover because of its simplicity. This algorithm tends to be the first sorting algorithm that is taught to students, and hence is a rather apt starting point.

Let's break it down.

Common Terms

• Data set - the array or list of items that is to be sorted.
• n - the number of elements in the data set

The Algorithm

The algorithm consists of a repeated iteration over the elements of the data set. In each iteration, adjacent elements are compared. If those two adjacent items are in the wrong order, they are swapped. The result of each iteration is that the next highest value is placed in the appropriate location in the data set. After repeating the iteration n - 1 times, the entire data set will be sorted.

The Bubble Sort fits in the category of Exchange Sorts due to the manner in which elements are moved inside the data set during the sorting process.

The Example

We start with an unsorted data set of 10 elements which we want to sort in ascending order:

33 98 74 13 55 20 77 45 64 83

The start of the first iteration looks at the first two elements (marked in red):

33 98 74 13 55 20 77 45 64 83

As per the algorithm we compare the two values, swapping them if the first item is bigger than the second. In this case, 98 is bigger than 33, so no swap is required.

Next we look at the two adjacent items, starting at the second item in the list:

33 98 74 13 55 20 77 45 64 83

In this case 98 is greater than 74, so the items must be swapped (marked in blue):

33 74 98 13 55 20 77 45 64 83

We do the same again, this time starting at the third item:

33 74 98 13 55 20 77 45 64 83

Again, we need to swap the items since they're not in order:

33 74 13 98 55 20 77 45 64 83

We repeat this process until we get to the end of the list (marked in green):

33 74 13 55 20 77 45 64 83 98

As we can see, the result is that the largest number is placed at the end of the list. This is the end of the first iteration.

We then iterate again but only cover the section of numbers that haven't already been sorted.

33 74 13 55 20 77 45 64 83 98

No swap required, move to the next pair.

33 74 13 55 20 77 45 64 83 98

Swap required:

33 13 74 55 20 77 45 64 83 98

Move to the next pair:

33 13 74 55 20 77 45 64 83 98

Again, swap required:

33 13 55 74 20 77 45 64 83 98

We continue this trend for another iteration before hitting the following case:

33 13 55 20 74 77 45 64 83 98

Here, there is no swap required, so we leave behind the number we've been "carrying" (74) and pick up number 77.

33 13 55 20 74 77 45 64 83 98

The result of the iteration is as follows:

33 13 55 20 74 45 64 77 83 98

As we can see, this "bubbles" each of the numbers to the top of the list in the order of highest to lowest. Here is how the rest of the numbers are sorted at the end of each following iteration:

13 33 20 55 45 64 74 77 83 98
13 20 33 45 55 64 74 77 83 98
13 20 33 45 55 64 74 77 83 98
13 20 33 45 55 64 74 77 83 98
13 20 33 45 55 64 74 77 83 98
13 20 33 45 55 64 74 77 83 98
13 20 33 45 55 64 74 77 83 98

Note that the last iteration results in two numbers being locked in due to the fact we no longer have any numbers to sort.

The Implementation

Here's a commented sample in C# which is easily translatable to many other languages:

/// <summary>
/// This is the basic bubble sort algorithm, hard-coded to work
/// with integer values.
/// </summary>
/// <param name="dataSet">Array of integers to sort.</param>
static void BubbleSortBasic(int[] dataSet)
{
    // loop n-1 times.
    for (int i = dataSet.Length - 1; i > 0 ; --i)
    {
        // for each loop, iterate through the first i
        // items (ie. the unsorted ones)
        for (int j = 0; j < i; ++j)
        {
            // if adjacent items need to be swapped
            if (dataSet[j] > dataSet[j + 1])
            {
                // swap them
                Swap(dataSet, j, j + 1);
            }
        }
    }
}

A general form of this algorithm which applies to any comparable type can be found in the source repository listed below.

A Minor Optimsation

The Bubble Sort can be optimised very slightly, though it's not guaranteed to provide much benefit depending on the structure of the data set that is to be sorted.

If, for any iteration, there are no items swapped then all of the items in the data set must be in the correct order. As a result, subsequent iterations are unnecessary. The optimised version is listed below, again in C#:

/// <summary>
/// This is the basic bubble sort algorithm, hard-coded to work
/// with integer values but with a slight difference - a minor
/// optimisation.
/// </summary>
/// <param name="dataSet">Array of integers to sort.</param>
static void BubbleSortBasicOptimised(int[] dataSet)
{
    // loop n-1 times.
    for (int i = dataSet.Length - 1; i > 0 ; --i)
    {
        // keep track of whether items were swapped
        // for this iteration
        bool swapped = false;
 
        // for each loop, iterate through the first i
        // items (ie. the unsorted ones)
        for (int j = 0; j < i; ++j)
        {
            // if adjacent items need to be swapped
            if (dataSet[j] > dataSet[j + 1])
            {
                // swap them
                Swap(dataSet, j, j + 1);
 
                // indicate that we found a swap
                swapped = true;
            }
        }
 
        // if nothing was swapped, then we should
        // already have everything in order
        if (!swapped)
        {
            break;
        }
    }
}

Caveat: This "optimisation" may actually result in lower performance, particularly in the case where every iteration results in a swap.

The Complexity

Space Complexity

Bubble Sorts are extremely efficient in terms of memory usage, due to the fact that all sorting and swapping operations are done on the original data set. Regardless of the size of the original data set, the amount of memory overhead is constant as we don't allocate any memory for each of the items. Hence, the space complexity is O(1) (in Big-O notation). This, of course, doesn't include the O(n) space complexity taken up by the original data set.

Time Complexity

This is where the Bubble Sort fails to shine. For each iteration i starting at n, we must loop through the previous i - 1 elements and swap if required.

The first iteration loops through n - 1 elements.
The second iteration loops through n - 2 elements.
The third iteration loops through n - 3 elements.
And so on.. which means the number of compares/swaps that we do is equal to:
(n - 1) + (n - 2) + (n - 3) ... + 2 + 1
This indicates that there n lots of n - i, or n² - ni (where ni varies for each iteration).

Give that the highest power of n is 2 this indicates a time complexity of O(n²).

Or in layman's terms: it's f**king slow

Bubble Sorts should only be used on data sets that are rather small. If you're dealing with medium-sized or larger sets of data, then the Bubble Sort is not the right algorithm to choose. So what would be a better option? You'll have to read the rest of this series and answer that yourself

Other Implementations

I'm slowly gathering a collection of implementations of all the sorting algorithms, including the ones listed above, that I'm covering in this series and posting them up on BitBucket. The Bubble Sort implementations can be found here.

Note: You'll need Mercurial if you want to pull directly from the repository, otherwise you'll have to copy and paste from the web.

Closing Thoughts

I haven't included information on multithreading because the post would be insanely big. I will put up a follow-up post covering multithreading another day. The code stored in the BitBucket repository contains a sample of how you might use multithreading in conjunction with this sorting algorithm.

To wrap up, Bubble Sorts are easy to understand and are a great place to start when learning sorting algorithms. Unfortunately, this simplicity results in a fairly expensive and slow sorting implementation which really isn't an option when dealing with anything other than small data sets.

References and Other Information

1. Wikipedia - Bubble sort

Next up, we'll be looking at the Cocktail Sort.

Disclaimer

I'm no expert, nor an authority. The post above is based on my experience and a bit of research. If you find something wrong with anything I've said please let me know. Comments and feedback are greatly appreciated.

That topic is Sorting.

On the surface it appears to be so simple. Just arrange some stuff in a certain order. What could be easier?

Unfortunately the implementations of various methods of sorting can be anything but easy. Determining which algorithm to choose can be difficult enough given that some lend themselves to sorting certain data types better than others do.

As a result, I've decided to start writing a series on the different well-known (and perhaps not-so-well-known) sorting algorithms. For each algorithm, my goal will be to:

1. Paint a very clear picture of how it functions. I aim to do this via pictures and discussion. If I get time, I will aim to provide an animated visualiser of the algorithm (though at the moment this might be a tough ask given time constraints).
2. Explain the order of complexity/Big-O notation so that it's clear just how expensive it is to use, along with details of best and worst cases.
3. Give examples of data sets where the algorithm fits well, and examples of where it doesn't.
4. Explain if and how the given algorithm can be used in a multhithreaded environment.
5. Demonstrate errors that are found frequently in various implementations (if there are any), and show how to resolve/avoid them.

By the end of the series, I hope that you (and I) will have a really solid understanding of the Sorting world.

To start off with, I'll be covering the age-old Bubble Sort. I'm choosing this because it's very simple, and is an easy target for me to get going.

As always, comments and feedback will be greatly appreciated. I'm hoping that I'll learn more from this exercise than you guys will.