Algorithm Analysis :  Big-O notation continue

 

Objectives of this lecture

q       Learn how to generate running time functions of algorithms for analysis using Big-O notation

q       Lean general rules that can be used to simplify the generation process

 

Generating running time function

q       To apply Big-O notation on an algorithm, we first need to represent the running time of the algorithm as a function.

q       In doing this, we should always remember that Big-Oh is an upper band.  Thus, we must be careful not to underestimate the running time.

q       To generate the function, we assume that each simple instruction such as addition, multiplication, comparison and assignment is computed in a unit of time.  We, thus need to count how many such instructions are executed.

 

Examples:

consider the following function:

 

unsigned int sum (int n)

{    unsigned int i, partial_sum;

 

/* 1 */       partial_sum = 0;

/* 2 */       for (i=0; i<=n; i++)

/* 3 */             partial_sum += i*i*i;

/* 4 */      return (partial_sum);

            }

 

Solution:

Line 1 & 4:  This counts for unit time each;  [2]

Line 3:     This counts for 3 units (two multiplications and one addition) and is executed n-times [3n]

Line 2:  This has hidden cost of initializing i (1 time); testing (n+1 times); and incrementing (n times)  [2n + 2]

 

Thus, the running time of the function is given by: 

f(n) = 2 + 3n + 2 +2n = 5n + 4.

 

This can be simplified using Big-O notation as  O(n).

 

General Simplification Rules:

q       Since the final answer to algorithm analysis is in terms of Big-O, the above detail computations is clearly time-wasting.  We can simplify it following the rules of Big-O notation.  For example, line 3 is obviously an O(1) statement, thus, there is no need to count how many units it executes.

q       This leads to the following general guidelines for use in generating running time function of algorithms.

 

1. for Loops: The running time of a for loop is at most the running time of the statements inside the loop times the number of iteration

 

2. Nested Loops: Analyze these inside-out.  The total running time of a statement inside a group of nested loops is the running time of the statements multiplied by the product of the sizes of all the loops.

E.g., the following program fragment is O(n²)

for (i=0; i<n; i++)

     for (j=0; j<n; j++)

           k++;

 

3. Consecutive Statements: Choose the maximum among them.

E.g., the following program fragment which has O(n) + O(n²) can be taken to be O(n²).

for (i=0; i<n; i++)

    a[i] = 0;

for (i=0; i<n; i++)

     for (j=0; j<n; j++)

           a[i] += a[j] +i +j;

 

4. if/else Statement: The running time of an if/else statement is never more than the running time of  the test, plus the larger of the running times of if and else parts

 

5. Function Calls: The running time of function calls is the result of analyzing the functions

 

6. Recursive Functions: These are evaluated by generating a recurrence relation.  We shall see how to do this later.

Examples: Maximum sub-sequence problem

The following function computes the maximum sub-sequence sum of a given sequence of (possibly negative) integers, a₁, . . .a_n .

E.g.  for the sequence –2, 11, -4, 13, -5, -2,

the answer is 20 (a₂ . . . a₄) 

 

 

 int max_subsequence_sum (int a[], int n)

{    int this_sum, max_sum, i, j, k;

 

/* 1 */       max_sum=0;

/* 2 */       for (i=0; i<n; i++)

/* 3 */             for (j=i; j<n; j++)

/* 4 */             {   this_sum=0;

/* 5 */                  for (k=i; k<=j; k++)

/* 6 */                      this_sum += a[k];

 

/* 7 */                  if (this_sum > max_sum)

/* 8 */                         max_sum = this_sum;

                          }

/* 9*/      return (max_sum);

            }

 

Solution:

We observe that the statements in line 5 and 6 which are O(1), are nested inside 3 loops.

The first loop is executed n times.

The second loop is executed n – i + 1, which can be small, but could also be n.  Since we are estimating the worst case, we take n.

The third loop executes j – i + 1, which again could be n.

Thus, statements in line 5 and 6 are executed  O(n³) times.

 

Statements 7 and 8 take O(n²) since they are contained in two loops and so can be ignored.  Other statements can also be ignored for  similar reasons.

Thus, the overall running time of the algorithm is O(n³).