What does 'analysis of algorithms' mean?
It is the process of finding the computational complexity of algorithms—the amount of time, storage, or other resources needed to execute them.
What are the two main resources measured in algorithm analysis?
Time (number of steps) and storage (memory locations used).
What is time complexity?
A function that relates the size of an algorithm's input to the number of steps it takes.
What is space complexity?
A function that relates the size of an algorithm's input to the number of storage locations it uses.
When is an algorithm considered efficient?
When the function for its resource use has small values or grows slowly compared to the growth in input size.
What are the three cases used to describe algorithm behavior for different inputs of the same size?
Best, worst, and average case.
When not specified, which case is usually used to describe an algorithm's performance?
The worst case, as an upper bound.
Who coined the term 'analysis of algorithms'?
Donald Knuth.
What is Big O notation used for?
To estimate the complexity of an algorithm for arbitrarily large input sizes, in an asymptotic sense.
What does O(log n) mean colloquially?
Logarithmic time—the number of steps grows proportionally to the logarithm of the input size.
Give an example of an algorithm that runs in O(log n) time.
Binary search on a sorted list.
What is a 'hidden constant' in asymptotic analysis?
A constant multiplicative factor that relates the efficiencies of any two reasonable implementations of the same algorithm.
Why are exact efficiency measures rarely used?
They usually require assumptions about the particular implementation, called a model of computation.
What is a model of computation?
An abstract computer or set of assumptions about which operations take unit time.
What is the uniform cost model?
It assigns a constant cost to every machine operation, regardless of the size of the numbers involved.
What is the logarithmic cost model?
It assigns a cost to every machine operation proportional to the number of bits involved.
When is the logarithmic cost model used?
When necessary, e.g., for arbitrary-precision arithmetic algorithms like those used in cryptography.
What is run-time analysis?
A theoretical classification that estimates how running time increases as input size (n) increases.
Why can't software profiling provide timing data for all possible inputs?
Because there are infinitely many possible inputs; theoretical methods are needed.
What is a drawback of empirical (benchmark) metrics for comparing algorithms?
Algorithms are platform-independent, so results vary with hardware, programming language, and operating system.
Give an example showing why empirical metrics can be misleading.
A fast computer using linear search may beat a slow computer using binary search for small lists, but for large lists binary search becomes far better.
What is the maximum number of time units needed for binary search on a sorted list of n elements, assuming unit-time lookups?
At most log2(n) + 1 time units.
Why must we be careful when counting an addition as one step?
If numbers can be arbitrarily large, the time for a single addition may not be constant.
What is the key point about published lower bounds for problems?
They are often for a restricted model of computation, so algorithms may be faster than naively thought possible.