Why the fastest algorithm often loses in real-world applications?

The process of selecting the most efficient algorithm is similar to selecting the appropriate tool to work on a job, that is, you need to know your problem and your limitations. Here is the way to do it in a systematic way:

1.   Start with Problem Analysis
The first step is to identify your type of problem. Is it a search problem, sorting problem, traversal of a graph or optimization problem? There are family of algorithms defined on different problems. An example is that when you are working with shortest paths, you are considering algorithms such as Dijkstra or A, and not sorting algorithms.

2.  Knowledge Your Data Characteristics.
The kind of input you have has a drastic impact on the choice of algorithm. Small datasets (about 1000 items) may be well served by simple O(n 2 ) algorithms, and large datasets require O(n log n ) or more. Think about your existing data whether it is already partially sorted, or has duplicates or even certain patterns that can be exploited by certain algorithms.

3.  Know the Big O Complexity
Your major metrics are time and space complexity. The hash tables have a better average look but consume a higher memory. Merge sort offers O(n log n) at the cost of extra space whereas heap sort offers O(n log n) with no extra space.

4.  Take into account Real-World Constraints.
Theoretical efficiency is not all. Theoretically faster algorithms may not do well in practice because of:

• Cache behavior (good locality algorithms tend to be good cache performers)
  Complexity in implementation (easier algorithms contain less bugs)
  Space limit (in-place algorithms vs. ones that need additional space)
  Stability requirements (should equal elements retain their relative positioning?)

Benchmark When in Doubt
To test important applications, apply and test several applicants using realistic data. Microbenchmarks are deceiving, use real-life tests. There are cases when a slower algorithm can be the winner because it has better constants or cache behavior.
Context-Specific Factors
Consider your environment:

Embedded systems: Prefer memory aware algorithms.
Real-time systems: Code performance should be focused on predictable behavior rather than the average case.
Distributed systems: The latency of a network may be many times larger than the difference in computations.
Interactive applications: Peak throughput is less important than constant response time.

The 80/20 Rule
Frequently, a well understood, simple algorithm that is good enough replaces an optimal solution that is complicated. The worst case of quick sort is O(n 2), but the average performance of the algorithm is O(n log n) which makes the algorithm practical under good pivot selection.
Decision Framework in Practice.

• Determine the type and size of problem.
  List candidate algorithms
  Compare abstract complexities.
  Take into account cost of implementation and maintenance.
  Use real-life data to test whether performance is a critical issue.
  Select the easiest option that will satisfy your needs.

Keep in mind: premature optimization is bad, but so is not caring about the efficiency of algorithms at all. The trick is to base the complexity of the solution on the real needs of the problem rather than to presume the simplest or the most complex solution.
The most effective algorithm is the one that can work well on your problem with the constraints you have not always the algorithm that looks good on paper.