5.1.6 Multi-criteria decision making

Multi-criteria decision making1

In decision-making under risk and uncertainty we evaluate the alternatives according to one criterion only. In many practical applications we must evaluate the alternatives against more criteria, sometimes controversial, often of differing importance expressed by their weights.
The evaluation methods introduced here are adaptable to many situations, as determined by the complexity of the problem, needs of the customer, experience of the decision team/analysts/facilitators, and the time and resources available. No decision making method is appropriate for all decisions.

The examples provided in Appendix 5.1-A.xls are intended to facilitate understanding and use of these methods.

Pros and Cons analysis
Pros and Cons Analysis is a qualitative comparison method in which positive aspects (pros) and negative aspects (cons) are identified for each alternative. Lists of the pros and cons, based on the input of subject matter experts, are compared each other for all alternatives. The alternative with the strongest pros and weakest cons is preferred. The documentation regarding the decision should include an exposition, which justifies why the preferred alternative pros are more important and cons are less significant than those of the other alternatives. Pros and Cons Analysis is suitable for simple decisions with few alternatives (2 to 4) and few discriminating criteria (1 to 5) of approximately equal value. It requires no mathematical skill and can be implemented rapidly.

Kepner-Tregoe (K-T) decision analysis
K-T is a quantitative comparison method in which a team of experts numerically score criteria and alternatives based on individual judgements/assessments. In K-T method each evaluation criterion is first scored based on its relative importance to the other criteria (1 = least; 10 = most). These scores become the criteria weights (see K-T example in Appendix 5.1-A.xls).
When the time comes to evaluate the alternatives, the alternatives are scored individually against each of the goal criteria based on their relative performance. A total score is then determined for each alternative by multiplying its score for each criterion by the criterion weights (relative weighting factor for each criterion) and then summing across all criteria. The preferred alternative will have the highest total score. K-T Decision Analysis is suitable for moderately complex decisions involving a few criteria. The method requires only basic arithmetic. Its main disadvantage is, for istance, that it may not explain how much better a score of “10” is than a score of “8”. Moreover, total alternative scores may be close together, making a clear choice difficult.

Analytic Hierarchy Process (AHP) – Saaty´s model
AHP is a quantitative comparison method used to select a preferred alternative by using pair-wise comparisons of the alternatives, based on their relative performance against the criteria. The basis of this technique is that humans are more capable of making relative judgements than absolute judgements. The pair-wise comparisons are made using a nine-point scale:

1 = Equal importance or preference
3 = Moderate importance or preference of one over another
5 = Strong or essential importance or preference
7 = Very strong or demonstrated importance or preference
9 = Extreme importance or preference

Matrices are developed wherein each criterion/alternative is compared against the others. If Criterion A is strongly more important compared to Criterion B (i.e. a value of “5”), then Criterion B has a value of 1/5 compared to Criterion A. Thus, for each comparative score given, the reciprocal is awarded to the opposite relationship. The normalized weight is calculated for each criterion using the geometric mean2 of each row in the matrix divided by the sum of the geometric means of all the criteria. This process is then repeated for the alternatives comparing them one to another to determine their relative value/importance for each criterion (i.e. determine the mean weighted value). The calculations are easily set up in a spreadsheet (see example in Appendix 5.1-A.xls). The order of comparisons can help simplify this process. Try to identify and begin with the most important criterion and proceed through the criteria to the least important. When comparing alternatives try to identify the one with the greatest benefits for each associated criterion, and begin with it. To identify the preferred alternative calculate the mean weighted value, i.e. multiply each normalized alternative score by the corresponding criterion weight, and sum the results for all of criteria. The preferred alternative will have the highest total score.
AHP, like the other methods, can rank alternatives according to quantitative or qualitative (subjective) data. Qualitative/subjective criteria are based on the evaluation team’s feelings or perceptions. A sensitivity analysis can be performed to determine how the alternative selection would change with different criteria weights. The whole process can be repeated and revised, until everyone is satisfied by the fact that all the important features needed to solve the problem, or the selection of the preferred alternative, have been covered.
AHP is a useful technique when there are multiple criteria, since most people cannot deal with more than seven decision considerations at a time.

Example 2: Example of multi-criteria decision-making process is presented in Appendix 5.1-A.xls. Sheet 1 defines the problem: to pick a replacement vehicle for the motor pool fleet.
The requirements are:

  1. made in USA (eliminates products not manufactured in the USA);
  2. minimum 4, maximum 6 passengers (eliminates vans, minibuses, sports cars);
  3. maximum cost $28 000 (eliminates high-end, luxury cars);
  4. new car, current model year (eliminates used vehicles).

We agreed on the following goals:

  1. maximize passenger comfort;
  2. maximize passenger safety;
  3. maximize fuel efficiency;
  4. maximize reliability;
  5. minimize investment cost.

Despite the limitations, many alternatives remain - see the preliminary selection of four models in sheet "Inputs".
Sheet “Pros_and_Cons” illustrates the procedure of Pros and Cons approach – in corresponding boxes we list arguments for and against each alternative and in the end select one with the highest score in the most important criteria. This approach can be easily extended by assigning numerical values to criteria and use so called “forced-field analysis”3.
Sheet “Kepner-Tregoe” can be used as a guide to the K-T method. In this method, we first assign weights to criteria (column “criteria weights”) and then evaluate the level of criteria satisfaction on the scale 0-100 for each alternative (“alternative score”). Finally we calculate total scores and select the alternatives with the highest total score. Please, study the formulas in column “total score”. You can use this sheet as a template for your problem.
Sheet “AHP (Saaty)” shows how to use the AHP method. This approach is the most sophisticated of all approaches presented here and, in order to understand it properly, we suggest to carefully studying all the formulas in this sheet.

1 This part has been taken from Baker D. et al., Guidebook to Decision-Making Methods, pages 6 - 8
2 The geometric mean is the nth root of the product of n values. Thus, the geometric mean of the scores: 1, 2, 3, and 10 is the fourth root of (1 x 2 x 3 x 10), which is the fourth root of 60, (60)1/4 = 2.78. The geometric mean is less affected by extreme values than is the arithmetic mean.
3 ?