Outcome Bias In Decision Evaluation
This summarizes it, “A judge who does not know the decision-maker’s probabilities may assume that the probability was higher for an outcome that occurred than for the same outcome, had it not occurred.”
Click Here To Learn About Outcome Bias In Decision Evaluation
Abstract (Via Upenn)
Subjects were given descriptions of decisions made by others under conditions of uncertainty, together with outcomes of those decisions. Some decisions were medical decisions made by a physician or a patient, and others were decisions about monetary gambles. Subjects rated the quality of thinking that went into the decisions, the competence of the decision maker, or their willingness to let the decision maker decide on their behalf. Subjects understood that they had all relevant information available to the decision maker. Subjects rated the thinking as better (or rated the decision maker as more competent, or indicated greater willingness to yield the decision) when the outcome was favorable than when it was unfavorable. In monetary gambles, subjects rated the thinking as better when the outcome of the option not chosen turned out poor than when it turned out well. Subjects who were asked felt that they should not take outcomes into account in making these evaluations, yet, they did so. In part, this effect of outcome knowledge on evaluation may be explained in terms of its effect on the salience of arguments for each side of the choice. Implications of these findings for the theory of rationality and for practical situations are discussed.
Excerpts (Via Upenn)
Evaluations of decisions are made in our personal lives, in organizations, in judging the performance of elected officials, and in certain legal disputes, such as malpractice suits, liability cases, and regulatory decisions. Because evaluations are made after the fact, there is often information available to the judge that was not available to the decision maker, including information about the outcome of the decision. It has often been suggested that such information is used unfairly, that reasonable decisions are criticized by Monday-morning quarterbacks who think they might have decided otherwise, and that decision makers end up being punished for their bad luck (e.g., Arnauld, 1662/1964; Berlin, 1984; Nichols, 1985).
The distinction between a good decision and a good outcome is a basic one to all decision analysts. The above quotation from Edwards (1984) is labeled by the author as `a very familiar elementary point.’ In this paper, we explore how well the distinction between decisions and outcomes is recognized outside the decision-analysis profession.
Information that is available only after a decision is made is irrelevant to the quality of the decision. Such information plays no direct role in the advice we might give decision makers ex ante or in the lessons they might learn (Baron, 1985, ch. 1). The outcome of a decision, by itself, cannot be used to improve a decision unless the decision maker is clairvoyant.
Information about possible outcomes and their probabilities falls into three relevant classes: Actor Information, known only to the decision maker at the time the decision is made; Judge Information, known only to the judge at the time the decision is evaluated; and Joint Information, known both to the decision-maker at the time of decision and to the judge at the time of evaluation. (In some cases, the decision maker and the judge will be the same person, at different times.) In the cases we consider, the judge has the outcome information, and the actor does not.
Although outcome information plays no direct role in the evaluation of decisions, it may play a very appropriate indirect role. In particular, it may affect a judge’s beliefs about Actor Information. A judge who does not know the decision-maker’s probabilities may assume that the probability was higher for an outcome that occurred than for the same outcome, had it not occurred. (Note, however, that outcome information tells us nothing about the utilities of a decision maker, even if we have no other information about them.) In the extreme, if we have no information except outcome, it is a reasonable prima-facie hypothesis that bad outcomes (e.g., space-shuttle accidents) result from badly made decisions. We do not usually set up commissions of inquiry to delve into policy decisions that turn out well.
Click Here To Learn About Outcome Bias In Decision Evaluation
