D&M Metacognition - Ch 6 notes

Chapter 6 -- Confidence Judgments

Retrospective Confidence Judgments -- AKA RC
This is when you generate an answer as well as judging our confidence in whether the answer is correct. A type of metamemory judgment. Useful in memory regulation. Confidence is central in our decisions to withhold or share an answer, as well as whether others will believe us. Calibration -- how well your level of judgment matches your level of performance.

Factors Influencing Retrospective Confidence

The Overconfidence Effect and The Hard-Easy Effect
People tend to be overconfident when challenging tests, and underconfident with easy ones. May be a systematic cognitive bias, but another theory is that experimental design is to blame; psychologists sometimes trick people.

Debiasing Techniques in Retrospective Confidence Judgments

Some techniques are process-oriented -- how the person judging will process and represent the relevant information. Other techniques are response-oriented -- informing people of the level to which others are overconfident. Response-oriented approaches may not be advisable since the judge is not pushed to understand the structure of the problem, source of the bias, etc. Process-oriented Technique: Give a reason for your answer.
Hypothesis: due to personal bias, participants are likely to be even more overconfident as they confirm their own answer.
Result: generating reasons did not impact overconfidence. But generating reasons they might be wrong did reduce overconfidence. But other studies have contradicted this finding.
Bottom line: giving reasons or counter reasons doesn't seem to be enough to consistently help people to do better.

Response-oriented Technique: Give participants feedback about their overconfidence.
Performance feedback -- overall accuracy feedback on a series **study of this showed almost immediate impact, taking overconfidence down to almost zero. Maybe this is the research basis on the benefit of the 'practice exam'?

Depressive Realism Hypothesis
Some studies show less overconfidence in people who are depressed. Certainly depression lowers overall performance. The depressive realism hypothsis posits that when people are depressed, they're more realistic; non-depressives are overconfident and unrealistic. Seems to be true that many people are overconfident. A competing hypothesis is that they're simply more negative, not more realistic -- they're selectively processing negative information.

Non-depressed and depressed people seem to have similar performance on memory tasks, but still judged selves poorly, seeming to validate the selective processing theory.

Confidence of Groups We know that RC when playing trivia alone can reveal substantial overconfidence. Are "two heads better than one" in this case? Groups perform better, although perhaps only slightly better. Confidence increases with groups, whether correct or incorrect -- even when students are trained in metamemory, confidence, and calibration. Ultimate conclusion: groupthink may improve performance, as well as inspiring undue confidence.

Theories about Retrospective Confidence

Why so overconfident? Pessimist: people are just bad at judging their internal state (Tversky and Kahneman 1974 - people use poor heuristics). Optimist: experiments are set up in ways that make people look bad (Gigerenzer - probabalistic mental model; experimenters use tricky questions).

Pessimists People don't always use probability correctly (ex: judging 'Linda' with two traits instead of one, when likelihood of two traits is likelihood of X * likelihood of Y) T & K found this 'conjunction fallacy' in numerous domains, and even experts make this error in their own domain. T & K say that people are led astray by a 'representative heuristic' -- characterizing results based on a population or pattern rather than 'playing the numbers'. Another faulty heuristic is the availability heuristic. This is when you judge something more probable because you can retrieve instances of it. Often accurate, but can produce bias due to the ease of recollecting something, or on your own experiences. Another faulty heuristic is the anchoring-and-adjustment heuristic. This is when you judge likelihood by beginning at an initial value and adjusting -- can be biased by the introduction of some figure (Now how much would you pay??) This is present even when the anchoring value is truly random, not just fixed at some manipulative point! People judge that the anchor they're being given must not be outrageous; surely the experimenter isn't trying to make something too easy or too difficult.

Optimists Judging probability -- ex: you're judging on a 100-point scale, but the question is a yes/no. People may be more sensitive to frequency of past events than absolute performance. Optimists claim that this is part of evolution -- people tended to have access primarily to their experiences, not the average experiences of all people, so tend to focus on that information. Optimists who re-ran the "Linda" experiment above but used frequency language instead of probability language, and fewer people were tripped up by the fallacy. But attempts to apply this finding more broadly have been inconclusive, so this is still an unknown.

Intuitively, the probabalistic mental model seems to explain low accuracy of RC judgements. Idea is that either you have the knowledge and feel quite confident about it, or you must infer an answer. If you must infer it, you may pick a valid/useful cue, or you might not; the cue you pick might be quite good but just happen to be wrong in this case. Example good heuristics for relative city size ("does the city have a basketball team") -- might be no good for "tricky pairs" as chosen by the experimenter. If the pairs were random, your heuristic would be good. Seems like this should be testable via real-life field conditions rather than in an experiment, if a real-life situation can be found that cleanly tests for this....

Key hypothesis: representative questions should show excellent calibration -- and this seems to be true.Oh good, optimists and pessimists are both right! Let's synthesize!

A hybrid approach

Humans judge their accuracy well if the questions reflect their real-world experiences, but are sometimes biased. Dougherty's model of 2001 integrates these approaches, combining both optimistic and pessimistic factors as well as incorporating a formal model of memory.

Relationship between RC and people's knowledge about metacognition

From Brewer et al - theory is that when making an RC judgment, people use both their knowledge and their metamemory beliefs. Hypothesis to test -- if people recall the episode in which they learned something, they're more confident in the item. Example study -- ask people to recall if a sentence is identical to one they studied. Experimenters used a "deceptive lure" where the meaning of the sentence was unchanged but a single word was changed out for a synonym. Result: recalling the study episode increased confidence even if you were deceived -- more confidence, but not more accuracy.

Convergence and Divergence in Thinking About Confidence

Debate continues as to whether it's more accurate to use heuristic-based or ecological approaches to assess human judgement. Ecological approach assumes some heuristic use. However, debate continues as to whether the issue is indeed with people using bad heuristics or bad experimental design.

Function of Retrospective Judgements

Ex: test with short-answer and multiple-choice questions. How do you decide on your answer, and whether to answer, and whether to change an answer? RC helps with this. In studies, people often are asked to recall all they can, which weights volume -- but in a jury trial, witnesses are told to tell the whole truth and nothing but the truth -- both volume and accuracy are weighted equally.