Choice-supportive bias
Choice-supportive bias or post-purchase rationalization is the tendency to retroactively ascribe positive attributes to an option one has selected and/or to demote the forgone options.[1] It is part of cognitive science, and is a distinct cognitive bias that occurs once a decision is made. For example, if a person chooses option A instead of option B, they are likely to ignore or downplay the faults of option A while amplifying or ascribing new negative faults to option B. Conversely, they are also likely to notice and amplify the advantages of option A and not notice or de-emphasize those of option B.
What is remembered about a decision can be as important as the decision itself, especially in determining how much regret or satisfaction one experiences.[2] Research indicates that the process of making and remembering choices yields memories that tend to be distorted in predictable ways.[2]
In cognitive science, one predictable way that memories of choice options are distorted is that positive aspects tend to be remembered as part of the chosen option, whether or not they originally were part of that option, and negative aspects tend to be remembered as part of rejected options.[2] Once an action has been taken, the ways in which we evaluate the effectiveness of what we did may be biased.[3] It is believed this may influence our future decision-making. These biases may be stored as memories, which are attributions that we make about our mental experiences based on their subjective qualities, our prior knowledge and beliefs, our motives and goals, and the social context. True and false memories arise by the same mechanism because when the brain processes and stores information, it cannot tell the difference where they came from.[4]
General definition
The tendency to remember one's choices as better than they actually were. In this respect, people tend to over attribute positive features to options they chose and negative features to options not chosen.[2]
Theory
Experiments in cognitive science and social psychology have revealed a wide variety of biases in areas such as statistical reasoning, social attribution, and memory.[3]
Choice-supportive memory distortion is thought to occur during the time of memory retrieval and was the result of the belief that, "I chose this option, therefore it must have been the better option."[5] Essentially, after a choice is made people tend to adjust their attitudes to be consistent with, the decision they have already made.[6][2][7] It is also possible that choice-supportive memories arise because an individual is only paying attention to certain pieces of information when making a decision or to post-choice cognitive dissonance.[5] In addition, biases can also arise because they are closely related to the high level cognitive operations and complex social interactions.[8]
Memory distortions may sometimes serve a purpose because it may be in our interest to not remember some details of an event or to forget others altogether.[9]
Making a selection
The objective of a choice is generally to pick the best option. Thus, after making a choice, a person is likely to maintain the belief that the chosen option was better than the options rejected. Every choice has an upside and a downside. The process of making a decision mostly relies upon previous experiences. Therefore, a person will remember not only the decision made but also the reasoning behind making that decision.
Motivation
Motivation may also play a role in this process because when a person remembers the option that they chose as being the best option, it should help reduce regret about their choice. This may represent a positive illusion that promotes well-being.
Cases when the individual is not in control
There are cases where an individual is not always in control of which options are received. People often end up with options that were not chosen but, instead were assigned by others, such as job assignments made by bosses, course instructors assigned by a registrar, or vacation spots selected by other family members.[10] However, being assigned (random or not) to an option leads to a different set of cognitions and memory attributions that tend to favor the alternative (non-received) option and may emphasize regret and disappointment.[11]
- Assigned options: Making a choice or having a choice made for you by other people in your best interest can prompt memory attributions that support that choice. Current experiments show no choice-supportive memory bias for assigned options.[10] However, choices which are made on a person's behalf in their best interest do show a tendency for choice-supportive memory bias.
- Random selection: People do not show choice-supportive biases when choices are made randomly for them.[12] This is because choice-supportive memory bias tends to arise during the act of making the decision.
How different forms of misremembering create different types of choice-supportive biases
Misattribution
Misattribution is a well-known commission error supported by researchers.[1][13] It results in a type of choice-supportive bias when information is attributed to the wrong source.[1] Consequently, the positive attributes of the forgone option are remembered as the positive attributes of the chosen option, or the negative attributes of the chosen option are remembered as the negative attributes of the foregone option. For example, if one had to choose between two pairs of trainers and the chosen pair fitted slightly tighter and the forgone option fitted perfectly, the chosen pair would be remembered as fitting perfectly whereas the forgone pair would be remembered as being slightly tighter (although this was not the case in reality)
While misattribution presupposes correct encoding and recall of the information in relation to a person's decision, the source of the information remains unclear or incorrect. Therefore it is not to be mistaken with completely false information.[1]
Fact distortion
Fact distortion results in a type of choice-supportive bias when the facts belonging to the chosen option are remembered in a distorted manner.[1] The distortion refers to the objective values and/or features of the chosen option being misremembered in a more preferential way to their actual values. Alternatively, the forgone option can be misremembered as being significantly less preferential then their actual values.[14] An example of fact distortion would be if you have to choose between buying one out of two cars which can both drive at a maximum speed of 130 mph, the foregone car would be remembered with a maximum speed of 100 mph, whereas the chosen car will be remembered with a maximum speed of 160 mph. Consequently, the facts concerning both cars have been distorted.
There are varieties of fact distortion that remain contested throughout the literature; as a result, there is no consensus on how it works. Some authors argue that the distortion mainly occurs when the favoured option is misremembered more preferentially, whereas other researchers argue that memory distortion does not occur or is only likely to take place during the post-decision stage.[15] Overall, some studies have argued that holding the belief that distortion cannot take place (as soon as the decision is made) means that facts cannot be distorted.[1]
Selective forgetting
Selective forgetting results in a form of choice-supportive bias when information is selectively forgotten.[1] In this respect, the positive attributes of the chosen option and the negative attributes of the forgone option are retained in memory at a higher rate and the alternatives are displaced from memory at a faster rate. An example of selective forgetting would be correctly remembering that your chosen pair of trainers were aesthetically pleasing, but forgetting that they were slightly tight. This omission error may be related to the limited capacity of memory overtime, otherwise known as transience.[8]
False memory
False memory in the context of choice-supportive biases is when items that were not part of the original decision are remembered as being presented. If these entirely new items are positive, they will be remembered as belonging to the chosen option and if they are negative, they will be remembered as belonging to the forgone option.[1] For example, a chosen pair of shoes might be remembered as good for running, although there was no information presented in respect to the shoes running capabilities.
This type of error is fundamentally different to the other types of misremembering in choice-supportive bias because it is not due to correct encoding and later confusion, but it is due to a completely false memory. This type of bias means that falsely remembered events can affect future attitudes and possibly decision-making.[16]
Potential influential factors
Alignability
Alignability in the context of choice-supportive bias refers to whether both options have directly comparable features. For example, an alignable characteristic could be whether both options can be measured in centimetres. In the context of decision making, alignability can influence choices or the ability to make a similarity judgement of a potential option.[17] The alignment process enables a person to draw similarities and difference which impact their choice-supportive biases. Research to support this can be displayed by the following example: when given a choice between two brands of popcorn, participants were more likely to choose the one with the superior alignable differences, such as “pops in its own bag” compared with “requires a microwaveable bowl” than the one with superior non-alignable differences, such as “not likely to burn” compared with those containing “some citric acid"[17][18]
Delay
The extent of the delay between encoding is a potential factor that could affect choice-supportive bias. If there is a larger delay between encoding (i.e. viewing the information about the options) and retrieval (i.e. memory tests) it is likely to result in more biased choices rather than the impact of the actual choice on choice-supportive bias.[1] Some studies have found that the extent of false memories increases over time. Whereas other researchers have shown that a 2-day delay between making choices and assessment of memory resulted in reasonably high (86%) recognition accuracy.[2][12] Therefore, these findings indicate that the influence of delays on choice-supportive bias remain varied, and the influence of delays could affect different types of memory distortions differently.[1]
Beliefs about the choices one has made
This factor refers to a person's perceived decisions concerning the choices they made, more specifically this includes memories that have been falsified to reflect a selected choice that the person did not actually make. Research illustrates that people favour the options they think they have chosen and remember the attributes of their "chosen choice" more vividly and favourably.[12] Essentially, this influences assesses how one believes the choices they made affects their subsequent memories.[12] As a result, people's memories are biased in favour of the option they thought they had selected rather than their actual choices.[12]
Individual differences
Individual differences in choice-supportive bias affect the way a person remembers their options and the way they make decisions and therefore it may influence the degree to which a person engages in choice-supportive bias. Factors such as age and individual characteristics can influence an individual's cognitive abilities, personality and thus their overall choice-supportive biases.[1] For example, it has been observed by correlations that people with better performance in tests of frontal or executive functioning were less prone to choice-supportive memory.[2]
Relation to self
People's conception of who they are, can be shaped by the memories of the choices they make; the college favored over the one renounced, the job chosen over the one rejected, the candidate elected instead of another one not selected.[19] Memories of chosen as well as forgone alternatives can affect one's sense of well-being. Regret for options not taken can cast a shadow, whereas satisfaction at having made the right choice can make a good outcome seem even better.[19]
Positive illusions
Choice-supportive bias often results in memories that depict the self in an overly favorable light. In general, cognitive biases loosen our grasp on reality because the line between reality and fantasy can become fuzzy if one's brain has failed to remember a particular event.[8] Positive illusions are generally mild and are important contributors to our sense of well being. However, we all need to be aware that they do exist as part of human nature.[8]
Memory storage
Human beings are blessed with having an intelligent and complex mind, which allows us to remember our past, be able to optimize the present, and plan for the future. Remembering involves a complex interaction between the current environment, what one expects to remember, and what is retained from the past.[8] The mechanisms of the brain that allow memory storage and retrieval serves us well most of the time, but occasionally gets us into trouble.
Memories change over time
There is now abundant evidence that memory content can undergo systematic changes. After some period of time and if the memory is not used often, it may become forgotten.
- Memory retention: It is recognized that retention is best for experiences that are pleasant, intermediate for experiences that are unpleasant, and worst for experiences that are neutral. Generic memories provide the basis for inferences that can bring about distortions. These distortions in memory do not displace an individual's specific memories, but they supplement and fill in the gaps when the memories are lost.[20] It has been shown that a wide variety of strategic and systematic processes are used to activate different areas of the brain in order to retrieve information.
- Credibility of a memory: People have a way to self-check memories, in which a person may consider the plausibility of the retrieved memory by asking themselves is this event even possible.[4] For example, if a person remembers seeing a pig fly, they must conclude that it was from a dream because pigs cannot fly in the real world. Memory does not provide people with perfect reproductions of what happened, it only consists of constructions and reconstructions of what happened.[4]
Brain areas of interest
There is extensive evidence that the amygdala is involved in effectively influencing memory.[21] Emotional arousal, usually fear based, activates the amygdala and results in the modulation of memory storage occurring in other brain regions. The forebrain is one of the targets of the amygdala. The forebrain receives input from amygdala and calculates the emotional significance of the stimulus, generates an emotional response, and transmits it to cerebral cortex. This can alter the way neurons respond to future input, and therefore cognitive biases, such as choice-supportive bias can influence future decisions.
Stress hormones affect memory
Effects of stress-related hormones, such as epinephrine and glucocorticoids are mediated by influences involving the amygdala.[21] It has been shown in experiments with rats that when they are given systemic injections of epinephrine while being trained to perform a task, they show an enhanced memory of performing the task. In effect the stronger the emotion that is tied to the memory, the more likely the individual is to remember. Therefore, if the memory is stored and retrieved properly it is less likely to be distorted.
Brain mapping
A PET scan or fMRI can be used to identify different regions of the brain that are activated during specific memory retrieval.
fMRI study
- True versus false memories: One study asked subjects to remember a series of events while being monitored by an fMRI to see which areas "light up". When an individual remembered a greater number of true memories than false memories, it showed a cluster spanning the right superior temporal gyrus and lateral occipital cortex. However, when the reverse occurred (when an individual remembered a greater number of false memories than true) the brain area that showed activation was the left insula.[22] These findings may provide some insight as to which areas of the brain are involved with storing memories and later retrieving them.
Influence of age
Studies now show that as people age, their process of memory retrieval changes. Although general memory problems are common to everyone because no memory is perfectly accurate, older adults are more likely than younger adults to show choice-supportive biases.
Aging of the brain
Normal aging may be accompanied by neuropathy in the frontal brain regions. Frontal regions help people encode or use specific memorial attributes to make source judgments, controls personality and the ability to plan for events. These areas can attribute to memory distortions and regulating emotion.
Regulation of emotion
In general, older adults are more likely to remember emotional aspects of situations than are younger adults. For example, on a memory characteristic questionnaire, older adults rated remembered events as having more associated thoughts and feelings than did younger adults. As a person ages, regulating personal emotion becomes a higher priority, whereas knowledge acquisition becomes less of a powerful motive. Therefore, choice-supportive bias would arise because their focus was on how they felt about the choice rather than on the factual details of the options. Studies have shown that when younger adults are encouraged to remember the emotional aspect of a choice, they are more likely to show choice-supportive bias. This may be related to older adults' greater tendency to show a positivity effect in memory.
Rely on familiarity
Older adults rely more than younger adults on categorical or general knowledge about an event to recognize particular elements from the event.[2] Older adults are also less likely to correctly remember contextual features of events, such as their color or location. This may be because older adults remember (or rely on) fewer source identifying characteristics than the young. Consequently, older adults must more often guess or base a response on less specific information, such as familiarity.[23] As a result, if they can't remember something, they are more likely to fill in the missing gaps with things that are familiar to them.[8]
Getting the 'gist'
Older adults are more reliant on gist-based retrieval. A number of studies suggest that using stereotypes or general knowledge to help remember an event is less cognitively demanding than relying on other types of memorial information and thus might require less reflective activity. This shift towards gist-based processes might occur as a compensation for age decrements in verbatim memory.[24]
Inhibition
The episodic memory and inhibition accounts of age-related increases in false memories. Inhibition of a memory may be related to an individual's hearing capacity and attention span. If the person cannot hear what is going on around them or is not paying much attention, the memory cannot be properly stored and therefore cannot be accurately retrieved.
Examples
Deciding between two used cars
Henkel and Mather tested the role of beliefs at the time of retrieval about which option was chosen by giving participants several hypothetical choices like deciding between two used cars.[5] After making several choices, participants left and were asked to return a week later. At that point, Henkel and Mather reminded them which option they had chosen for each choice and gave them a list of the features of the two options; some new positive and negative features were mixed in with the old features. Next, participants were asked to indicate whether each option was new, had been associated with the option they chose, or had been associated with the option they rejected. Participants favored whichever option Henkel and Mather had told them they had chosen in their memories.[5] These findings show that beliefs at the time of retrieval about which option was chosen shape both which features are attributed to the options and how vividly they are remembered.
Remembering high school grades
One study looked at the accuracy and distortion in memory for high school grades. The relation between accuracy and distortion of autobiographical memory content was examined by verifying 3,220 high school grades recalled by 99 freshman college students.[20] It was shown that most errors inflated the actual high school grade, meaning that these distortions are attributed to memory reconstructions in a positive and emotionally gratifying direction. In addition, their findings indicate that the process of distortion does not cause the actual unpleasant memory loss of getting the bad grade.[20] This is because there was no correlation found between the percentage of accuracy recall and the degree of asymmetry, or distortion. This shows that the distortion in memories of high school grades arises after the content has been forgotten by another mechanism.
A 50 year study of college grades
Many similar studies have been performed, such as a fifty-year study of remembering college grades. In this study one to 54 years after graduating, 276 alumni correctly recalled 3,025 of 3,967 college grades. The number of omission errors increased with the retention interval and better students made fewer errors.[25] The accuracy of recall increased with confidence in recall. Eighty-one percent of errors of commission inflated the actual grade. These data suggested that distortions occur soon after graduation, remain constant during the retention interval, and are greater for better students and for courses students enjoyed most. Therefore, sometime in between when the memory is stored and when it is retrieved some time later, the distortion may arise.[25]
Methods for testing
Written scenario memory tests
Researchers have used written scenarios in which participants are asked to make a choice between two options. Later, on a memory test, participants are given a list of positive and negative features, some of which were in the scenario and some of which are new. A choice-supportive bias is seen when both correct and incorrect attributions tend to favor the chosen option, with positive features more likely to be attributed to the chosen option and negative features to the rejected option.
- Deception: Henkel and Mather (2007) found that giving people false reminders about which option they chose in a previous experiment session led people to remember the option they were told they had chosen as being better than the other option. This reveals that choice-supportive biases arise in large part when remembering past choices, rather than being the result of biased processing at the time of the choice.
Deese/Roediger–McDermott paradigm
The Deese–Roediger–McDermott paradigm (DRM) consists of a participant listening to an experimenter read lists of thematically related words (e.g. table, couch, lamp, desk); then after some period of time the experimenter will ask if a word was presented in the list. Participants often report that related but non-presented words (e.g. chair) were included in the encoding series, essentially suggesting that they 'heard' the experimenter say these non-presented words (or critical lures). Incorrect 'yes' responses to critical lures, often referred to as false memories, are remarkably high under standard DRM conditions.[26]
Relation to cognitive dissonance
The theory of cognitive dissonance proposes that people have a motivational drive to reduce dissonance. Choice-supportive bias is potentially related to the aspect of cognitive dissonance explored by Jack Brehm (1956) as postdecisional dissonance. Within the context of cognitive dissonance, choice-supportive bias would be seen as reducing the conflict between "I prefer X" and "I have committed to Y".
Debiasing
A study of the Lady Macbeth effect showed reduced choice-supportive bias by having participants engage in washing.[27] However, the underlying general effect has not replicated in larger studies[28][29]
See also
References
- Lind, Martina; Visentini, Mimì; Mäntylä, Timo; Del Missier, Fabio (2017-12-04). "Choice-Supportive Misremembering: A New Taxonomy and Review". Frontiers in Psychology. 8: 2062. doi:10.3389/fpsyg.2017.02062. ISSN 1664-1078. PMC 5723021. PMID 29255436.
- Mather, M.; Johnson, M.K. (2000). "Choice-supportive source monitoring: Do our decisions seem better to us as we age?" (PDF). Psychology and Aging. 15 (4): 596–606. doi:10.1037/0882-7974.15.4.596. PMID 11144319. Archived from the original (PDF) on 2016-07-06.
- "But That's Crazy! Cognitive Bias in Decision-making | Duncan Pierce." Duncan Pierce | Duncanpierce.org. Web. 18 Sept. 2010. <http://duncanpierce.org/cognitive_bias_workshop Archived 2014-02-22 at the Wayback Machine>.
- Johnson, Marcia K. (2006). "Memory and reality" (PDF). American Psychologist. 61 (8): 760–771. doi:10.1037/0003-066X.61.8.760. PMID 17115808. Archived from the original on 2010-08-20.
{{cite journal}}
: CS1 maint: bot: original URL status unknown (link) - "Memory Distortion in Decision Making". University of Southern California. Archived from the original on 2015-03-11.
- Hornsby, A.N.; Love, B.C. (2020). "How decisions and the desire for coherency shape subjective preferences over time". Cognition. 200 (1): 104244. doi:10.1016/j.cognition.2020.104244. PMC 7315129. PMID 32222615.
- Festinger, Leon, 1919-1989 (1962). A theory of cognitive dissonance. Stanford, California. ISBN 0804709114. OCLC 921356.
{{cite book}}
: CS1 maint: location missing publisher (link) CS1 maint: multiple names: authors list (link) - Schacter, Daniel L. (2002). The Seven Sins of Memory: How the Mind Forgets and Remembers. Boston: Houghton Mifflin Harcourt. ISBN 9780547347455.
- Gordon, Ruthanna; Franklin, Nancy; Beck, Jennifer (2005). "Wishful Thinking and Source Monitoring". Memory & Cognition. 33 (3): 418–29. doi:10.3758/BF03193060. PMID 16156178.
- Mather, M.; Shafir, E.; Johnson, M.K. (2003). "Remembering chosen and assigned options" (PDF). Memory & Cognition. 31 (3): 422–433. doi:10.3758/BF03194400. PMID 12795484. Archived from the original (PDF) on 2012-07-13.
- Stoll Benney, Kristen; Henkel, Linda (2006). "The Role of Free Choice in Memory for past Decisions". Memory. 14 (8): 1001–011. CiteSeerX 10.1.1.1027.9662. doi:10.1080/09658210601046163. PMID 17077034. S2CID 6021663.
- Henkel, L.A.; Mather, M. (2007). "Memory attributions for choices: How beliefs shape our memories" (PDF). Journal of Memory and Language. 57 (2): 163–176. doi:10.1016/j.jml.2006.08.012. Archived from the original (PDF) on 2016-03-04.
- Schacter, Daniel L. (1999). "The seven sins of memory: Insights from psychology and cognitive neuroscience". American Psychologist. 54 (3): 182–203. doi:10.1037/0003-066X.54.3.182. ISSN 1935-990X. PMID 10199218. S2CID 14882268.
- DeKay, Michael L. (2015). "Predecisional Information Distortion and the Self-Fulfilling Prophecy of Early Preferences in Choice". Current Directions in Psychological Science. 24 (5): 405–411. doi:10.1177/0963721415587876. ISSN 0963-7214. S2CID 146237009.
- Svenson, Ola (2009). "Post-decision consolidation and distortion of facts". Judgment and Decision Making. 4 (5): 397–407. doi:10.1017/S1930297500001224. hdl:1794/22047. S2CID 18889316.
- Laney, Cara; Morris, Erin K.; Bernstein, Daniel M.; Wakefield, Briana M.; Loftus, Elizabeth F. (2008). "Asparagus, a Love Story: Healthier Eating Could Be Just a False Memory Away". Experimental Psychology. 55 (5): 291–300. doi:10.1027/1618-3169.55.5.291. ISSN 1618-3169. PMID 25116296.
- Mather, Mara; Knight, Marisa; McCaffrey, Michael (2005). "The Allure of the Alignable: Younger and Older Adults' False Memories of Choice Features" (PDF). Journal of Experimental Psychology: General. 134 (1): 38–51. doi:10.1037/0096-3445.134.1.38. PMID 15702962. Archived from the original (PDF) on 2014-02-26.
- Zhang, Shi; Markman, Arthur B. (2001-07-01). "Processing Product Unique Features: Alignability and Involvement in Preference Construction". Journal of Consumer Psychology. 11 (1): 13–27. doi:10.1207/15327660152054012. ISSN 1532-7663.
- Mather, M.; Shafir, E.; Johnson, M.K. (2000). "Misrememberance of options past: Source monitoring and choice" (PDF). Psychological Science. 11 (2): 132–138. doi:10.1111/1467-9280.00228. PMID 11273420. S2CID 2468289. Archived (PDF) from the original on 2012-10-03. Retrieved 2020-07-20.
- Bahrick, Harry P.; Hall, Lynda K.; Berger, Stephanie A. (1996). "Accuracy and Distortion in Memory for High School Grades". Psychological Science. 7 (5): 265–71. doi:10.1111/j.1467-9280.1996.tb00372.x. S2CID 143801485.
- McGaugh, James L.; Cahill, Larry; Roozendaal, Benno (1996). "Involvement of the Amygdala in Memory Storage: Interaction with Other Brain Systems". Proceedings of the National Academy of Sciences. 93 (24): 13508–3514. Bibcode:1996PNAS...9313508M. doi:10.1073/pnas.93.24.13508. PMC 33638. PMID 8942964.
- Baym, C. L.; Gonsalves, B. D. (2010). "Comparison of neural activity that leads to true memories, false memories, and forgetting: An fMRI study of the misinformation effect". Cogn Affect Behav Neurosci. 10 (3): 339–348. doi:10.3758/CABN.10.3.339. PMID 20805535.
- Dodson, C.; Bawa, S.; Slotnick, S. (2007). "Aging, source memory, and misrecollections". Journal of Experimental Psychology: Learning, Memory, and Cognition. 33 (1): 169–181. doi:10.1037/0278-7393.33.1.169. PMID 17201560. S2CID 16475820.
- Lovden, M. (2003). "The episodic memory and inhibition accounts of age-related increases in false memories: A consistency check". Journal of Memory and Language. 49 (2): 268–283. doi:10.1016/S0749-596X(03)00069-X.
- Bahrick, H. P.; Hall, L. K.; Da Costa, L. A. (2008). "Fifty years of memory of college grades: Accuracy and distortions". Emotion. 8 (1): 13–22. doi:10.1037/1528-3542.8.1.13. PMID 18266512.
- Foley, M.; Hughes, K.; Librot, H.; Paysnick, A. (2009). "Imagery Encoding Effects on Memory in the DRM Paradigm: A Test of Competing Predictions". Applied Cognitive Psychology. 23 (6): 828–848. doi:10.1002/acp.1516.
- Lee, Spike W. S.; Schwarz, Norbert (2010). "Washing away postdecisional dissonance". Science. 328 (5979): 709. Bibcode:2010Sci...328..709L. doi:10.1126/science.1186799. PMID 20448177. S2CID 18611420.
- Fayard, Jennifer; et al. (2009). "Is cleanliness next to godliness? Dispelling old wives' tales: Failure to replicate Zhong and Liljenquist (2006)" (PDF). Journal of Articles in Support of the Null Hypothesis. 6: 21–30. CiteSeerX 10.1.1.214.2427.
- Earp, Brian D.; Everett, Jim A. C.; Madva, Elizabeth N.; Hamlin, J. Kiley (2014). "Out, Damned Spot: Can the "Macbeth Effect" be Replicated?". Basic and Applied Social Psychology. 36: 91–98. doi:10.1080/01973533.2013.856792. S2CID 51472032.